Money Talks: Analyzing Financial Market Data
Submitted by Deepak Shenoy (@deepakshenoy) on Tuesday, 26 March 2013
Section: Analytics and Visualization Technical level: Intermediate
Financial markets produce a ton of data, but how can we look at them in useful ways, as compared to "looks-great-what-do-I-do-now". By useful I mean to traders, to fraud-detectors, to investors and even to company management. Learn about the techniques of market data analysis from someone who's done all the wrong things, sometimes in spectacular fashion.
Financial markets are action packed, fast paced dens of excitement constantly creating new stories and burying past ones, but what they produce is, essentially, big data. Every trade is out there somewhere, data is provided - free or at a cost - on how prices have moved, how companies have fared, how insiders have traded and how governments have reacted.
What I'm going to speak about: 1) What kind of "raw" information exists: Prices, volumes, futures, options, bonds, currencies, commodities - the list is endless. Much of this is available around the clock, some of these can be traded from anywhere, and a significant percentage is traded purely through algorithms today.
2) Analyzing the base information: From creating indexes, to "trading pairs" to "option strategies", the idea of a big data strategy for markets is to create derivative data that is tradeable or provides outlier information that is required for action.
3) The sucker theory: It's easy to get all mathematical and assume that if we take any of these variables and find "correlations" we can automatically find patterns that will provide great profits in the future. Much about correlation is accidental, like the writing of shakespeare by enough monkeys with typewriters; in financial markets, correlation is even sometimes created to encourage suckers. I'll speak of the pitfalls of such analysis.
4) It's often evident that fraud or manipulation is evident in market data; even in the Enron case or in the housing downturn in the US, the data actually revealed problems way before everyone else realized what was happening. A quick take on understanding what can be done with big data to weed out market fraud.
I'll have a lot of time for Q&A, but if you want to speak on something specific, please post a comment.
You should know something about financial markets like buying or selling of shares. It would be useful to have a basic idea of time-series analysis, like moving averages or statistical outliers, but it's not mandatory as the talk will be in very simple language.
I write about money, markets and trading at Capital Mind (capitalmind.in) and actively trade and educate people about financial markets. I am part of a new startup that will revolutionize analytics in macro-economics and bond markets in India. As a former geek, I have some experience in programming and plan to demonstrate how much of a fossil I now am. In my spare time, I like to climb stationary walls.