System and method for forecasting fluctuations in future data and particularly for forecasting security prices by news analysis

Abstract

A system and method for predicting price fluctuations in financial markets. Our approach utilizes both market history and public news articles, published before the beginning of trading each day, to produce a set of recommended investment actions. We empirically show that these markets are surprisingly predictable, even by purely market-historical techniques. Furthermore, analyzing relevant news articles captures information features independent of the markets history, and combining the two methods significantly improves results. Capturing usable features from news articles requires some linguistic sophistication the standard naïve bag-f-words approach does not yield predictive features. Instead, we use part-of-speech tagging, dependency parsing and semantic role labeling to generate features that improve system accuracy. We evaluate our system on eight political prediction markets from 2004 and show that we can make effective investment decisions based on our systems predictions, whose profits greatly exceed those generated by a baseline system.

Dependencies are inferred from the claim text.