Evaluating sentiment in financial news articles: Working paper series--11-10

We investigate the pairing of a financial news article prediction system, AZFinText, with sentiment analysis techniques. From our comparisons we found that news articles of a subjective nature were easier to predict in both price direction (59.0% vs 50.4% without sentiment) and through a simple trading engine (3.30% return vs 2.41% without sentiment). Looking into sentiment further, we found that news articles of a negative sentiment were easiest to predict in both price direction (50.9% vs 50.4% without sentiment) and our simple trading engine (3.04% return vs 2.41% without sentiment). Investigating the negative sentiment further, we found that AZFinText was best able to predict price decreases in articles of a positive sentiment (53.5%) and price increases in articles of a negative or neutral sentiment (52.4% and 49.5% respectively)

Web Mining For Financial Market Prediction Based On Online Sentiments

Financial market prediction is a critically important research topic in financial data mining because of its potential commerce application and attractive profits. Previous studies in financial market prediction mainly focus on financial and economic indicators. Web information, as an information repository, has been used in customer relationship management and recommendation, but it is rarely considered to be useful in financial market prediction. In this paper, a combined web mining and sentiment analysis method is proposed to forecast financial markets using web information. In the proposed method, a spider is firstly employed to crawl tweets from Twitter. Secondly, Opinion Finder is offered to mining the online sentiments hidden in tweets. Thirdly, some new sentiment indicators are suggested and a stochastic time effective function (STEF) is introduced to integrate everyday sentiments. Fourthly, support vector regressions (SVRs) are used to model the relationship between online sentiments and financial market prices. Finally, the selective model can be serviced for financial market prediction. To validate the proposed method, Standard and Poor’s 500 Index (S&P 500) is used for evaluation. The empirical results show that our proposed forecasting method outperforms the traditional forecasting methods, and meanwhile, the proposed method can also capture individual behavior in financial market quickly and easily. These findings imply that the proposed method is a promising approach for financial market prediction

Stock price change prediction using news text mining

Along with the advent of the Internet as a new way of propagating news in a digital format, came the need to understand and transform this data into information. This work presents a computational framework that aims to predict the changes of stock prices along the day, given the occurrence of news articles related to the companies listed in the Down Jones Index. For this task, an automated process that gathers, cleans, labels, classifies, and simulates investments was developed. This process integrates the existing data mining and text algorithms, with the proposal of new techniques of alignment between news articles and stock prices, pre-processing, and classifier ensemble. The result of experiments in terms of classification measures and the Cumulative Return obtained through investment simulation outperformed the other results found after an extensive review in the related literature. This work also argues that the classification measure of Accuracy and incorrect use of cross validation technique have too few to contribute in terms of investment recommendation for financial market. Altogether, the developed methodology and results contribute with the state of art in this emerging research field, demonstrating that the correct use of text mining techniques is an applicable alternative to predict stock price movements in the financial market.Com o advento da Internet como um meio de propagação de notícias em formato digital, veio a necessidade de entender e transformar esses dados em informação. Este trabalho tem como objetivo apresentar um processo computacional para predição de preços de ações ao longo do dia, dada a ocorrência de notícias relacionadas às companhias listadas no índice Down Jones. Para esta tarefa, um processo automatizado que coleta, limpa, rotula, classifica e simula investimentos foi desenvolvido. Este processo integra algoritmos de mineração de dados e textos já existentes, com novas técnicas de alinhamento entre notícias e preços de ações, pré-processamento, e assembleia de classificadores. Os resultados dos experimentos em termos de medidas de classificação e o retorno acumulado obtido através de simulação de investimentos foram maiores do que outros resultados encontrados após uma extensa revisão da literatura. Este trabalho também discute que a acurácia como medida de classificação, e a incorreta utilização da técnica de validação cruzada, têm muito pouco a contribuir em termos de recomendação de investimentos no mercado financeiro. Ao todo, a metodologia desenvolvida e resultados contribuem com o estado da arte nesta área de pesquisa emergente, demonstrando que o uso correto de técnicas de mineração de dados e texto é uma alternativa aplicável para a predição de movimentos no mercado financeiro

The Futility of Utility: how market dynamics marginalize Adam Smith

Econometrics is based on the nonempiric notion of utility. Prices, dynamics, and market equilibria are supposed to be derived from utility. Utility is usually treated by economists as a price potential, other times utility rates are treated as Lagrangians. Assumptions of integrability of Lagrangians and dynamics are implicitly and uncritically made. In particular, economists assume that price is the gradient of utility in equilibrium, but I show that price as the gradient of utility is an integrability condition for the Hamiltonian dynamics of an optimization problem in econometric control theory. One consequence is that, in a nonintegrable dynamical system, price cannot be expressed as a function of demand or supply variables. Another consequence is that utility maximization does not describe equiulibrium. I point out that the maximization of Gibbs entropy would describe equilibrium, if equilibrium could be achieved, but equilibrium does not describe real markets. To emphasize the inconsistency of the economists' notion of 'equilibrium', I discuss both deterministic and stochastic dynamics of excess demand and observe that Adam Smith's stabilizing hand is not to be found either in deterministic or stochastic dynamical models of markets, nor in the observed motions of asset prices. Evidence for stability of prices of assets in free markets simply has not been found.Comment: 46 pages. accepte

Using Predictive Analytics to Reduce Uncertainty in Enterprise Risk Management

Traditional economic and business forecasting about corporate credit has relied on statistics from government agencies, annual reports and financial statements. These statistics are often published with significant delay, which limits their usefulness for predicting changes in creditworthiness. Yet, a delay in responding to changes in a company’s credit rating can have significant financial and risk consequences. With the widespread adoption of search engines, social media and related information technologies, it is possible to obtain data on literally trillions of economic decisions almost the instant that they are made. In this study, we investigated the power of these online activity data combined with data on firms’ business ecosystem to predict the likelihood of counterparty credit downgrade risk. The research offers a novel approach that contributes to the fields of information systems, finance, and social science by providing new insights on the role of these data types on firms’ financial risk

Exploratory topic modeling with distributional semantics

As we continue to collect and store textual data in a multitude of domains, we are regularly confronted with material whose largely unknown thematic structure we want to uncover. With unsupervised, exploratory analysis, no prior knowledge about the content is required and highly open-ended tasks can be supported. In the past few years, probabilistic topic modeling has emerged as a popular approach to this problem. Nevertheless, the representation of the latent topics as aggregations of semi-coherent terms limits their interpretability and level of detail. This paper presents an alternative approach to topic modeling that maps topics as a network for exploration, based on distributional semantics using learned word vectors. From the granular level of terms and their semantic similarity relations global topic structures emerge as clustered regions and gradients of concepts. Moreover, the paper discusses the visual interactive representation of the topic map, which plays an important role in supporting its exploration.Comment: Conference: The Fourteenth International Symposium on Intelligent Data Analysis (IDA 2015

Machine Learning Stock Market Prediction Studies: Review and Research Directions

Stock market investment strategies are complex and rely on an evaluation of vast amounts of data. In recent years, machine learning techniques have increasingly been examined to assess whether they can improve market forecasting when compared with traditional approaches. The objective for this study is to identify directions for future machine learning stock market prediction research based upon a review of current literature. A systematic literature review methodology is used to identify relevant peer-reviewed journal articles from the past twenty years and categorize studies that have similar methods and contexts. Four categories emerge: artificial neural network studies, support vector machine studies, studies using genetic algorithms combined with other techniques, and studies using hybrid or other artificial intelligence approaches. Studies in each category are reviewed to identify common findings, unique findings, limitations, and areas that need further investigation. The final section provides overall conclusions and directions for future research