Stock portfolio selection using learning-to-rank algorithms with news sentiment

In this study, we apply learning-to-rank algorithms to design trading strategies using relative performance of a group of stocks based on investors' sentiment toward these stocks. We show that learning-to-rank algorithms are effective in producing reliable rankings of the best and the worst performing stocks based on investors' sentiment. More specifically, we use the sentiment shock and trend indicators introduced in the previous studies, and we design stock selection rules of holding long positions of the top 25% stocks and short positions of the bottom 25% stocks according to rankings produced by learning-to-rank algorithms. We then apply two learning-to-rank algorithms, ListNet and RankNet, in stock selection processes and test long-only and long-short portfolio selection strategies using 10 years of market and news sentiment data. Through backtesting of these strategies from 2006 to 2014, we demonstrate that our portfolio strategies produce risk-adjusted returns superior to the S&P500 index return, the hedge fund industry average performance - HFRIEMN, and some sentiment-based approaches without learning-to-rank algorithm during the same period

The role of investor attention in global asset price variation during the invasion of Ukraine

We study the impact of event-specific attention indices -- based on Google Trends -- in predictive price variation models before and during the Russian invasion of Ukraine in February 2022. We extend our analyses to the importance of geographical proximity and economic openness to Russia within 51 global equity markets. Our results demonstrate that 36 countries show significant attention to the conflict at the onset of and during the invasion, which helps predict volatility. We find that the impact of attention is more significant in countries with a higher degree of economic openness to Russia and those nearer to it.Comment: This paper includes 17 pages, 3 figures and 8 table

네트워크 프랙탈 성질에 기반한 금융시장 예측 및 거래전략

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 산업공학과, 2020. 8. 장우진.Extensive academic research was performed for the financial market as it is closely connected to practical economy. Research in traditional financial economics resulted in economic indicators and the econometrics was instrumental for quantitative research in financial market. However, it proved to be difficult to predict the market behavior as it is a result of complex interaction among many agents with their own agenda. An effective tool to predict a change in market would be beneficial for policy makers and market participants to assist them with rational and consistent decision making. On the other hand, inconsistent prediction would lead to a suboptimal and inconsistent market activity which sometimes result in sudden collapse in the market as it did in 2008 Financial crisis and 1997 Asian financial crisis. The purpose of this dissertation is to develop approach based on econophysics and machine learning to systematically analyze the financial market. The main focus of this dissertation involves the network structure of stock market. To predict the change in market behavior, it is critical to understand the relationship or correlation among the market participants beforehand, and complex network analysis is one of the most prominent methods for such study. The fractal theory was employed as the primary approach to analyze the network structure of financial market. The empirical study shows that the network of financial market exhibits fractal properties. Also, analysis of fractal dimension and network topology led to two key discoveries. First, the fractal dimension and the Strong effective repulsion between distinct network nodes known as the hub are closely related. Second, the fractal dimension reveals the shortcut of network structure. Through further analysis, these two properties were proved to be useful for risk management in financial market. Three fractal measures were proposed to specify network structure for ease of implementation in future studies. In the second step, the fractal measures were implemented in a financial market to assess its ability to predict the market movement. Recently, studies were conducted to determine if a new measure or index improves the prediction accuracy for financial time series. These studies are advantageous for future studies as it proposes new indices for other implementation and further analysis rather than studying the precision of their own method. In this paper, machine learning algorithms were employed to assess the predictive properties of fractal measures. Empirical experiments were performed to predict direction of market movement, which is effectively a classification task, and prediction for returns, a regression task. The studies concludes that the fractal measure proposed were effective in prediction for long-term stock returns of more than three months period. Finally, a model to improve trading strategy based on learning-to-rank algorithm and the fractal measures was introduced. Previous studies are often based on the modern portfolio theory(MPT), but it is insufficient for real-world application as it doesnt provide any implication for rebalancing period of portfolio. The optimal rebalancing model proposed in this study allows its application with traditional portfolio methods. The experiments were carried out in two steps. The model learns to predict the better time period to perform rebalancing between two time periods in the future, followed by the empirical simulation to apply the model in real world trading scenario. Two traditional portfolio methods, equal weighted and maximized Sharpe ratio, were taken for experiment. The result affirms that the optimal rebalancing model was able to capture the better time period of rebalancing portfolio. In addition, the model outperformed a simple rebalancing method of fixed time period. When the fractal measures were employed as an input variable, the model performance was further improved. The primary contribution achieved through this model is that it allows application and expansion into all traditional portfolio models. Also, the fractal measures observed in the network structure grants insight regarding the market behavior and empirically proved that the measure provides benefit in prediction for the real-world stock market.금융시장에 대한 연구는 전반적인 경제 활동과 밀접한 연관성이 있기 때문에, 다양한 학계의 지식들과 연계되어 광범위하게 연구되고 있다. 전통적인 경제학 이론을 바탕으로 여러가지 경제 지표들이 개발되었고, 계량경제학의 발전으로 이를 정량적으로 분석하는 연구가 진행되었다. 하지만, 서로 다른 특징을 갖는 시장참여자들의 행위로 이루어진 금융시장의 복잡한 특성 때문에, 기존의 경제학 기반의 방법론들만으로 금융시장의 변화를 정밀하게 예측하기에는 한계가 있었다. 만약 금융시장의 변화를 효율적으로 예측 할 수 있다면, 국가 정책이나 기업들 및 시장 참여자들은 합리적인 의사결정을 통해서 건전한 금융 활동을 할 수 있을 것이다. 반면에 이러한 금융시장의 변화를 효율적으로 예측하지 못해 비이상적인 금융 활동이 지속된다면, 최악의 경우에는 글로벌 금융 위기와 같은 대규모 시장 붕괴 현상이 발생할 수 있을 것이아. 따라서 본 학위논문에서는 경제물리학과 머신러닝을 융합하여 체계적으로 금융시장 분석을 진행하고자 한다. 본 학위논문에서는 금융시장의 다양한 섹터 중에서 주식시장 네트워크 구조를 분석하는데 초점을 맞춘다. 미래의 주식시장의 변화를 올바르게 예측하기 위해서는 주식시장 구성원들간의 관계 파악이 선행되어야 하는데, 이에 대표적인 분석 방법이 복잡계 네트워크 분석(Complex network analysis)이기 때문이다. 본 연구에서는 주식시장 네트워크 구조를 분석하는 여러 방법론들 중 프랙탈 이론(Fractal theory)의 도입을 제안한다. 실험 결과 주식시장 네트워크의 구조는 프랙탈 특성을 가짐을 밝혀냈다. 또한, 측정된 프랙탈 차원(Fractal dimension)과 네트워크의 토폴로지(Topology)와의 관계를 살펴본 결과 두 가지 주요한 주식시장 네트워크의 구조적인 특징을 발견할 수 있었다. 첫번째는, 프랙탈 차원과 소위 허브(Hub)라고 불리우는 네트워크 상에서 연결이 많이된 노드들간의 강한 반발(Strong effective repulsion) 현상과 연관성이 있다는 점이다. 두번째는, 프랙탈 차원으로 네트워크의 지름길(Shortcut) 구조를 관찰할 수 있었다. 또한 이 두 가지 네트워크의 구조적인 특성은 주식시장의 위험 관리(Risk management) 관점에서 유용하게 쓰일 수 있음을 분석했다. 그리고, 위 특성들을 다른 연구들에 쉽게 적용 가능하도록 네트워크 구조를 표현하는 3가지 프랙탈 지표(Fractal measures)들을 제안한다. 다음 단계로 주식시장에서 측정한 프랙탈 지표가 미래에 주가 지수의 예측력 향상에 도움이 되는지를 검증했다. 최근 다양한 분야에서 새롭게 발견한 지표들이 금융 시계열 데이터에 대하여 예측력 향상에 도움이 되는지를 검증하는 연구들이 진행되고있다. 이러한 연구들은 발견한 지표들 만을 사용하여 정밀한 예측을 하는 목적이 아닌, 발견한 지표들이 예측력 향상에 도움이 된다는 점을 밝혀내는데 주 목적이 있다. 이렇게 예측력 향상이 있는것이 밝혀진 지표들은 다른 연구나 산업에 쉽게 적용할 수 있는 장점이 있다. 본 학위논문에서는 몇 가지 머신러닝 알고리즘을 활용하여 측정한 프랙탈 지표가 미래의 주가 지수의 예측력 향상에 도움이 되는지를 검증했다. 검증 실험은 가장 단순한 미래 주가 지수의 방향 분류(Classification) 부터, 주가 지수 수익률의 예측(Prediction) 까지 이루어진다. 그 결과 제안한 프랙탈 지표들은 약 3개월 이후의 장기 미래의 주가 지수에 대해 일관성있는 예측력 향상 효과가 있음을 밝혀냈다. 마지막으로 제안한 프랙탈 지표들과 Learning-to-rank 알고리즘을 활용하여 기존의 주식시장에 연구되었던 거래 전략(Trading strategy)의 성능을 개선할 수 있는 모델을 제안한다. 기존의 주식시장에서 연구된 거래 전략들 중 큰 비율을 차지하는 연구들은 현대 포트폴리오 이론(Modern portfolio theory)에 기반한 포트폴리오 구성 방법에 대한 연구들이다. 하지만, 실제 투자에 적용하기 위해서는 포트폴리오를 구성하는 방법론 뿐 만 아니라, 언제 포트폴리오를 재구성해야 하는지를 판단하는것 또한 중요한 의사결정 요소이다. 본 학위논문에서 제안하는 모델은 기존에 연구된 방법론들에 유연하게 접목하여 활용할 수 있는 최적 리밸런싱 시점 판단 모델(Optimal rebalancing model)이다. 실험은 두단계로 진행된다. 먼저, 제안한 모델로 학습 데이터 내의 서로 다른 두 시점 중 미래의 더 나은 성능을 보이는 리밸런싱 지점을 예측할수 있는지를 학습한다. 그후에, 학습된 모델들 중 좋은 성능을 갖는 파라미터를 선택하고, 시뮬레이션 분석을 통해서 실제 거래전략에 적용 가능성을 평가한다. 실험에서 사용된 기존의 포트폴리오 구성 방법론은, 관련 연구들에서 가장 대표적인 벤치마크로 활용되는 자산 균등 분배 포트폴리오 방식과 샤프 비율(Sharpe ratio) 최대화 포트폴리오 방식이다. 실험결과 두 방식 모두에서 본 연구에서 제안한 최적 리밸런싱 시점 판단 모델이 더 나은 포트폴리오 구성 시점을 구해냈다. 또한 시뮬레이션 결과 일정주기로 리밸런싱하는 동일한 포트폴리오 구성방식보다 더 나은 성능을 보였다. 특히 입력 변수로 프랙탈 지표들을 추가했을 때 가장 좋은 성능을 보임을 관찰했다. 본 모델은 연구된 기존의 모든 포트폴리오 구성 방법론들에 적용할 수 있는 확장성의 관점에서 중요한 기여가 있다. 그리고 프랙탈 지표를 통해서 관찰되는 네트워크의 구조적 특징들이 미래 시장을 판단하는데 도움이 됨을 보임으로써, 제안한 프랙탈 지표들이 실제 주식시장에 적용 가능한 실용적인 특성을 나타냄도 검증했다.Chapter 1 Introduction 1 1.1 Research Motivation and Purpose 1 1.2 Organization of the Research 5 Chapter 2 Literature Review 7 2.1 Complex Network 7 2.2 Market Prediction with Machine Learning 8 2.3 Trading Strategies 11 Chapter 3 Fractal Structure in Stock Market 13 3.1 Network Fractality 13 3.1.1 Threshold Network 13 3.1.2 Fractal Dimension 15 3.1.3 Fractal Measures 17 3.2 Fractal Analysis on Stock Market 21 3.2.1 Data Description 21 3.2.2 Fractality of S&P500 Network 25 3.2.3 Network Topology and Fractal Measures 27 3.3 Summary and Discussion 43 Chapter 4 Stock Market Prediction with Fractality 45 4.1 Classification of Stock Market 45 4.1.1 Classification Model 45 4.1.2 Classification Results 50 4.2 Fractal Measures and Predictive Power 55 4.2.1 Prediction of Stock Market Return 55 4.2.2 Parameter Analysis 59 4.2.3 Predictive Power Results 60 4.3 Summary and Discussion 66 Chapter 5 Trading Strategy with Optimal Rebalancing Model 69 5.1 Optimal Rebalancing Model 69 5.1.1 Portfolio Selection Method 69 5.1.2 Learning-to-rank algorithm 71 5.1.3 Proposed Modeling Method 73 5.1.4 Model Results 77 5.2 Simulation Analysis 84 5.2.1 Simulation Structure 84 5.2.2 Simulation Results 88 5.3 Summary and Discussion 101 Chapter 6 Conclusion 103 6.1 Conclusions 103 6.2 Future Works 106 Bibliography 107 국문초록 117Docto

Predictive Analytics on Emotional Data Mined from Digital Social Networks with a Focus on Financial Markets

This dissertation is a cumulative dissertation and is comprised of five articles. User-Generated Content (UGC) comprises a substantial part of communication via social media. In this dissertation, UGC that carries and facilitates the exchange of emotions is referred to as “emotional data.” People “produce” emotional data, that is, they express their emotions via tweets, forum posts, blogs, and so on, or they “consume” it by being influenced by expressed sentiments, feelings, opinions, and the like. Decisions often depend on shared emotions and data – which again lead to new data because decisions may change behaviors or results. “Emotional Data Intelligence” ultimately seeks an answer to the question of how all the different emotions expressed in public online sources influence decision-making processes. The overarching research topic of this dissertation follows the question whether network structures and emotional sentiment data extracted from digital social networks contain predictive information or they are just noise. Underlying data was collected from different social media sources, such as Twitter, blogs, message boards, or online news and social networking sites, such as Xing. By means of methodologies of social network analysis (SNA), sentiment analysis, and predictive analysis the individual contributions of this dissertation study whether sentiment data from social media or online social networking structures can predict real-world behaviors. The focus lies on the analysis of emotional data and network structures and its predictive power for financial markets. With the formal construction of the data analyses methodologies introduced in the individual contributions this dissertation contributes to the theories of social network analysis, sentiment analysis, and predictive analytics

Stock market movement prediction using machine learning techniques and graph-based approaches

Machine learning techniques are preferred now than the statistical methods for stock movement prediction due to their efficiency and effectiveness. Stock market movement prediction is impacted significantly by choice of input features and prediction algorithms. We focus on a specific event of ex-dividend day and use event-specific input features of cum-dividend period for predicting price movement on the ex-dividend day. Performance improves significantly when these event-specific optimum input features are used along with machine learning models. The relative order or ranking of stocks is more important than the price or return of a single stock for better investment decisions. Stock ranking performance can be improved by incorporating the stock relationship information in the prediction task. We employ a graph-based approach for stock ranking prediction and use the stock relationship information as the input of the machine learning model. Investing in the top-k stocks is more profitable than the others. Thus, the performance measure for stock ranking prediction should be top-weighted and bounded for any value of k. Existing evaluation measures lack these properties, and we propose normalized rank biased overlap for top-k (NRBO@k) stocks for stock ranking prediction. Moreover, we show that the list-wise loss function can significantly improve the stock ranking performance in a graph-based approach. We find that node embedding techniques such as Node2Vec can significantly reduce graph-based approaches’ training time for the stock ranking prediction. In our survey study, we discuss the existing graph-based works from five perspectives: i) stock market graph formulation, ii) stock market graph filtering, iii) stock market graph clustering, iv) stock movement prediction, and v) portfolio optimization. This survey contains a concise description of major techniques and algorithms relevant to graph-based approaches for the stock market