Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey

Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimization problem could potentially exist. In fact, the process of sequential computation of optimal component weights that maximize the portfolio's expected return subject to a certain risk budget can be reformulated as a discrete-time Markov Decision Process (MDP) and hence as a stochastic optimal control, where the system being controlled is a portfolio consisting of multiple investment components, and the control is its component weights. Consequently, the problem could be solved using model-free Reinforcement Learning (RL) without knowing specific component dynamics. By examining existing methods of both value-based and policy-based model-free RL for the portfolio optimization problem, we identify some of the key unresolved questions and difficulties facing today's portfolio managers of applying model-free RL to their investment portfolios

A stochastic reachability approach to portfolio construction in finance industry

In finance industry portfolio construction deals with how to divide the investors' wealth across an asset-classes' menu in order to maximize the investors' gain. Main approaches in use at the present are based on variations of the classical Markowitz model. However, recent evolutions of the world market showed limitations of this method and motivated many researchers and practitioners to study alternative methodologies to portfolio construction. In this paper we propose one approach to optimal portfolio construction based on recent results on stochastic reachability, which overcome some of the limits of current approaches. Given a sequence of target sets that the investors would like their portfolio to stay within, the optimal portfolio allocation is synthesized in order to maximize the joint probability for the portfolio value to fulfill the target sets requirements. A case study in the US market is given which shows benefits from the proposed methodology in portfolio construction. A comparison with traditional approaches is included.Comment: 15 pages, 7 figure

Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework

We approach the continuous-time mean-variance (MV) portfolio selection with reinforcement learning (RL). The problem is to achieve the best tradeoff between exploration and exploitation, and is formulated as an entropy-regularized, relaxed stochastic control problem. We prove that the optimal feedback policy for this problem must be Gaussian, with time-decaying variance. We then establish connections between the entropy-regularized MV and the classical MV, including the solvability equivalence and the convergence as exploration weighting parameter decays to zero. Finally, we prove a policy improvement theorem, based on which we devise an implementable RL algorithm. We find that our algorithm outperforms both an adaptive control based method and a deep neural networks based algorithm by a large margin in our simulations.Comment: 39 pages, 5 figure

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Optimal funding and investment strategies in defined contribution pension plans under Epstein-Zin utility

A defined contribution pension plan allows consumption to be redistributed from the plan member’s working life to retirement in a manner that is consistent with the member’s personal preferences. The plan’s optimal funding and investment strategies therefore depend on the desired pattern of consumption over the lifetime of the member. We investigate these strategies under the assumption that the member has an Epstein-Zin utility function, which allows a separation between risk aversion and the elasticity of intertemporal substitution, and we also take into account the member’s human capital. We show that a stochastic lifestyling approach, with an initial high weight in equity-type investments and a gradual switch into bond-type investments as the retirement date approaches is an optimal investment strategy. In addition, the optimal contribution rate each year is not constant over the life of the plan but reflects trade-offs between the desire for current consumption, bequest and retirement savings motives at different stages in the life cycle, changes in human capital over the life cycle, and attitude to risk

Network models of financial systemic risk: A review

The global financial system can be represented as a large complex network in which banks, hedge funds and other financial institutions are interconnected to each other through visible and invisible financial linkages. Recently, a lot of attention has been paid to the understanding of the mechanisms that can lead to a breakdown of this network. This can happen when the existing financial links turn from being a means of risk diversification to channels for the propagation of risk across financial institutions. In this review article, we summarize recent developments in the modeling of financial systemic risk. We focus in particular on network approaches, such as models of default cascades due to bilateral exposures or to overlapping portfolios, and we also report on recent findings on the empirical structure of interbank networks. The current review provides a landscape of the newly arising interdisciplinary field lying at the intersection of several disciplines, such as network science, physics, engineering, economics, and ecology.Comment: 33 pages, 6 figure

Practical Deep Reinforcement Learning Approach for Stock Trading

Stock trading strategy plays a crucial role in investment companies. However, it is challenging to obtain optimal strategy in the complex and dynamic stock market. We explore the potential of deep reinforcement learning to optimize stock trading strategy and thus maximize investment return. 30 stocks are selected as our trading stocks and their daily prices are used as the training and trading market environment. We train a deep reinforcement learning agent and obtain an adaptive trading strategy. The agent's performance is evaluated and compared with Dow Jones Industrial Average and the traditional min-variance portfolio allocation strategy. The proposed deep reinforcement learning approach is shown to outperform the two baselines in terms of both the Sharpe ratio and cumulative returns

Master’s Program in Money and Finance (MMF)

The Master’s program in Money and Finance (MMF) is an innovative joint venture of the Department of Money and Macroeconomics and of the Department of Finance, both located in the new House of Finance. The program offers promising students from all over the world an intellectually stimulating and challenging setting in which to prepare for their professional careers in central banking, commercial banking, insurance and other financial services. By being located in Frankfurt, one of the world's leading financial centers and the only city in the world with two central banks (the ECB and the German Bundesbank), it offers unique opportunities for interaction with practitioners. The program is taught exclusively in English; knowledge of German is not required for admission to, or completion of the program. It has been designed with a view to establishing itself as a leading Masters program integrating studies in monetary economics, macroeconomics and finance and a major gateway to high-profile jobs in the banking and financial sector

The History of the Quantitative Methods in Finance Conference Series. 1992-2007

This report charts the history of the Quantitative Methods in Finance (QMF) conference from its beginning in 1993 to the 15th conference in 2007. It lists alphabetically the 1037 speakers who presented at all 15 conferences and the titles of their papers.

A Novel Two-Stage Dynamic Decision Support based Optimal Threat Evaluation and Defensive Resource Scheduling Algorithm for Multi Air-borne threats

This paper presents a novel two-stage flexible dynamic decision support based optimal threat evaluation and defensive resource scheduling algorithm for multi-target air-borne threats. The algorithm provides flexibility and optimality by swapping between two objective functions, i.e. the preferential and subtractive defense strategies as and when required. To further enhance the solution quality, it outlines and divides the critical parameters used in Threat Evaluation and Weapon Assignment (TEWA) into three broad categories (Triggering, Scheduling and Ranking parameters). Proposed algorithm uses a variant of many-to-many Stable Marriage Algorithm (SMA) to solve Threat Evaluation (TE) and Weapon Assignment (WA) problem. In TE stage, Threat Ranking and Threat-Asset pairing is done. Stage two is based on a new flexible dynamic weapon scheduling algorithm, allowing multiple engagements using shoot-look-shoot strategy, to compute near-optimal solution for a range of scenarios. Analysis part of this paper presents the strengths and weaknesses of the proposed algorithm over an alternative greedy algorithm as applied to different offline scenarios.Comment: 8 Pages, International Journal of Computer Science and Information Security, IJCSI