Information (In)Efficiency in Prediction Markets

We analyze the extent to which simple markets can be used to aggregate dispersed information into efficient forecasts of unknown future events. From the examination of case studies in a variety of financial settings we enumerate and suggest solutions to various pitfalls of these simple markets. Despite the potential problems, we show that market-generated forecasts are typically fairly accurate in a variety of prediction contexts, and that they outperform most moderately sophisticated benchmarks. We also show how conditional contracts can be used to discover the markets belief about correlations between events, and how with further assumptions these correlations can be used to make decisions

Adaptive Algorithms For Classification On High-Frequency Data Streams: Application To Finance

Mención Internacional en el título de doctorIn recent years, the problem of concept drift has gained importance in the financial domain. The succession of manias, panics and crashes have stressed the nonstationary nature and the likelihood of drastic structural changes in financial markets. The most recent literature suggests the use of conventional machine learning and statistical approaches for this. However, these techniques are unable or slow to adapt to non-stationarities and may require re-training over time, which is computationally expensive and brings financial risks. This thesis proposes a set of adaptive algorithms to deal with high-frequency data streams and applies these to the financial domain. We present approaches to handle different types of concept drifts and perform predictions using up-to-date models. These mechanisms are designed to provide fast reaction times and are thus applicable to high-frequency data. The core experiments of this thesis are based on the prediction of the price movement direction at different intraday resolutions in the SPDR S&P 500 exchange-traded fund. The proposed algorithms are benchmarked against other popular methods from the data stream mining literature and achieve competitive results. We believe that this thesis opens good research prospects for financial forecasting during market instability and structural breaks. Results have shown that our proposed methods can improve prediction accuracy in many of these scenarios. Indeed, the results obtained are compatible with ideas against the efficient market hypothesis. However, we cannot claim that we can beat consistently buy and hold; therefore, we cannot reject it.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: Gustavo Recio Isasi.- Secretario: Pedro Isasi Viñuela.- Vocal: Sandra García Rodrígue

Early Warning Analysis for Social Diffusion Events

There is considerable interest in developing predictive capabilities for social diffusion processes, for instance to permit early identification of emerging contentious situations, rapid detection of disease outbreaks, or accurate forecasting of the ultimate reach of potentially viral ideas or behaviors. This paper proposes a new approach to this predictive analytics problem, in which analysis of meso-scale network dynamics is leveraged to generate useful predictions for complex social phenomena. We begin by deriving a stochastic hybrid dynamical systems (S-HDS) model for diffusion processes taking place over social networks with realistic topologies; this modeling approach is inspired by recent work in biology demonstrating that S-HDS offer a useful mathematical formalism with which to represent complex, multi-scale biological network dynamics. We then perform formal stochastic reachability analysis with this S-HDS model and conclude that the outcomes of social diffusion processes may depend crucially upon the way the early dynamics of the process interacts with the underlying network's community structure and core-periphery structure. This theoretical finding provides the foundations for developing a machine learning algorithm that enables accurate early warning analysis for social diffusion events. The utility of the warning algorithm, and the power of network-based predictive metrics, are demonstrated through an empirical investigation of the propagation of political memes over social media networks. Additionally, we illustrate the potential of the approach for security informatics applications through case studies involving early warning analysis of large-scale protests events and politically-motivated cyber attacks

New forecast tools to enhance the value of VRE on the electricity market: Deliverable D4.9

Project TradeRES - New Markets Design & Models for 100% Renewable Power Systems: https://traderes.eu/about/ABSTRACT: The present deliverable was developed as part of the research activities of the TradeRES project Task 4.4 - Enhancing the value of VRE on the electricity markets with advanced forecasting and ramping tools. This report presents the first version of deliverable 4.9, which consists on the description and implementation of the forecasting techniques aiming to identify and explore the time synergies of meteorological effects and electricity market designs in order to maximize the value of variable renewable energy systems and minimize market imbalances. An overview of key aspects that characterize a power forecast system is presented in this deliverable through a literature review. This overview addresses the: i) forecast time horizon; ii) type of approach (physical, statistical or hybrid); iii) data pre-processing procedures; iv) type of forecast output; and v) the most common metrics used to evaluate the performance of the forecast systems.N/

Using Explainable AI to Understand Bond Excess Returns

Recent empirical evidence indicates that bond excess returns can be predicted using machine learning models. While the predictive power of machine learning models is intriguing, they typically lack transparency. We introduce SHapley Additive exPlanations (SHAP), a state-of-the-art explainable artificial technique, to open the black box of these models. Our analysis identifies the key determinants that drive the predictions of bond excess returns in machine learning models and how these determinants are related to bond excess returns. Thereby, our approach facilitates an in-depth interpretation of the predictions of bond excess returns made by machine learning models

A Bayesian network approach to county-level corn yield prediction using historical data and expert knowledge

Machine learning has become a popular technology that has not only turbo-charged the existing problems in the AI but it has also emerged as the powerful toolkit to solve some of the interesting problems across the various interdisciplinary domains. The availability of food is the biggest problem of the 21st century and many experts have raised their concerns as we continue to see a rise in the global human population. There have been many efforts in this direction which include but not limited to improvement in the seeds quality, good management practices, prior knowledge about the expected yield, etc. In this work, we propose a data-driven approach that is ‘gray box’ i.e. that seamlessly utilizes expert knowledge in constructing a statistical network model for corn yield forecasting. Our multivariate gray box model is developed on Bayesian network analysis to build a Directed Acyclic Graph (DAG) between predictors and yield. Starting from a complete graph connecting various carefully chosen variables and yield, expert knowledge is used to prune or strengthen edges connecting variables. Subsequently, the structure (connectivity and edge weights) of the DAG that maximizes the likelihood of observing the training data is identified via optimization. We curated an extensive set of historical data (1948 − 2012) for each of the 99 counties in Iowa as data to train the model. We discuss preliminary results, and specifically focus on (a) the structure of the learned network and how it corroborates with known trends, and (b) how partial information still produces reasonable predictions (predictions with gappy data), and show that incorporating the missing information improves predictions

Econometrics meets sentiment : an overview of methodology and applications

The advent of massive amounts of textual, audio, and visual data has spurred the development of econometric methodology to transform qualitative sentiment data into quantitative sentiment variables, and to use those variables in an econometric analysis of the relationships between sentiment and other variables. We survey this emerging research field and refer to it as sentometrics, which is a portmanteau of sentiment and econometrics. We provide a synthesis of the relevant methodological approaches, illustrate with empirical results, and discuss useful software