Recursive Predictability Tests for Real-Time Data

We propose a sequential test for predictive ability. The test is designed for recursive regressions in which the researcher is interested in recursively assessing whether some economic variables have predictive or explanatory content for another variable. It is common in the forecasting literature to assess predictive ability by using "one-shot" tests at each estimation period. We show that this practice: (i) leads to size distortions; (ii) selects overfitted models and provides spurious evidence of in-sample predictive ability; (iii) may lower the accuracy of the model selected by the test. The usefulness of the proposed test is shown in well-know empirical applications to the real-time predictive content of money for output, and the selection between linear and non-linear models.

Predictive context biases perceptual selection during binocular rivalry

Prediction may be a fundamental principle of sensory processing, such that the brain continuously generates predictions about forthcoming sensory information. However, little is known about how prediction contributes to the selection of a conscious percept from among competing alternatives. Here, we used binocular rivalry to investigate the effects of prediction on perceptual selection. In binocular rivalry, incompatible images presented to the two eyes result in a perceptual alternation between the images, even though the visual stimuli remain constant. If predictive signals influence the competition between neural representations of rivalrous images, this influence should generate a bias in perceptual selection that depends on predictive context. To manipulate predictive context, we developed a novel binocular rivalry paradigm in which orthogonal rivalrous test gratings were immediately preceded by rotating gratings presented identically to the two eyes. One of the rivalrous gratings had an orientation that was consistent with the preceding rotation direction (it was the expected next image in the series), and the other had an inconsistent orientation. We found that human observers were more likely to perceive the consistent grating, suggesting that predictive context biased selection in favor of the predicted percept. This prediction effect depended on only recent stimulus history, and it could be dissociated from another stimulus history effect related to orientation-specific adaptation. Since binocular rivalry between orthogonal gratings is thought to be resolved at an early stage of visual processing, these results suggest that predictive signals may exist at low levels of the visual processing hierarchy and that these signals can bias conscious perception. In the future, this paradigm could be used to test whether visual percepts are generated from the combination of prior information and incoming sensory information according to Bayesian principles

Automatic Hyperparameter Tuning Method for Local Outlier Factor, with Applications to Anomaly Detection

In recent years, there have been many practical applications of anomaly detection such as in predictive maintenance, detection of credit fraud, network intrusion, and system failure. The goal of anomaly detection is to identify in the test data anomalous behaviors that are either rare or unseen in the training data. This is a common goal in predictive maintenance, which aims to forecast the imminent faults of an appliance given abundant samples of normal behaviors. Local outlier factor (LOF) is one of the state-of-the-art models used for anomaly detection, but the predictive performance of LOF depends greatly on the selection of hyperparameters. In this paper, we propose a novel, heuristic methodology to tune the hyperparameters in LOF. A tuned LOF model that uses the proposed method shows good predictive performance in both simulations and real data sets.Comment: 15 pages, 5 figure

PREDICTIVE MODEL MARKETS: DESIGN PRINCIPLES FOR MANAGING ENTERPRISE-LEVEL ADVANCED ANALYTICS

As advanced analytics penetrate a wide range of business applications, companies face the challenge of managing analytics-based assets, such as predictive models. Tasks ahead include model selection, scoring and deployment planning. One way to optimize model selection is to tap the combined knowledge of company staff through a “prediction market,” a virtual market designed to reveal participants’ aggregate wisdom by seeing where people “invest” their money. In the context of predictive-model selection, this paper refers to such devices as predictive-model markets. This paper examines design possibilities for building experimental markets that can ultimately be used to test whether predictive-model markets will improve model selection and deployment. The researchers test two types of incentives for participation: economic and social. Study results indicate that such markets can effectively work using either; a surprising finding is that social incentives did not improve effectiveness when added to economic incentives

The influence of feature selection methods on accuracy, stability and interpretability of molecular signatures

Motivation: Biomarker discovery from high-dimensional data is a crucial problem with enormous applications in biology and medicine. It is also extremely challenging from a statistical viewpoint, but surprisingly few studies have investigated the relative strengths and weaknesses of the plethora of existing feature selection methods. Methods: We compare 32 feature selection methods on 4 public gene expression datasets for breast cancer prognosis, in terms of predictive performance, stability and functional interpretability of the signatures they produce. Results: We observe that the feature selection method has a significant influence on the accuracy, stability and interpretability of signatures. Simple filter methods generally outperform more complex embedded or wrapper methods, and ensemble feature selection has generally no positive effect. Overall a simple Student's t-test seems to provide the best results. Availability: Code and data are publicly available at http://cbio.ensmp.fr/~ahaury/

Prediction of Recovery From Severe Hemorrhagic Shock Using Logistic Regression.

This paper implements logistic regression models (LRMs) and feature selection for creating a predictive model for recovery form hemorrhagic shock (HS) with resuscitation using blood in the multiple experimental rat animal protocols. A total of 61 animals were studied across multiple HS experiments, which encompassed two different HS protocols and two resuscitation protocols using blood stored for short periods using five different techniques. Twenty-seven different systemic hemodynamics, cardiac function, and blood gas parameters were measured in each experiment, of which feature selection deemed only 25% of the them as relevant. The reduced feature set was used to train a final logistic regression model. A final test set accuracy is 84% compared to 74% for a baseline classifier using only MAP and HR measurements. Receiver operating characteristics (ROC) curve analysis and Cohens kappa statistics were also used as measures of performance, with the final reduced model outperforming the model, including all parameters. Our results suggest that LRMs trained with a combination of systemic hemodynamics, cardiac function, and blood gas parameters measured at multiple timepoints during HS can successfully classify HS recovery groups. Our results show the predictive ability of traditional and novel hemodynamic and cardiac function features and their combinations, many of which had not previously been taken into consideration, for monitoring HS. Furthermore, we have devised an effective methodology for feature selection and shown ways in which the performance of such predictive models should be assessed in future studies

Evolutionary data selection for enhancing models of intraday forex time series

The hypothesis in this paper is that a significant amount of intraday market data is either noise or redundant, and that if it is eliminated, then predictive models built using the remaining intraday data will be more accurate. To test this hypothesis, we use an evolutionary method (called Evolutionary Data Selection, EDS) to selectively remove out portions of training data that is to be made available to an intraday market predictor. After performing experiments in which data-selected and non-data-selected versions of the same predictive models are compared, it is shown that EDS is effective and does indeed boost predictor accuracy. It is also shown in the paper that building multiple models using EDS and placing them into an ensemble further increases performance. The datasets for evaluation are large intraday forex time series, specifically series from the EUR/USD, the USD/JPY and the EUR/JPY markets, and predictive models for two primary tasks per market are built: intraday return prediction and intraday volatility prediction