Expert-augmented machine learning.

Machine learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption is limited by the level of trust afforded by given models. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be performed by a computer or an expert. In reality, the optimal learning strategy may involve combining the complementary strengths of humans and machines. Here, we present expert-augmented machine learning (EAML), an automated method that guides the extraction of expert knowledge and its integration into machine-learned models. We used a large dataset of intensive-care patient data to derive 126 decision rules that predict hospital mortality. Using an online platform, we asked 15 clinicians to assess the relative risk of the subpopulation defined by each rule compared to the total sample. We compared the clinician-assessed risk to the empirical risk and found that, while clinicians agreed with the data in most cases, there were notable exceptions where they overestimated or underestimated the true risk. Studying the rules with greatest disagreement, we identified problems with the training data, including one miscoded variable and one hidden confounder. Filtering the rules based on the extent of disagreement between clinician-assessed risk and empirical risk, we improved performance on out-of-sample data and were able to train with less data. EAML provides a platform for automated creation of problem-specific priors, which help build robust and dependable machine-learning models in critical applications

DNA repair deficiency biomarkers and the 70-gene ultra-high risk signature as predictors of veliparib/carboplatin response in the I-SPY 2 breast cancer trial.

Veliparib combined with carboplatin (VC) was an experimental regimen evaluated in the biomarker-rich neoadjuvant I-SPY 2 trial for breast cancer. VC showed improved efficacy in the triple negative signature. However, not all triple negative patients achieved pathologic complete response and some HR+HER2- patients responded. Pre-specified analysis of five DNA repair deficiency biomarkers (BRCA1/2 germline mutation; PARPi-7, BRCA1ness, and CIN70 expression signatures; and PARP1 protein) was performed on 116 HER2- patients (VC: 72 and concurrent controls: 44). We also evaluated the 70-gene ultra-high risk signature (MP1/2), one of the biomarkers used to define subtype in the trial. We used logistic modeling to assess biomarker performance. Successful biomarkers were combined using a simple voting scheme to refine the 'predicted sensitive' group and Bayesian modeling used to estimate the pathologic complete response rates. BRCA1/2 germline mutation status associated with VC response, but its low prevalence precluded further evaluation. PARPi-7, BRCA1ness, and MP1/2 specifically associated with response in the VC arm but not the control arm. Neither CIN70 nor PARP1 protein specifically predicted VC response. When we combined the PARPi-7 and MP1/2 classifications, the 42% of triple negative patients who were PARPi7-high and MP2 had an estimated pCR rate of 75% in the VC arm. Only 11% of HR+/HER2- patients were PARPi7-high and MP2; but these patients were also more responsive to VC with estimated pathologic complete response rates of 41%. PARPi-7, BRCA1ness and MP1/2 signatures may help refine predictions of VC response, thereby improving patient care

Comparison of Classifiers for Radar Emitter Type Identification

ARTMAP neural network classifiers are considered for the identification of radar emitter types from their waveform parameters. These classifiers can represent radar emitter type classes with one or more prototypes, perform on-line incremental learning to account for novelty encountered in the field, and process radar pulse streams at high speed, making them attractive for real-time applications such as electronic support measures (ESM). The performance of four ARTMAP variants- ARTMAP (Stage 1), ARTMAP-IC, fuzzy ARTMAP and Gaussian ARTMAP - is assessed with radar data gathered in the field. The k nearest neighbor (kNN) and radial basis function (RDF) classifiers are used for reference. Simulation results indicate that fuzzy ARTMAP and Gaussian ARTMAP achieve an average classification rate consistently higher than that of the other ARTMAP classifers and comparable to that of kNN and RBF. ART-EMAP, ARTMAP-IC and fuzzy ARTMAP require fewer training epochs than Gaussian ARTMAP and RBF, and substantially fewer prototype vectors (thus, smaller physical memory requirements and faster fielded performance) than Gaussian ARTMAP, RBF and kNN. Overall, fuzzy ART MAP performs at least as well as the other classifiers in both accuracy and computational complexity, and better than each of them in at least one of these aspects of performance. Incorporation into fuzzy ARTMAP of the MT- feature of ARTMAP-IC is found to be essential for convergence during on-line training with this data set.Defense Advanced Research Projects Agency and the Office of Naval Research (N000I4-95-1-409 (S.G. and M.A.R.); National Science Foundation (IRI-97-20333) (S.G.); Natural Science and Engineering Research Council of Canada (E.G.); Office of Naval Research (N00014-95-1-0657

Heuristic model selection for leading indicators in Russia and Germany

Business tendency survey indicators are widely recognized as a key instrument for business cycle forecasting. Their leading indicator property is assessed with regard to forecasting industrial production in Russia and Germany. For this purpose, vector autoregressive (VAR) models are specified and estimated to construct forecasts. As the potential number of lags included is large, we compare full–specified VAR models with subset models obtained using a Genetic Algorithm enabling ’holes’ in multivariate lag structures. The problem is complicated by the fact that a structural break and seasonal variation of indicators have to be taken into account. The models allow for a comparison of the dynamic adjustment and the forecasting performance of the leading indicators for bothLeading indicators, business cycle forecasts, VAR, model selection, genetic algorithms.

Heuristic model selection for leading indicators in Russia and Germany

Business tendency survey indicators are widely recognized as a key instrument for business cycle forecasting. Their leading indicator property is assessed with regard to forecasting industrial production in Russia and Germany. For this purpose, vector autoregressive (VAR) models are specified and estimated to construct forecasts. As the potential number of lags included is large, we compare full–specified VAR models with subset models obtained using a Genetic Algorithm enabling ’holes’ in multivariate lag structures. The problem is complicated by the fact that a structural break and seasonal variation of indicators have to be taken into account. The models allow for a comparison of the dynamic adjustment and the forecasting performance of the leading indicators for both countries revealing marked differences between Russia and Germany.Leading indicators, business cycle forecasts, VAR, model selection, genetic algorithms

Diagnosis and Prediction of Market Rebounds in Financial Markets

We introduce the concept of "negative bubbles" as the mirror image of standard financial bubbles, in which positive feedback mechanisms may lead to transient accelerating price falls. To model these negative bubbles, we adapt the Johansen-Ledoit-Sornette (JLS) model of rational expectation bubbles with a hazard rate describing the collective buying pressure of noise traders. The price fall occurring during a transient negative bubble can be interpreted as an effective random downpayment that rational agents accept to pay in the hope of profiting from the expected occurrence of a possible rally. We validate the model by showing that it has significant predictive power in identifying the times of major market rebounds. This result is obtained by using a general pattern recognition method which combines the information obtained at multiple times from a dynamical calibration of the JLS model. Error diagrams, Bayesian inference and trading strategies suggest that one can extract genuine information and obtain real skill from the calibration of negative bubbles with the JLS model. We conclude that negative bubbles are in general predictably associated with large rebounds or rallies, which are the mirror images of the crashes terminating standard bubbles.Comment: 49 pages, 14 figure

Neural Network-Based Equations for Predicting PGA and PGV in Texas, Oklahoma, and Kansas

Parts of Texas, Oklahoma, and Kansas have experienced increased rates of seismicity in recent years, providing new datasets of earthquake recordings to develop ground motion prediction models for this particular region of the Central and Eastern North America (CENA). This paper outlines a framework for using Artificial Neural Networks (ANNs) to develop attenuation models from the ground motion recordings in this region. While attenuation models exist for the CENA, concerns over the increased rate of seismicity in this region necessitate investigation of ground motions prediction models particular to these states. To do so, an ANN-based framework is proposed to predict peak ground acceleration (PGA) and peak ground velocity (PGV) given magnitude, earthquake source-to-site distance, and shear wave velocity. In this framework, approximately 4,500 ground motions with magnitude greater than 3.0 recorded in these three states (Texas, Oklahoma, and Kansas) since 2005 are considered. Results from this study suggest that existing ground motion prediction models developed for CENA do not accurately predict the ground motion intensity measures for earthquakes in this region, especially for those with low source-to-site distances or on very soft soil conditions. The proposed ANN models provide much more accurate prediction of the ground motion intensity measures at all distances and magnitudes. The proposed ANN models are also converted to relatively simple mathematical equations so that engineers can easily use them to predict the ground motion intensity measures for future events. Finally, through a sensitivity analysis, the contributions of the predictive parameters to the prediction of the considered intensity measures are investigated.Comment: 5th Geotechnical Earthquake Engineering and Soil Dynamics Conference, Austin, TX, USA, June 10-13. (2018