Fidelity-Weighted Learning

Training deep neural networks requires many training samples, but in practice training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing. This creates a fundamental quality versus-quantity trade-off in the learning process. Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data? We argue that if the learner could somehow know and take the label-quality into account when learning the data representation, we could get the best of both worlds. To this end, we propose "fidelity-weighted learning" (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data. FWL modulates the parameter updates to a student network (trained on the task we care about) on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher (who has access to the high-quality labels). Both student and teacher are learned from the data. We evaluate FWL on two tasks in information retrieval and natural language processing where we outperform state-of-the-art alternative semi-supervised methods, indicating that our approach makes better use of strong and weak labels, and leads to better task-dependent data representations.Comment: Published as a conference paper at ICLR 201

Accounting for prediction uncertainty when inferring subsurface fault slip

This study lays the groundwork for a new generation of earthquake source models based on a general formalism that rigorously quantifies and incorporates the impact of uncertainties in fault slip inverse problems. We distinguish two sources of uncertainty when considering the discrepancy between data and forward model predictions. The first class of error is induced by imperfect measurements and is often referred to as observational error. The second source of uncertainty is generally neglected and corresponds to the prediction error, that is the uncertainty due to imperfect forward modelling. Yet the prediction error can be shown to scale approximately with the size of earthquakes and thus can dwarf the observational error, particularly for large events. Both sources of uncertainty can be formulated using the misfit covariance matrix, C_χ, which combines a covariance matrix for observation errors, C_d and a covariance matrix for prediction errors, C_p, associated with inaccurate model predictions. We develop a physically based stochastic forward model to treat the model prediction uncertainty and show how C_p can be constructed to explicitly account for some of the inaccuracies in the earth model. Based on a first-order perturbation approach, our formalism relates C_p to uncertainties on the elastic parameters of different regions (e.g. crust, mantle, etc.). We demonstrate the importance of including C_p using a simple example of an infinite strike-slip fault in the quasi-static approximation. In this toy model, we treat only uncertainties in the 1-D depth distribution of the shear modulus. We discuss how this can be extended to general 3-D cases and applied to other parameters (e.g. fault geometry) using our formalism for C_p. The improved modelling of C_p is expected to lead to more reliable images of the earthquake rupture, that are more resistant to overfitting of data and include more realistic estimates of uncertainty on inferred model parameters

Model selection in systems biology depends on experimental design.

Experimental design attempts to maximise the information available for modelling tasks. An optimal experiment allows the inferred models or parameters to be chosen with the highest expected degree of confidence. If the true system is faithfully reproduced by one of the models, the merit of this approach is clear - we simply wish to identify it and the true parameters with the most certainty. However, in the more realistic situation where all models are incorrect or incomplete, the interpretation of model selection outcomes and the role of experimental design needs to be examined more carefully. Using a novel experimental design and model selection framework for stochastic state-space models, we perform high-throughput in-silico analyses on families of gene regulatory cascade models, to show that the selected model can depend on the experiment performed. We observe that experimental design thus makes confidence a criterion for model choice, but that this does not necessarily correlate with a model's predictive power or correctness. Finally, in the special case of linear ordinary differential equation (ODE) models, we explore how wrong a model has to be before it influences the conclusions of a model selection analysis

Non-Parametric Causality Detection: An Application to Social Media and Financial Data

According to behavioral finance, stock market returns are influenced by emotional, social and psychological factors. Several recent works support this theory by providing evidence of correlation between stock market prices and collective sentiment indexes measured using social media data. However, a pure correlation analysis is not sufficient to prove that stock market returns are influenced by such emotional factors since both stock market prices and collective sentiment may be driven by a third unmeasured factor. Controlling for factors that could influence the study by applying multivariate regression models is challenging given the complexity of stock market data. False assumptions about the linearity or non-linearity of the model and inaccuracies on model specification may result in misleading conclusions. In this work, we propose a novel framework for causal inference that does not require any assumption about the statistical relationships among the variables of the study and can effectively control a large number of factors. We apply our method in order to estimate the causal impact that information posted in social media may have on stock market returns of four big companies. Our results indicate that social media data not only correlate with stock market returns but also influence them.Comment: Physica A: Statistical Mechanics and its Applications 201

Practical guidelines for modelling post-entry spread in invasion ecology

In this article we review a variety of methods to enable understanding and modelling the spread of a pest or pathogen post-entry. Building upon our experience of multidisciplinary research in this area, we propose practical guidelines and a framework for model development, to help with the application of mathematical modelling in the field of invasion ecology for post-entry spread. We evaluate the pros and cons of a range of methods, including references to examples of the methods in practice. We also show how issues of data deficiency and uncertainty can be addressed. The aim is to provide guidance to the reader on the most suitable elements to include in a model of post-entry dispersal in a risk assessment, under differing circumstances. We identify both the strengths and weaknesses of different methods and their application as part of a holistic, multidisciplinary approach to biosecurity research