65 research outputs found
An Approximate Bayesian Long Short-Term Memory Algorithm for Outlier Detection
Long Short-Term Memory networks trained with gradient descent and
back-propagation have received great success in various applications. However,
point estimation of the weights of the networks is prone to over-fitting
problems and lacks important uncertainty information associated with the
estimation. However, exact Bayesian neural network methods are intractable and
non-applicable for real-world applications. In this study, we propose an
approximate estimation of the weights uncertainty using Ensemble Kalman Filter,
which is easily scalable to a large number of weights. Furthermore, we optimize
the covariance of the noise distribution in the ensemble update step using
maximum likelihood estimation. To assess the proposed algorithm, we apply it to
outlier detection in five real-world events retrieved from the Twitter
platform
Optimal Data Split Methodology for Model Validation
The decision to incorporate cross-validation into validation processes of
mathematical models raises an immediate question - how should one partition the
data into calibration and validation sets? We answer this question
systematically: we present an algorithm to find the optimal partition of the
data subject to certain constraints. While doing this, we address two critical
issues: 1) that the model be evaluated with respect to predictions of a given
quantity of interest and its ability to reproduce the data, and 2) that the
model be highly challenged by the validation set, assuming it is properly
informed by the calibration set. This framework also relies on the interaction
between the experimentalist and/or modeler, who understand the physical system
and the limitations of the model; the decision-maker, who understands and can
quantify the cost of model failure; and the computational scientists, who
strive to determine if the model satisfies both the modeler's and decision
maker's requirements. We also note that our framework is quite general, and may
be applied to a wide range of problems. Here, we illustrate it through a
specific example involving a data reduction model for an ICCD camera from a
shock-tube experiment located at the NASA Ames Research Center (ARC).Comment: Submitted to International Conference on Modeling, Simulation and
Control 2011 (ICMSC'11), San Francisco, USA, 19-21 October, 201
From Causal Pairs to Causal Graphs
Causal structure learning from observational data remains a non-trivial task
due to various factors such as finite sampling, unobserved confounding factors,
and measurement errors. Constraint-based and score-based methods tend to suffer
from high computational complexity due to the combinatorial nature of
estimating the directed acyclic graph (DAG). Motivated by the `Cause-Effect
Pair' NIPS 2013 Workshop on Causality Challenge, in this paper, we take a
different approach and generate a probability distribution over all possible
graphs informed by the cause-effect pair features proposed in response to the
workshop challenge. The goal of the paper is to propose new methods based on
this probabilistic information and compare their performance with traditional
and state-of-the-art approaches. Our experiments, on both synthetic and real
datasets, show that our proposed methods not only have statistically similar or
better performances than some traditional approaches but also are
computationally faster.Comment: ICMLA 202
Validating Predictions of Unobserved Quantities
The ultimate purpose of most computational models is to make predictions,
commonly in support of some decision-making process (e.g., for design or
operation of some system). The quantities that need to be predicted (the
quantities of interest or QoIs) are generally not experimentally observable
before the prediction, since otherwise no prediction would be needed. Assessing
the validity of such extrapolative predictions, which is critical to informed
decision-making, is challenging. In classical approaches to validation, model
outputs for observed quantities are compared to observations to determine if
they are consistent. By itself, this consistency only ensures that the model
can predict the observed quantities under the conditions of the observations.
This limitation dramatically reduces the utility of the validation effort for
decision making because it implies nothing about predictions of unobserved QoIs
or for scenarios outside of the range of observations. However, there is no
agreement in the scientific community today regarding best practices for
validation of extrapolative predictions made using computational models. The
purpose of this paper is to propose and explore a validation and predictive
assessment process that supports extrapolative predictions for models with
known sources of error. The process includes stochastic modeling, calibration,
validation, and predictive assessment phases where representations of known
sources of uncertainty and error are built, informed, and tested. The proposed
methodology is applied to an illustrative extrapolation problem involving a
misspecified nonlinear oscillator
- …