2,161 research outputs found
An Approximate Bayesian Long Short-Term Memory Algorithm for Outlier Detection
Long Short-Term Memory networks trained with gradient descent and
back-propagation have received great success in various applications. However,
point estimation of the weights of the networks is prone to over-fitting
problems and lacks important uncertainty information associated with the
estimation. However, exact Bayesian neural network methods are intractable and
non-applicable for real-world applications. In this study, we propose an
approximate estimation of the weights uncertainty using Ensemble Kalman Filter,
which is easily scalable to a large number of weights. Furthermore, we optimize
the covariance of the noise distribution in the ensemble update step using
maximum likelihood estimation. To assess the proposed algorithm, we apply it to
outlier detection in five real-world events retrieved from the Twitter
platform
Data Assimilation by Artificial Neural Networks for an Atmospheric General Circulation Model: Conventional Observation
This paper presents an approach for employing artificial neural networks (NN)
to emulate an ensemble Kalman filter (EnKF) as a method of data assimilation.
The assimilation methods are tested in the Simplified Parameterizations
PrimitivE-Equation Dynamics (SPEEDY) model, an atmospheric general circulation
model (AGCM), using synthetic observational data simulating localization of
balloon soundings. For the data assimilation scheme, the supervised NN, the
multilayer perceptrons (MLP-NN), is applied. The MLP-NN are able to emulate the
analysis from the local ensemble transform Kalman filter (LETKF). After the
training process, the method using the MLP-NN is seen as a function of data
assimilation. The NN were trained with data from first three months of 1982,
1983, and 1984. A hind-casting experiment for the 1985 data assimilation cycle
using MLP-NN were performed with synthetic observations for January 1985. The
numerical results demonstrate the effectiveness of the NN technique for
atmospheric data assimilation. The results of the NN analyses are very close to
the results from the LETKF analyses, the differences of the monthly average of
absolute temperature analyses is of order 0.02. The simulations show that the
major advantage of using the MLP-NN is better computational performance, since
the analyses have similar quality. The CPU-time cycle assimilation with MLP-NN
is 90 times faster than cycle assimilation with LETKF for the numerical
experiment.Comment: 17 pages, 16 figures, monthly weather revie
Data-driven Soft Sensors in the Process Industry
In the last two decades Soft Sensors established themselves as a valuable alternative to the traditional means for the acquisition of critical process variables, process monitoring and other tasks which are related to process control. This paper discusses characteristics of the process industry data which are critical for the development of data-driven Soft Sensors. These characteristics are common to a large number of process industry fields, like the chemical industry, bioprocess industry, steel industry, etc. The focus of this work is put on the data-driven Soft Sensors because of their growing popularity, already demonstrated usefulness and huge, though yet not completely realised, potential. A comprehensive selection of case studies covering the three most important Soft Sensor application fields, a general introduction to the most popular Soft Sensor modelling techniques as well as a discussion of some open issues in the Soft Sensor development and maintenance and their possible solutions are the main contributions of this work
Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks
The standard probabilistic perspective on machine learning gives rise to
empirical risk-minimization tasks that are frequently solved by stochastic
gradient descent (SGD) and variants thereof. We present a formulation of these
tasks as classical inverse or filtering problems and, furthermore, we propose
an efficient, gradient-free algorithm for finding a solution to these problems
using ensemble Kalman inversion (EKI). Applications of our approach include
offline and online supervised learning with deep neural networks, as well as
graph-based semi-supervised learning. The essence of the EKI procedure is an
ensemble based approximate gradient descent in which derivatives are replaced
by differences from within the ensemble. We suggest several modifications to
the basic method, derived from empirically successful heuristics developed in
the context of SGD. Numerical results demonstrate wide applicability and
robustness of the proposed algorithm.Comment: 41 pages, 14 figure
- …