229 research outputs found
On the Convergence of Locally Adaptive and Scalable Diffusion-Based Sampling Methods for Deep Bayesian Neural Network Posteriors
Achieving robust uncertainty quantification for deep neural networks
represents an important requirement in many real-world applications of deep
learning such as medical imaging where it is necessary to assess the
reliability of a neural network's prediction. Bayesian neural networks are a
promising approach for modeling uncertainties in deep neural networks.
Unfortunately, generating samples from the posterior distribution of neural
networks is a major challenge. One significant advance in that direction would
be the incorporation of adaptive step sizes, similar to modern neural network
optimizers, into Monte Carlo Markov chain sampling algorithms without
significantly increasing computational demand. Over the past years, several
papers have introduced sampling algorithms with claims that they achieve this
property. However, do they indeed converge to the correct distribution? In this
paper, we demonstrate that these methods can have a substantial bias in the
distribution they sample, even in the limit of vanishing step sizes and at full
batch size
Graph Structural Residuals: A Learning Approach to Diagnosis
Traditional model-based diagnosis relies on constructing explicit system
models, a process that can be laborious and expertise-demanding. In this paper,
we propose a novel framework that combines concepts of model-based diagnosis
with deep graph structure learning. This data-driven approach leverages data to
learn the system's underlying structure and provide dynamic observations,
represented by two distinct graph adjacency matrices. Our work facilitates a
seamless integration of graph structure learning with model-based diagnosis by
making three main contributions: (i) redefining the constructs of system
representation, observations, and faults (ii) introducing two distinct versions
of a self-supervised graph structure learning model architecture and (iii)
demonstrating the potential of our data-driven diagnostic method through
experiments on a system of coupled oscillators
Robustness and Generalization Performance of Deep Learning Models on Cyber-Physical Systems: A Comparative Study
Deep learning (DL) models have seen increased attention for time series
forecasting, yet the application on cyber-physical systems (CPS) is hindered by
the lacking robustness of these methods. Thus, this study evaluates the
robustness and generalization performance of DL architectures on multivariate
time series data from CPS. Our investigation focuses on the models' ability to
handle a range of perturbations, such as sensor faults and noise, and assesses
their impact on overall performance. Furthermore, we test the generalization
and transfer learning capabilities of these models by exposing them to
out-of-distribution (OOD) samples. These include deviations from standard
system operations, while the core dynamics of the underlying physical system
are preserved. Additionally, we test how well the models respond to several
data augmentation techniques, including added noise and time warping. Our
experimental framework utilizes a simulated three-tank system, proposed as a
novel benchmark for evaluating the robustness and generalization performance of
DL algorithms in CPS data contexts. The findings reveal that certain DL model
architectures and training techniques exhibit superior effectiveness in
handling OOD samples and various perturbations. These insights have significant
implications for the development of DL models that deliver reliable and robust
performance in real-world CPS applications.Comment: Accepted at the IJCAI 2023 Workshop of Artificial Intelligence for
Time Series Analysis (AI4TS
Using Autoencoders and AutoDiff to Reconstruct Missing Variables in a Set of Time Series
Existing black box modeling approaches in machine learning suffer from a
fixed input and output feature combination. In this paper, a new approach to
reconstruct missing variables in a set of time series is presented. An
autoencoder is trained as usual with every feature on both sides and the neural
network parameters are fixed after this training. Then, the searched variables
are defined as missing variables at the autoencoder input and optimized via
automatic differentiation. This optimization is performed with respect to the
available features loss calculation. With this method, different input and
output feature combinations of the trained model can be realized by defining
the searched variables as missing variables and reconstructing them. The
combination can be changed without training the autoencoder again. The approach
is evaluated on the base of a strongly nonlinear electrical component. It is
working well for one of four variables missing and generally even for multiple
missing variables
Evaluation of Cognitive Architectures for Cyber-Physical Production Systems
Cyber-physical production systems (CPPS) integrate physical and computational
resources due to increasingly available sensors and processing power. This
enables the usage of data, to create additional benefit, such as condition
monitoring or optimization. These capabilities can lead to cognition, such that
the system is able to adapt independently to changing circumstances by learning
from additional sensors information. Developing a reference architecture for
the design of CPPS and standardization of machines and software interfaces is
crucial to enable compatibility of data usage between different machine models
and vendors. This paper analysis existing reference architecture regarding
their cognitive abilities, based on requirements that are derived from three
different use cases. The results from the evaluation of the reference
architectures, which include two instances that stem from the field of
cognitive science, reveal a gap in the applicability of the architectures
regarding the generalizability and the level of abstraction. While reference
architectures from the field of automation are suitable to address use case
specific requirements, and do not address the general requirements, especially
w.r.t. adaptability, the examples from the field of cognitive science are well
usable to reach a high level of adaption and cognition. It is desirable to
merge advantages of both classes of architectures to address challenges in the
field of CPPS in Industrie 4.0
On a Uniform Causality Model for Industrial Automation
The increasing complexity of Cyber-Physical Systems (CPS) makes industrial
automation challenging. Large amounts of data recorded by sensors need to be
processed to adequately perform tasks such as diagnosis in case of fault. A
promising approach to deal with this complexity is the concept of causality.
However, most research on causality has focused on inferring causal relations
between parts of an unknown system. Engineering uses causality in a
fundamentally different way: complex systems are constructed by combining
components with known, controllable behavior. As CPS are constructed by the
second approach, most data-based causality models are not suited for industrial
automation. To bridge this gap, a Uniform Causality Model for various
application areas of industrial automation is proposed, which will allow better
communication and better data usage across disciplines. The resulting model
describes the behavior of CPS mathematically and, as the model is evaluated on
the unique requirements of the application areas, it is shown that the Uniform
Causality Model can work as a basis for the application of new approaches in
industrial automation that focus on machine learning
- …