5 research outputs found
A Multi-level procedure for enhancing accuracy of machine learning algorithms
We propose a multi-level method to increase the accuracy of machine learning
algorithms for approximating observables in scientific computing, particularly
those that arise in systems modeled by differential equations. The algorithm
relies on judiciously combining a large number of computationally cheap
training data on coarse resolutions with a few expensive training samples on
fine grid resolutions. Theoretical arguments for lowering the generalization
error, based on reducing the variance of the underlying maps, are provided and
numerical evidence, indicating significant gains over underlying single-level
machine learning algorithms, are presented. Moreover, we also apply the
multi-level algorithm in the context of forward uncertainty quantification and
observe a considerable speed-up over competing algorithms
Higher-order Quasi-Monte Carlo Training of Deep Neural Networks
We present a novel algorithmic approach and an error analysis leveraging
Quasi-Monte Carlo points for training deep neural network (DNN) surrogates of
Data-to-Observable (DtO) maps in engineering design. Our analysis reveals
higher-order consistent, deterministic choices of training points in the input
data space for deep and shallow Neural Networks with holomorphic activation
functions such as tanh. These novel training points are proved to facilitate
higher-order decay (in terms of the number of training samples) of the
underlying generalization error, with consistency error bounds that are free
from the curse of dimensionality in the input data space, provided that DNN
weights in hidden layers satisfy certain summability conditions. We present
numerical experiments for DtO maps from elliptic and parabolic PDEs with
uncertain inputs that confirm the theoretical analysis
Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks
We present a novel active learning algorithm, termed as iterative surrogate
model optimization (ISMO), for robust and efficient numerical approximation of
PDE constrained optimization problems. This algorithm is based on deep neural
networks and its key feature is the iterative selection of training data
through a feedback loop between deep neural networks and any underlying
standard optimization algorithm. Under suitable hypotheses, we show that the
resulting optimizers converge exponentially fast (and with exponentially
decaying variance), with respect to increasing number of training samples.
Numerical examples for optimal control, parameter identification and shape
optimization problems for PDEs are provided to validate the proposed theory and
to illustrate that ISMO significantly outperforms a standard deep neural
network based surrogate optimization algorithm