8 research outputs found
Recommended from our members
Efficient and dimension independent methods for neural network surrogate construction and training
In this dissertation I investigate how to efficiently construct neural network surrogates for parametric maps defined by PDEs, and how to use second order information to improve solutions to the related neural network training problem. Many-query problems arising in scientific applications (such as optimization, uncertainty quantification and inference problems) require evaluation of an input output mapping parametrized by a high dimensional nonlinear PDE model. The cost of these evaluations makes solution using the model prohibitive, and efficient accurate surrogates are the key to solving these problems in practice. In this work I investigate neural network surrogates that use model information to detect informed subspaces of the input and output where the parametric map can be represented efficiently. These compact representations require relatively few data to train and outperform conventional data-driven approaches which require large training data sets. Once a neural network is designed, training is a major issue. One seeks to find optimal weights for a neural network that generalize to data not seen during training. In this work I investigate how second order information can be efficiently exploited to design optimizers that have fast convergence and good generalization properties. These optimizers are shown to outperform conventional methods in numerical experiments.Computational Science, Engineering, and Mathematic
Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators
We propose a novel machine learning framework for solving optimization
problems governed by large-scale partial differential equations (PDEs) with
high-dimensional random parameters. Such optimization under uncertainty (OUU)
problems may be computational prohibitive using classical methods, particularly
when a large number of samples is needed to evaluate risk measures at every
iteration of an optimization algorithm, where each sample requires the solution
of an expensive-to-solve PDE. To address this challenge, we propose a new
neural operator approximation of the PDE solution operator that has the
combined merits of (1) accurate approximation of not only the map from the
joint inputs of random parameters and optimization variables to the PDE state,
but also its derivative with respect to the optimization variables, (2)
efficient construction of the neural network using reduced basis architectures
that are scalable to high-dimensional OUU problems, and (3) requiring only a
limited number of training data to achieve high accuracy for both the PDE
solution and the OUU solution. We refer to such neural operators as multi-input
reduced basis derivative informed neural operators (MR-DINOs). We demonstrate
the accuracy and efficiency our approach through several numerical experiments,
i.e. the risk-averse control of a semilinear elliptic PDE and the steady state
Navier--Stokes equations in two and three spatial dimensions, each involving
random field inputs. Across the examples, MR-DINOs offer -- reductions in execution time, and are able to produce OUU solutions of
comparable accuracies to those from standard PDE based solutions while being
over more cost-efficient after factoring in the cost of
construction
Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems
We explore using neural operators, or neural network representations of
nonlinear maps between function spaces, to accelerate infinite-dimensional
Bayesian inverse problems (BIPs) with models governed by nonlinear parametric
partial differential equations (PDEs). Neural operators have gained significant
attention in recent years for their ability to approximate the
parameter-to-solution maps defined by PDEs using as training data solutions of
PDEs at a limited number of parameter samples. The computational cost of BIPs
can be drastically reduced if the large number of PDE solves required for
posterior characterization are replaced with evaluations of trained neural
operators. However, reducing error in the resulting BIP solutions via reducing
the approximation error of the neural operators in training can be challenging
and unreliable. We provide an a priori error bound result that implies certain
BIPs can be ill-conditioned to the approximation error of neural operators,
thus leading to inaccessible accuracy requirements in training. To reliably
deploy neural operators in BIPs, we consider a strategy for enhancing the
performance of neural operators, which is to correct the prediction of a
trained neural operator by solving a linear variational problem based on the
PDE residual. We show that a trained neural operator with error correction can
achieve a quadratic reduction of its approximation error, all while retaining
substantial computational speedups of posterior sampling when models are
governed by highly nonlinear PDEs. The strategy is applied to two numerical
examples of BIPs based on a nonlinear reaction--diffusion problem and
deformation of hyperelastic materials. We demonstrate that posterior
representations of the two BIPs produced using trained neural operators are
greatly and consistently enhanced by error correction