Online interpolation point refinement for reduced order models using a genetic algorithm

A genetic algorithm procedure is demonstrated that refines the selection of interpolation points of the discrete empirical interpolation method (DEIM) when used for constructing reduced order models for time dependent and/or parametrized nonlinear partial differential equations (PDEs) with proper orthogonal decomposition. The method achieves nearly optimal interpolation points with only a few generations of the search, making it potentially useful for {\em online} refinement of the sparse sampling used to construct a projection of the nonlinear terms. With the genetic algorithm, points are optimized to jointly minimize reconstruction error and enable dynamic regime classification. The efficiency of the method is demonstrated on two canonical nonlinear PDEs: the cubic-quintic Ginzburg-Landau equation and the Navier-Stokes equation for flow around a cylinder. Using the former model, the procedure can be compared to the ground-truth optimal interpolation points, showing that the genetic algorithm quickly achieves nearly optimal performance and reduced the reconstruction error by nearly an order of magnitude.Comment: 18 pages, 8 figure

An efficient, globally convergent method for optimization under uncertainty using adaptive model reduction and sparse grids

This work introduces a new method to efficiently solve optimization problems constrained by partial differential equations (PDEs) with uncertain coefficients. The method leverages two sources of inexactness that trade accuracy for speed: (1) stochastic collocation based on dimension-adaptive sparse grids (SGs), which approximates the stochastic objective function with a limited number of quadrature nodes, and (2) projection-based reduced-order models (ROMs), which generate efficient approximations to PDE solutions. These two sources of inexactness lead to inexact objective function and gradient evaluations, which are managed by a trust-region method that guarantees global convergence by adaptively refining the sparse grid and reduced-order model until a proposed error indicator drops below a tolerance specified by trust-region convergence theory. A key feature of the proposed method is that the error indicator---which accounts for errors incurred by both the sparse grid and reduced-order model---must be only an asymptotic error bound, i.e., a bound that holds up to an arbitrary constant that need not be computed. This enables the method to be applicable to a wide range of problems, including those where sharp, computable error bounds are not available; this distinguishes the proposed method from previous works. Numerical experiments performed on a model problem from optimal flow control under uncertainty verify global convergence of the method and demonstrate the method's ability to outperform previously proposed alternatives.Comment: 27 pages, 6 figures, 1 tabl

Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)

We introduce a kernel approximation strategy that enables computation of the Gaussian process log marginal likelihood and all hyperparameter derivatives in $\mathcal{O}(p)$ time. Our GRIEF kernel consists of $p$ eigenfunctions found using a Nystrom approximation from a dense Cartesian product grid of inducing points. By exploiting algebraic properties of Kronecker and Khatri-Rao tensor products, computational complexity of the training procedure can be practically independent of the number of inducing points. This allows us to use arbitrarily many inducing points to achieve a globally accurate kernel approximation, even in high-dimensional problems. The fast likelihood evaluation enables type-I or II Bayesian inference on large-scale datasets. We benchmark our algorithms on real-world problems with up to two-million training points and $10^{33}$ inducing points.Comment: Appears in the proceedings of the International Conference on Machine Learning (ICML), 201

Reduced-order modeling with artificial neurons for gravitational-wave inference

Gravitational-wave data analysis is rapidly absorbing techniques from deep learning, with a focus on convolutional networks and related methods that treat noisy time series as images. We pursue an alternative approach, in which waveforms are first represented as weighted sums over reduced bases (reduced-order modeling); we then train artificial neural networks to map gravitational-wave source parameters into basis coefficients. Statistical inference proceeds directly in coefficient space, where it is theoretically straightforward and computationally efficient. The neural networks also provide analytic waveform derivatives, which are useful for gradient-based sampling schemes. We demonstrate fast and accurate coefficient interpolation for the case of a four-dimensional binary-inspiral waveform family, and discuss promising applications of our framework in parameter estimation.Comment: Published versio

Sparse multiresolution representations with adaptive kernels

Reproducing kernel Hilbert spaces (RKHSs) are key elements of many non-parametric tools successfully used in signal processing, statistics, and machine learning. In this work, we aim to address three issues of the classical RKHS based techniques. First, they require the RKHS to be known a priori, which is unrealistic in many applications. Furthermore, the choice of RKHS affects the shape and smoothness of the solution, thus impacting its performance. Second, RKHSs are ill-equipped to deal with heterogeneous degrees of smoothness, i.e., with functions that are smooth in some parts of their domain but vary rapidly in others. Finally, the computational complexity of evaluating the solution of these methods grows with the number of data points, rendering these techniques infeasible for many applications. Though kernel learning, local kernel adaptation, and sparsity have been used to address these issues, many of these approaches are computationally intensive or forgo optimality guarantees. We tackle these problems by leveraging a novel integral representation of functions in RKHSs that allows for arbitrary centers and different kernels at each center. To address the complexity issues, we then write the function estimation problem as a sparse functional program that explicitly minimizes the support of the representation leading to low complexity solutions. Despite their non-convexity and infinite dimensionality, we show these problems can be solved exactly and efficiently by leveraging duality, and we illustrate this new approach in simulated and real data

A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models

The pattern theory of Grenander is a mathematical framework where patterns are represented by probability models on random variables of algebraic structures. In this paper, we review three families of probability models, namely, the discriminative models, the descriptive models, and the generative models. A discriminative model is in the form of a classifier. It specifies the conditional probability of the class label given the input signal. A descriptive model specifies the probability distribution of the signal, based on an energy function defined on the signal. A generative model assumes that the signal is generated by some latent variables via a transformation. We shall review these models within a common framework and explore their connections. We shall also review the recent developments that take advantage of the high approximation capacities of deep neural networks

Classification of Spatio-Temporal Data via Asynchronous Sparse Sampling: Application to Flow Around a Cylinder

We present a novel method for the classification and reconstruction of time dependent, high-dimensional data using sparse measurements, and apply it to the flow around a cylinder. Assuming the data lies near a low dimensional manifold (low-rank dynamics) in space and has periodic time dependency with a sparse number of Fourier modes, we employ compressive sensing for accurately classifying the dynamical regime. We further show that we can reconstruct the full spatio-temporal behavior with these limited measurements, extending previous results of compressive sensing that apply for only a single snapshot of data. The method can be used for building improved reduced-order models and designing sampling/measurement strategies that leverage time asynchrony.Comment: 19 pages, 4 figure

A Goal-Oriented Adaptive Discrete Empirical Interpolation Method

In this study we propose a-posteriori error estimation results to approximate the precision loss in quantities of interests computed using reduced order models. To generate the surrogate models we employ Proper Orthogonal Decomposition and Discrete Empirical Interpolation Method. First order expansions of the components of the quantity of interest obtained as the product between the components gradient and model residuals are summed up to generate the error estimation result. Efficient versions are derived for explicit and implicit Euler schemes and require only one reduced forward and adjoint models and high-fidelity model residuals estimation. Then we derive an adaptive DEIM algorithm to enhance the accuracy of these quantities of interests. The adaptive DEIM algorithm uses dual weighted residuals singular vectors in combination with the non-linear term basis. Both the a-posteriori error estimation results and the adaptive DEIM algorithm were assessed using the 1D-Burgers and Shallow Water Equation models and the numerical experiments shows very good agreement with the theoretical results.Comment: 34 pages, 19 figure

Environment Identification in Flight using Sparse Approximation of Wing Strain

This paper addresses the problem of identifying different flow environments from sparse data collected by wing strain sensors. Insects regularly perform this feat using a sparse ensemble of noisy strain sensors on their wing. First, we obtain strain data from numerical simulation of a Manduca sexta hawkmoth wing undergoing different flow environments. Our data-driven method learns low-dimensional strain features originating from different aerodynamic environments using proper orthogonal decomposition (POD) modes in the frequency domain, and leverages sparse approximation to classify a set of strain frequency signatures using a dictionary of POD modes. This bio-inspired machine learning architecture for dictionary learning and sparse classification permits fewer costly physical strain sensors while being simultaneously robust to sensor noise. A measurement selection algorithm identifies frequencies that best discriminate the different aerodynamic environments in low-rank POD feature space. In this manner, sparse and noisy wing strain data can be exploited to robustly identify different aerodynamic environments encountered in flight, providing insight into the stereotyped placement of neurons that act as strain sensors on a Manduca sexta hawkmoth wing.Comment: 23 pages, 12 figures. Revised version. Accepted for publication in Journal of Fluids and Structure

Transported snapshot model order reduction approach for parametric, steady-state fluid flows containing parameter dependent shocks

A new model order reduction approach is proposed for parametric steady-state nonlinear fluid flows characterized by shocks and discontinuities whose spatial locations and orientations are strongly parameter dependent. In this method, solutions in the predictive regime are approximated using a linear superposition of parameter dependent basis. The sought after parametric reduced-basis are obtained by transporting the snapshots in a spatially and parametrically dependent transport field. Key to the proposed approach is the observation that the transport fields are typically smooth and continuous, despite the solution themselves not being so. As a result, the transport fields can be accurately expressed using a low-order polynomial expansion. Similar to traditional projection-based model order reduction approaches, the proposed method is formulated mathematically as a residual minimization problem for the generalized coordinates. The proposed approach is also integrated with well-known hyper-reduction strategies to obtain significant computational speed-ups. The method is successfully applied to the reduction of a parametric 1-D flow in a converging-diverging nozzle, a parametric 2-D supersonic flow over a forward facing step and a parametric 2-D jet diffusion flame in a combustor