106,896 research outputs found
Well Posedness and Convergence Analysis of the Ensemble Kalman Inversion
The ensemble Kalman inversion is widely used in practice to estimate unknown
parameters from noisy measurement data. Its low computational costs,
straightforward implementation, and non-intrusive nature makes the method
appealing in various areas of application. We present a complete analysis of
the ensemble Kalman inversion with perturbed observations for a fixed ensemble
size when applied to linear inverse problems. The well-posedness and
convergence results are based on the continuous time scaling limits of the
method. The resulting coupled system of stochastic differential equations
allows to derive estimates on the long-time behaviour and provides insights
into the convergence properties of the ensemble Kalman inversion. We view the
method as a derivative free optimization method for the least-squares misfit
functional, which opens up the perspective to use the method in various areas
of applications such as imaging, groundwater flow problems, biological problems
as well as in the context of the training of neural networks
Value iteration with deep neural networks for optimal control of input-affine nonlinear systems
This paper proposes a new algorithm with deep neural networks to solve optimal control problems for continuous-time input nonlinear systems based on a value iteration algorithm. The proposed algorithm applies the networks to approximating the value functions and control inputs in the iterations. Consequently, the partial differential equations of the original algorithm reduce to the optimization problems for the parameters of the networks. Although the conventional algorithm can obtain the optimal control with iterative computations, each of the computations needs to be completed precisely, and it is hard to achieve sufficient precision in practice. Instead, the proposed method provides a practical method using deep neural networks and overcomes the difficulty based on a property of the networks, under which our convergence analysis shows that the proposed algorithm can achieve the minimum of the value function and the corresponding optimal controller. The effectiveness of the proposed method even with reasonable computational resources is demonstrated in two numerical simulations
Generalized Policy Iteration for Optimal Control in Continuous Time
This paper proposes the Deep Generalized Policy Iteration (DGPI) algorithm to
find the infinite horizon optimal control policy for general nonlinear
continuous-time systems with known dynamics. Unlike existing adaptive dynamic
programming algorithms for continuous time systems, DGPI does not require the
admissibility of initialized policy, and input-affine nature of controlled
systems for convergence. Our algorithm employs the actor-critic architecture to
approximate both policy and value functions with the purpose of iteratively
solving the Hamilton-Jacobi-Bellman equation. Both the policy and value
functions are approximated by deep neural networks. Given any arbitrary initial
policy, the proposed DGPI algorithm can eventually converge to an admissible,
and subsequently an optimal policy for an arbitrary nonlinear system. We also
relax the update termination conditions of both the policy evaluation and
improvement processes, which leads to a faster convergence speed than
conventional Policy Iteration (PI) methods, for the same architecture of
function approximators. We further prove the convergence and optimality of the
algorithm with thorough Lyapunov analysis, and demonstrate its generality and
efficacy using two detailed numerical examples
Neural Parametric Fokker-Planck Equations
In this paper, we develop and analyze numerical methods for high dimensional
Fokker-Planck equations by leveraging generative models from deep learning. Our
starting point is a formulation of the Fokker-Planck equation as a system of
ordinary differential equations (ODEs) on finite-dimensional parameter space
with the parameters inherited from generative models such as normalizing flows.
We call such ODEs neural parametric Fokker-Planck equation. The fact that the
Fokker-Planck equation can be viewed as the -Wasserstein gradient flow of
Kullback-Leibler (KL) divergence allows us to derive the ODEs as the
constrained -Wasserstein gradient flow of KL divergence on the set of
probability densities generated by neural networks. For numerical computation,
we design a variational semi-implicit scheme for the time discretization of the
proposed ODE. Such an algorithm is sampling-based, which can readily handle
Fokker-Planck equations in higher dimensional spaces. Moreover, we also
establish bounds for the asymptotic convergence analysis of the neural
parametric Fokker-Planck equation as well as its error analysis for both the
continuous and discrete (forward-Euler time discretization) versions. Several
numerical examples are provided to illustrate the performance of the proposed
algorithms and analysis
- …