4,124 research outputs found
Scalable Approach to Uncertainty Quantification and Robust Design of Interconnected Dynamical Systems
Development of robust dynamical systems and networks such as autonomous
aircraft systems capable of accomplishing complex missions faces challenges due
to the dynamically evolving uncertainties coming from model uncertainties,
necessity to operate in a hostile cluttered urban environment, and the
distributed and dynamic nature of the communication and computation resources.
Model-based robust design is difficult because of the complexity of the hybrid
dynamic models including continuous vehicle dynamics, the discrete models of
computations and communications, and the size of the problem. We will overview
recent advances in methodology and tools to model, analyze, and design robust
autonomous aerospace systems operating in uncertain environment, with stress on
efficient uncertainty quantification and robust design using the case studies
of the mission including model-based target tracking and search, and trajectory
planning in uncertain urban environment. To show that the methodology is
generally applicable to uncertain dynamical systems, we will also show examples
of application of the new methods to efficient uncertainty quantification of
energy usage in buildings, and stability assessment of interconnected power
networks
Selectively decentralized reinforcement learning
Indiana University-Purdue University Indianapolis (IUPUI)The main contributions in this thesis include the selectively decentralized method in solving multi-agent reinforcement learning problems and the discretized Markov-decision-process (MDP) algorithm to compute the sub-optimal learning policy in completely unknown learning and control problems. These contributions tackle several challenges in multi-agent reinforcement learning: the unknown and dynamic nature of the learning environment, the difficulty in computing the closed-form solution of the learning problem, the slow learning performance in large-scale systems, and the questions of how/when/to whom the learning agents should communicate among themselves. Through this thesis, the selectively decentralized method, which evaluates all of the possible communicative strategies, not only increases the learning speed, achieves better learning goals but also could learn the communicative policy for each learning agent. Compared to the other state-of-the-art approaches, this thesis’s contributions offer two advantages. First, the selectively decentralized method could incorporate a wide range of well-known algorithms, including the discretized MDP, in single-agent reinforcement learning; meanwhile, the state-of-the-art approaches usually could be applied for one class of algorithms. Second, the discretized MDP algorithm could compute the sub-optimal learning policy when the environment is described in general nonlinear format; meanwhile, the other state-of-the-art approaches often assume that the environment is in limited format, particularly in feedback-linearization form. This thesis also discusses several alternative approaches for multi-agent learning, including Multidisciplinary Optimization. In addition, this thesis shows how the selectively decentralized method could successfully solve several real-worlds problems, particularly in mechanical and biological systems
Error estimates of penalty schemes for quasi-variational inequalities arising from impulse control problems
This paper proposes penalty schemes for a class of weakly coupled systems of
Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) arising from
stochastic hybrid control problems of regime-switching models with both
continuous and impulse controls. We show that the solutions of the penalized
equations converge monotonically to those of the HJBQVIs. We further establish
that the schemes are half-order accurate for HJBQVIs with Lipschitz
coefficients, and first-order accurate for equations with more regular
coefficients. Moreover, we construct the action regions and optimal impulse
controls based on the error estimates and the penalized solutions. The penalty
schemes and convergence results are then extended to HJBQVIs with possibly
negative impulse costs. We also demonstrate the convergence of monotone
discretizations of the penalized equations, and establish that policy iteration
applied to the discrete equation is monotonically convergent with an arbitrary
initial guess in an infinite dimensional setting. Numerical examples for
infinite-horizon optimal switching problems are presented to illustrate the
effectiveness of the penalty schemes over the conventional direct control
scheme.Comment: Accepted for publication in SIAM Journal on Control and Optimizatio
Piecewise Constant Policy Approximations to Hamilton-Jacobi-Bellman Equations
An advantageous feature of piecewise constant policy timestepping for
Hamilton-Jacobi-Bellman (HJB) equations is that different linear approximation
schemes, and indeed different meshes, can be used for the resulting linear
equations for different control parameters. Standard convergence analysis
suggests that monotone (i.e., linear) interpolation must be used to transfer
data between meshes. Using the equivalence to a switching system and an
adaptation of the usual arguments based on consistency, stability and
monotonicity, we show that if limited, potentially higher order interpolation
is used for the mesh transfer, convergence is guaranteed. We provide numerical
tests for the mean-variance optimal investment problem and the uncertain
volatility option pricing model, and compare the results to published test
cases
A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems
A PDE-based accelerated gradient algorithm is proposed to seek optimal
feedback controls of McKean-Vlasov dynamics subject to nonsmooth costs, whose
coefficients involve mean-field interactions both on the state and action. It
exploits a forward-backward splitting approach and iteratively refines the
approximate controls based on the gradients of smooth costs, the proximal maps
of nonsmooth costs, and dynamically updated momentum parameters. At each step,
the state dynamics is realized via a particle approximation, and the required
gradient is evaluated through a coupled system of nonlocal linear PDEs. The
latter is solved by finite difference approximation or neural network-based
residual approximation, depending on the state dimension. Exhaustive numerical
experiments for low and high-dimensional mean-field control problems, including
sparse stabilization of stochastic Cucker-Smale models, are presented, which
reveal that our algorithm captures important structures of the optimal feedback
control, and achieves a robust performance with respect to parameter
perturbation.Comment: Add Sections 2.3 and 2.4 for theoretical convergence result
Deep Learning for Mean Field Games with non-separable Hamiltonians
This paper introduces a new method based on Deep Galerkin Methods (DGMs) for
solving high-dimensional stochastic Mean Field Games (MFGs). We achieve this by
using two neural networks to approximate the unknown solutions of the MFG
system and forward-backward conditions. Our method is efficient, even with a
small number of iterations, and is capable of handling up to 300 dimensions
with a single layer, which makes it faster than other approaches. In contrast,
methods based on Generative Adversarial Networks (GANs) cannot solve MFGs with
non-separable Hamiltonians. We demonstrate the effectiveness of our approach by
applying it to a traffic flow problem, which was previously solved using the
Newton iteration method only in the deterministic case. We compare the results
of our method to analytical solutions and previous approaches, showing its
efficiency. We also prove the convergence of our neural network approximation
with a single hidden layer using the universal approximation theorem
- …