802 research outputs found
Mitigating the Curse of Dimensionality: Sparse Grid Characteristics Method for Optimal Feedback Control and HJB Equations
We address finding the semi-global solutions to optimal feedback control and
the Hamilton--Jacobi--Bellman (HJB) equation. Using the solution of an HJB
equation, a feedback optimal control law can be implemented in real-time with
minimum computational load. However, except for systems with two or three state
variables, using traditional techniques for numerically finding a semi-global
solution to an HJB equation for general nonlinear systems is infeasible due to
the curse of dimensionality. Here we present a new computational method for
finding feedback optimal control and solving HJB equations which is able to
mitigate the curse of dimensionality. We do not discretize the HJB equation
directly, instead we introduce a sparse grid in the state space and use the
Pontryagin's maximum principle to derive a set of necessary conditions in the
form of a boundary value problem, also known as the characteristic equations,
for each grid point. Using this approach, the method is spatially causality
free, which enjoys the advantage of perfect parallelism on a sparse grid.
Compared with dense grids, a sparse grid has a significantly reduced size which
is feasible for systems with relatively high dimensions, such as the -D
system shown in the examples. Once the solution obtained at each grid point,
high-order accurate polynomial interpolation is used to approximate the
feedback control at arbitrary points. We prove an upper bound for the
approximation error and approximate it numerically. This sparse grid
characteristics method is demonstrated with two examples of rigid body attitude
control using momentum wheels
Approximate Newton Methods for Policy Search in Markov Decision Processes
Approximate Newton methods are standard optimization tools which aim to maintain the benefits of Newton's method, such as a fast rate of convergence, while alleviating its drawbacks, such as computationally expensive calculation or estimation of the inverse Hessian. In this work we investigate approximate Newton methods for policy optimization in Markov decision processes (MDPs). We first analyse the structure of the Hessian of the total expected reward, which is a standard objective function for MDPs. We show that, like the gradient, the Hessian exhibits useful structure in the context of MDPs and we use this analysis to motivate two Gauss-Newton methods for MDPs. Like the Gauss- Newton method for non-linear least squares, these methods drop certain terms in the Hessian. The approximate Hessians possess desirable properties, such as negative definiteness, and we demonstrate several important performance guarantees including guaranteed ascent directions, invariance to affine transformation of the parameter space and convergence guarantees. We finally provide a unifying perspective of key policy search algorithms, demonstrating that our second Gauss- Newton algorithm is closely related to both the EM-algorithm and natural gradient ascent applied to MDPs, but performs significantly better in practice on a range of challenging domains
Visual Servoing
The goal of this book is to introduce the visional application by excellent researchers in the world currently and offer the knowledge that can also be applied to another field widely. This book collects the main studies about machine vision currently in the world, and has a powerful persuasion in the applications employed in the machine vision. The contents, which demonstrate that the machine vision theory, are realized in different field. For the beginner, it is easy to understand the development in the vision servoing. For engineer, professor and researcher, they can study and learn the chapters, and then employ another application method
Current Trends in Symmetric Polynomials with their Applications
This Special Issue presents research papers on various topics within many different branches of mathematics, applied mathematics, and mathematical physics. Each paper presents mathematical theories, methods, and their application based on current and recently developed symmetric polynomials. Also, each one aims to provide the full understanding of current research problems, theories, and applications on the chosen topics and includes the most recent advances made in the area of symmetric functions and polynomials
- …