Differential Dynamic Programming for time-delayed systems

Trajectory optimization considers the problem of deciding how to control a dynamical system to move along a trajectory which minimizes some cost function. Differential Dynamic Programming (DDP) is an optimal control method which utilizes a second-order approximation of the problem to find the control. It is fast enough to allow real-time control and has been shown to work well for trajectory optimization in robotic systems. Here we extend classic DDP to systems with multiple time-delays in the state. Being able to find optimal trajectories for time-delayed systems with DDP opens up the possibility to use richer models for system identification and control, including recurrent neural networks with multiple timesteps in the state. We demonstrate the algorithm on a two-tank continuous stirred tank reactor. We also demonstrate the algorithm on a recurrent neural network trained to model an inverted pendulum with position information only.Comment: 7 pages, 6 figures, conference, Decision and Control (CDC), 2016 IEEE 55th Conference o

Dynamics of Simple Balancing Models with State Dependent Switching Control

Time-delayed control in a balancing problem may be a nonsmooth function for a variety of reasons. In this paper we study a simple model of the control of an inverted pendulum by either a connected movable cart or an applied torque for which the control is turned off when the pendulum is located within certain regions of phase space. Without applying a small angle approximation for deviations about the vertical position, we see structurally stable periodic orbits which may be attracting or repelling. Due to the nonsmooth nature of the control, these periodic orbits are born in various discontinuity-induced bifurcations. Also we show that a coincidence of switching events can produce complicated periodic and aperiodic solutions.Comment: 36 pages, 12 figure

Flat systems, equivalence and trajectory generation

Flat systems, an important subclass of nonlinear control systems introduced via differential-algebraic methods, are defined in a differential geometric framework. We utilize the infinite dimensional geometry developed by Vinogradov and coworkers: a control system is a diffiety, or more precisely, an ordinary diffiety, i.e. a smooth infinite-dimensional manifold equipped with a privileged vector field. After recalling the definition of a Lie-Backlund mapping, we say that two systems are equivalent if they are related by a Lie-Backlund isomorphism. Flat systems are those systems which are equivalent to a controllable linear one. The interest of such an abstract setting relies mainly on the fact that the above system equivalence is interpreted in terms of endogenous dynamic feedback. The presentation is as elementary as possible and illustrated by the VTOL aircraft

Inverted pendulum stabilization: Characterization of codimension-three triple zero bifurcation via multiple delayed proportional gains

International audienceThe paper considers the problem of stabilization of systems possessing a multiple zero eigenvalue at the origin. The controller that we propose, uses multiple delayed measurements instead of derivative terms. Doing so, we increase the performances of the closed loop in presence of system uncertainties and/or noisy measurements. The problem formulation and the analysis is presented through a classical engineering problem which is the stabilization of an inverted pendulum on a cart moving horizontally. On one hand, we perform a nonlinear analysis of the center dynamics described by a three dimensional system of ordinary differential equations with a codimension-three triple zero bifurcation. On the other hand, we present the complementary stability analysis of the corresponding linear time invariant system with two delays describing the behavior around the equilibrium. The aim of this analysis is to characterize the possible local bifurcations. Finally, the proposed control scheme is numerically illustrated and discussed

Human balance control: Dead zones, intermittency and micro-chaos

Summary. The development of strategies to minimize the risk of falling in the elderly represents a major challenge for aging, in industrialized societies. The cor-rective movements made by humans to maintain balance are small amplitude, inter-mittent and ballistic. Small amplitude, complex oscillations (micro-chaos) frequently arise in industrial settings when a time-delayed digital processor attempts to stabilize an unstable equilibrium. Taken together these observations motivate considerations of the effects of a sensory threshold on the stabilization of an inverted pendulum by time-delayed feedback. In the resulting switching-type delay differential equations, the sensory threshold is a strong small-scale nonlinearity which has no effect on large-scale stabilization, but may produce complex, small amplitude dynamics in-cluding limit cycle oscillations and micro-chaos. A close mathematical relationship exists between a scalar model for balance control and the micro-chaotic map that arises in some models of digitally controlled machines. Surprisingly, transient, time-dependent, bounded solutions (transient stabilization) can arise even for parameter ranges where the equilibrium is asymptotically unstable. In other words the combi-nation of a sensory threshold with a time-delayed sampled feedback can increase the range of parameter values for which balance can be maintained, at least transiently. Neuro-biological observations suggest that sensory thresholds can be manipulated either passively by changing posture or actively using efferent feedback. Thus it may be possible to minimize the risk of falling by means of strategies that manipulate sensory thresholds by using physiotherapy and appropriate exercises.

Intermittent Feedback-Control Strategy for Stabilizing Inverted Pendulum on Manually Controlled Cart as Analogy to Human Stick Balancing

The stabilization of an inverted pendulum on a manually controlled cart (cart-inverted-pendulum; CIP) in an upright position, which is analogous to balancing a stick on a fingertip, is considered in order to investigate how the human central nervous system (CNS) stabilizes unstable dynamics due to mechanical instability and time delays in neural feedback control. We explore the possibility that a type of intermittent time-delayed feedback control, which has been proposed for human postural control during quiet standing, is also a promising strategy for the CIP task and stick balancing on a fingertip. Such a strategy hypothesizes that the CNS exploits transient contracting dynamics along a stable manifold of a saddle-type unstable upright equilibrium of the inverted pendulum in the absence of control by inactivating neural feedback control intermittently for compensating delay-induced instability. To this end, the motions of a CIP stabilized by human subjects were experimentally acquired, and computational models of the system were employed to characterize the experimental behaviors. We first confirmed fat-tailed non-Gaussian temporal fluctuation in the acceleration distribution of the pendulum, as well as the power-law distributions of corrective cart movements for skilled subjects, which was previously reported for stick balancing. We then showed that the experimental behaviors could be better described by the models with an intermittent delayed feedback controller than by those with the conventional continuous delayed feedback controller, suggesting that the human CNS stabilizes the upright posture of the pendulum by utilizing the intermittent delayed feedback-control strategy