Stochastic nonlinear control: A unified framework for stability, dissipativity, and optimality

In this work, we develop connections between stochastic stability theory and stochastic optimal control. In particular, first we develop Lyapunov and converse Lyapunov theorems for stochastic semistable nonlinear dynamical systems. Semistability is the property whereby the solutions of a stochastic dynamical system almost surely converge to (not necessarily isolated) Lyapunov stable in probability equilibrium points determined by the system initial conditions. Then we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control for nonlinear stochastic dynamical systems. Specifically, we provide a simplified and tutorial framework for stochastic optimal control and focus on connections between stochastic Lyapunov theory and stochastic Hamilton-Jacobi-Bellman theory. In particular, we show that asymptotic stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function which can clearly be seen to be the solution to the steady-state form of the stochastic Hamilton-Jacobi-Bellman equation, and hence, guaranteeing both stochastic stability and optimality. Moreover, extensions to stochastic finite-time and partial-state stability and optimal stabilization are also addressed. Finally, we extended the notion of dissipativity theory for deterministic dynamical systems to controlled Markov diffusion processes and show the utility of the general concept of dissipation for stochastic systems.Ph.D

Stability of Singular Equilibria in Quasilinear Implicit Differential Equations

AbstractThis paper addresses stability properties of singular equilibria arising in quasilinear implicit ODEs. Under certain assumptions, local dynamics near a singular point may be described through a continuous or directionally continuous vector field. This fact motivates a classification of geometric singularities into weak and strong ones. Stability in the weak case is analyzed through certain linear matrix equations, a singular version of the Lyapunov equation being especially relevant in the study. Weak stable singularities include singular zeros having a spherical domain of attraction which contains other singular points. Regarding strong equilibria, stability is proved via a Lyapunov–Schmidt approach under additional hypotheses. The results are shown to be relevant in singular root-finding problems

Finding complex balanced and detailed balanced realizations of chemical reaction networks

Reversibility, weak reversibility and deficiency, detailed and complex balancing are generally not "encoded" in the kinetic differential equations but they are realization properties that may imply local or even global asymptotic stability of the underlying reaction kinetic system when further conditions are also fulfilled. In this paper, efficient numerical procedures are given for finding complex balanced or detailed balanced realizations of mass action type chemical reaction networks or kinetic dynamical systems in the framework of linear programming. The procedures are illustrated on numerical examples.Comment: submitted to J. Math. Che

On the Existence of Block-Diagonal Solutions to Lyapunov and H∞\mathcal{H}_{\infty} Riccati Inequalities

In this paper, we describe sufficient conditions when block-diagonal solutions to Lyapunov and $\mathcal{H}_{\infty}$ Riccati inequalities exist. In order to derive our results, we define a new type of comparison systems, which are positive and are computed using the state-space matrices of the original (possibly nonpositive) systems. Computing the comparison system involves only the calculation of $\mathcal{H}_{\infty}$ norms of its subsystems. We show that the stability of this comparison system implies the existence of block-diagonal solutions to Lyapunov and Riccati inequalities. Furthermore, our proof is constructive and the overall framework allows the computation of block-diagonal solutions to these matrix inequalities with linear algebra and linear programming. Numerical examples illustrate our theoretical results.Comment: This is an extended technical report. The main results have been accepted for publication as a technical note in the IEEE Transactions on Automatic Contro

Switching and stability properties of conewise linear systems

Being a unique phenomenon in hybrid systems, mode switch is of fundamental importance in dynamic and control analysis. In this paper, we focus on global long-time switching and stability properties of conewise linear systems (CLSs), which are a class of linear hybrid systems subject to state-triggered switchings recently introduced for modeling piecewise linear systems. By exploiting the conic subdivision structure, the “simple switching behavior” of the CLSs is proved. The infinite-time mode switching behavior of the CLSs is shown to be critically dependent on two attracting cones associated with each mode; fundamental properties of such cones are investigated. Verifiable necessary and sufficient conditions are derived for the CLSs with infinite mode switches. Switch-free CLSs are also characterized by exploring the polyhedral structure and the global dynamical properties. The equivalence of asymptotic and exponential stability of the CLSs is established via the uniform asymptotic stability of the CLSs that in turn is proved by the continuous solution dependence on initial conditions. Finally, necessary and sufficient stability conditions are obtained for switch-free CLSs

Control Barrier Function Based Design of Gradient Flows for Constrained Nonlinear Programming

This paper considers the problem of designing a continuous time dynamical system to solve constrained nonlinear optimization problems such that the feasible set is forward invariant and asymptotically stable. The invariance of the feasible set makes the dynamics anytime, when viewed as an algorithm, meaning that it is guaranteed to return a feasible solution regardless of when it is terminated. The system is obtained by augmenting the gradient flow of the objective function with inputs, then designing a feedback controller to keep the state evolution within the feasible set using techniques from the theory of control barrier functions. The equilibria of the system correspond exactly to critical points of the optimization problem. Since the state of the system corresponds to the primal optimizer, and the steady-state input at equilibria corresponds to the dual optimizer, the method can be interpreted as a primal-dual approach. The resulting closed-loop system is locally Lipschitz continuous, so classical solutions to the system exist. We characterize conditions under which local minimizers are Lyapunov stable, drawing connections between various constraint qualification conditions and the stability of the local minimizer. The algorithm is compared to other continuous time methods for optimization

On the Mathematics of the Law of Mass Action

In 1864,Waage and Guldberg formulated the "law of mass action." Since that time, chemists, chemical engineers, physicists and mathematicians have amassed a great deal of knowledge on the topic. In our view, sufficient understanding has been acquired to warrant a formal mathematical consolidation. A major goal of this consolidation is to solidify the mathematical foundations of mass action chemistry -- to provide precise definitions, elucidate what can now be proved, and indicate what is only conjectured. In addition, we believe that the law of mass action is of intrinsic mathematical interest and should be made available in a form that might transcend its application to chemistry alone. We present the law of mass action in the context of a dynamical theory of sets of binomials over the complex numbers.Comment: 40 pages, no figure

Control of Cooperative Haptics-Enabled Teleoperation Systems with Application to Minimally Invasive Surgery

Robot-Assisted Minimally Invasive Surgical (RAMIS) systems frequently have a structure of cooperative teleoperator systems where multiple master-slave pairs are used to collaboratively execute a task. Although multiple studies indicate that haptic feedback improves the realism of tool-tissue interaction to the surgeon and leads to better performance for surgical procedures, current telesurgical systems typically do not provide force feedback, mainly because of the inherent stability issues. The research presented in this thesis is directed towards the development of control algorithms for force reflecting cooperative surgical teleoperator systems with improved stability and transparency characteristics. In the case of cooperative force reflecting teleoperation over networks, conventional passivity based approaches may have limited applicability due to potentially non-passive slave-slave interactions and irregular communication delays imposed by the network. In this thesis, an alternative small gain framework for the design of cooperative network-based force reflecting teleoperator systems is developed. Using the small gain framework, control algorithms for cooperative force-reflecting teleoperator systems are designed that guarantee stability in the presence of multiple network-induced communication constraints. Furthermore, the design conservatism typically associated with the small-gain approach is eliminated by using the Projection-Based Force Reflection (PBFR) algorithms. Stability results are established for networked cooperative teleoperator systems under different types of force reflection algorithms in the presence of irregular communication delays. The proposed control approach is consequently implemented on a dual-arm (two masters/two slaves) robotic MIS testbed. The testbed consists of two Haptic Wand devices as masters and two PA10-7C robots as the slave manipulators equipped with da Vinci laparoscopic surgical instruments. The performance of the proposed control approach is evaluated in three different cooperative surgical tasks, which are knot tightening, pegboard transfer, and object manipulation. The experimental results obtained indicate that the PBFR algorithms demonstrate statistically significant performance improvement in comparison with the conventional direct force reflection algorithms. One possible shortcoming of using PBFR algorithms is that implementation of these algorithms may lead to attenuation of the high-frequency component of the contact force which is important, in particular, for haptic perception of stiff surfaces. In this thesis, a solution to this problem is proposed which is based on the idea of separating the different frequency bands in the force reflection signal and consequently applying the projection-based principle to the low-frequency component, while reflecting the high-frequency component directly. The experimental results demonstrate that substantial improvement in transient fidelity of the force feedback is achieved using the proposed method without negative effects on the stability of the system