Differential-Algebraic Equations and Beyond: From Smooth to Nonsmooth Constrained Dynamical Systems

The present article presents a summarizing view at differential-algebraic equations (DAEs) and analyzes how new application fields and corresponding mathematical models lead to innovations both in theory and in numerical analysis for this problem class. Recent numerical methods for nonsmooth dynamical systems subject to unilateral contact and friction illustrate the topicality of this development.Comment: Preprint of Book Chapte

A Discrete Geometric Optimal Control Framework for Systems with Symmetries

This paper studies the optimal motion control of mechanical systems through a discrete geometric approach. At the core of our formulation is a discrete Lagrange-d’Alembert- Pontryagin variational principle, from which are derived discrete equations of motion that serve as constraints in our optimization framework. We apply this discrete mechanical approach to holonomic systems with symmetries and, as a result, geometric structure and motion invariants are preserved. We illustrate our method by computing optimal trajectories for a simple model of an air vehicle flying through a digital terrain elevation map, and point out some of the numerical benefits that ensue

A Brief Survey on Non-standard Constraints: Simulation and Optimal Control

In terms of simulation and control holonomic constraints are well documented and thus termed standard. As non-standard constraints, we understand non-holonomic and unilateral constraints. We limit this survey to mechanical systems with a finite number of degrees of freedom. The long-term behavior of non- holonomic integrators as compared to structure-preserving integrators for holonomically constrained systems is briefly discussed. Some recent research regarding the treatment of unilaterally constrained systems by event-driven or time-stepping schemes for time integration and in the context of optimal control problems is outlined

Geometric Numerical Integration

The subject of this workshop was numerical methods that preserve geometric properties of the flow of an ordinary or partial differential equation. This was complemented by the question as to how structure preservation affects the long-time behaviour of numerical methods

Geometric numerical integration for optimisation

In this thesis, we study geometric numerical integration for the optimisation of various classes of functionals. Numerical integration and the study of systems of differential equations have received increased attention within the optimisation community in the last decade, as a means for devising new optimisation schemes as well as to improve our understanding of the dynamics of existing schemes. Discrete gradient methods from geometric numerical integration preserve structures of first-order gradient systems, including the dissipative structure of schemes such as gradient flows, and thus yield iterative methods that are unconditionally dissipative, i.e. decrease the objective function value for all time steps. We look at discrete gradient methods for optimisation in several settings. First, we provide a comprehensive study of discrete gradient methods for optimisation of continuously differentiable functions. In particular, we prove properties such as well-posedness of the discrete gradient update equation, convergence rates, convergence of the iterates, and propose methods for solving the discrete gradient update equation with superior stability and convergence rates. Furthermore, we present results from numerical experiments which support the theory. Second, motivated by the existence of derivative-free discrete gradients, and seeking to solve nonsmooth optimisation problems and more generally black-box problems, including for parameter optimisation problems, we propose methods based on the Itoh--Abe discrete gradient method for solving nonconvex, nonsmooth optimisation problems with derivative-free methods. In this setting, we prove well-posedness of the method, and convergence guarantees within the nonsmooth, nonconvex Clarke subdifferential framework for locally Lipschitz continuous functions. The analysis is shown to hold in various settings, namely in the unconstrained and constrained setting, including epi-Lipschitzian constraints, and for stochastic and deterministic optimisation methods. Building on the work of derivative-free discrete gradient methods and the concept of structure preservation in geometric numerical integration, we consider discrete gradient methods applied to other differential systems with dissipative structures. In particular, we study the inverse scale space flow, linked to the well-known Bregman methods, which are central to variational optimisation problems and regularisation methods for inverse problems. In this setting, we propose and implement derivative-free schemes that exploit structures such as sparsity to achieve superior convergence rates in numerical experiments, and prove convergence guarantees for these methods in the nonsmooth, nonconvex setting. Furthermore, these schemes can be seen as generalisations of the Gauss-Seidel method and successive-over-relaxation. Finally, we return to parameter optimisation problems, namely nonsmooth bilevel optimisation problems, and propose a framework to employ first-order methods for these problems, when the underlying variational optimisation problem admits a nonsmooth structure in the partial smoothness framework. In this setting, we prove piecewise differentiability of the parameter-dependent solution mapping, and study algorithmic differentiation approaches to evaluating the derivatives. Furthermore, we prove that the algorithmic derivatives converge to the implicit derivatives. Thus we demonstrate that, although some parameter tuning problems must inevitably be treated as black-box optimisation problems, for a large number of variational problems one can exploit the structure of nonsmoothness to perform gradient-based bilevel optimisation

Discrete mechanics and variational integrators

This paper gives a review of integration algorithms for finite dimensional mechanical systems that are based on discrete variational principles. The variational technique gives a unified treatment of many symplectic schemes, including those of higher order, as well as a natural treatment of the discrete Noether theorem. The approach also allows us to include forces, dissipation and constraints in a natural way. Amongst the many specific schemes treated as examples, the Verlet, SHAKE, RATTLE, Newmark, and the symplectic partitioned Runge–Kutta schemes are presented

A Nonsmooth Dynamical Systems Perspective on Accelerated Extensions of ADMM

The acceleration technique introduced by Nesterov for gradient descent is widely used in optimization but its principles are not yet fully understood. Recently, significant progress has been made to close this understanding gap through a continuous-time dynamical systems perspective associated with gradient methods for smooth and unconstrained problems. Here we extend this perspective to nonsmooth and linearly constrained problems by deriving nonsmooth dynamical systems related to variants of the relaxed and accelerated alternating direction method of multipliers (ADMM). More specifically, we introduce two new ADMM variants, one based on Nesterov's acceleration and the other inspired by the heavy ball method, and derive differential inclusions that model these algorithms in the continuous-time limit. Through a nonsmooth Lyapunov analysis, we obtain rates of convergence for these dynamical systems in the convex and strongly convex settings that illustrate an interesting tradeoff between Nesterov and heavy ball types of acceleration.Comment: several corrections and improvements, new numerical sectio