Singular value decay of operator-valued differential Lyapunov and Riccati equations

We consider operator-valued differential Lyapunov and Riccati equations, where the operators $B$ and $C$ may be relatively unbounded with respect to $A$ (in the standard notation). In this setting, we prove that the singular values of the solutions decay fast under certain conditions. In fact, the decay is exponential in the negative square root if $A$ generates an analytic semigroup and the range of $C$ has finite dimension. This extends previous similar results for algebraic equations to the differential case. When the initial condition is zero, we also show that the singular values converge to zero as time goes to zero, with a certain rate that depends on the degree of unboundedness of $C$. A fast decay of the singular values corresponds to a low numerical rank, which is a critical feature in large-scale applications. The results reported here provide a theoretical foundation for the observation that, in practice, a low-rank factorization usually exists.Comment: Corrected some misconceptions, which lead to more general results (e.g. exponential stability is no longer required). Also fixed some off-by-one errors, improved the presentation, and added/extended several remarks on possible generalizations. Now 22 pages, 8 figure

Adaptive high-order splitting schemes for large-scale differential Riccati equations

We consider high-order splitting schemes for large-scale differential Riccati equations. Such equations arise in many different areas and are especially important within the field of optimal control. In the large-scale case, it is critical to employ structural properties of the matrix-valued solution, or the computational cost and storage requirements become infeasible. Our main contribution is therefore to formulate these high-order splitting schemes in a efficient way by utilizing a low-rank factorization. Previous results indicated that this was impossible for methods of order higher than 2, but our new approach overcomes these difficulties. In addition, we demonstrate that the proposed methods contain natural embedded error estimates. These may be used e.g. for time step adaptivity, and our numerical experiments in this direction show promising results.Comment: 23 pages, 7 figure

Finite element convergence analysis for the thermoviscoelastic Joule heating problem

We consider a system of equations that model the temperature, electric potential and deformation of a thermoviscoelastic body. A typical application is a thermistor; an electrical component that can be used e.g. as a surge protector, temperature sensor or for very precise positioning. We introduce a full discretization based on standard finite elements in space and a semi-implicit Euler-type method in time. For this method we prove optimal convergence orders, i.e. second-order in space and first-order in time. The theoretical results are verified by several numerical experiments in two and three dimensions.Comment: 20 pages, 6 figures, 2 table

Multiscale differential Riccati equations for linear quadratic regulator problems

We consider approximations to the solutions of differential Riccati equations in the context of linear quadratic regulator problems, where the state equation is governed by a multiscale operator. Similarly to elliptic and parabolic problems, standard finite element discretizations perform poorly in this setting unless the grid resolves the fine-scale features of the problem. This results in unfeasible amounts of computation and high memory requirements. In this paper, we demonstrate how the localized orthogonal decomposition method may be used to acquire accurate results also for coarse discretizations, at the low cost of solving a series of small, localized elliptic problems. We prove second-order convergence (except for a logarithmic factor) in the $L^2$ operator norm, and first-order convergence in the corresponding energy norm. These results are both independent of the multiscale variations in the state equation. In addition, we provide a detailed derivation of the fully discrete matrix-valued equations, and show how they can be handled in a low-rank setting for large-scale computations. In connection to this, we also show how to efficiently compute the relevant operator-norm errors. Finally, our theoretical results are validated by several numerical experiments.Comment: Accepted for publication in SIAM J. Sci. Comput. This version differs from the previous one only by the addition of Remark 7.2 and minor changes in formatting. 21 pages, 12 figure

Sub-linear convergence of a tamed stochastic gradient descent method in Hilbert space

In this paper, we introduce the tamed stochastic gradient descent method (TSGD) for optimization problems. Inspired by the tamed Euler scheme, which is a commonly used method within the context of stochastic differential equations, TSGD is an explicit scheme that exhibits stability properties similar to those of implicit schemes. As its computational cost is essentially equivalent to that of the well-known stochastic gradient descent method (SGD), it constitutes a very competitive alternative to such methods. We rigorously prove (optimal) sub-linear convergence of the scheme for strongly convex objective functions on an abstract Hilbert space. The analysis only requires very mild step size restrictions, which illustrates the good stability properties. The analysis is based on a priori estimates more frequently encountered in a time integration context than in optimization, and this alternative approach provides a different perspective also on the convergence of SGD. Finally, we demonstrate the usability of the scheme on a problem arising in a context of supervised learning

Low-rank second-order splitting of large-scale differential Riccati equations

Abstract-We apply first-and second-order splitting schemes to the differential Riccati equation. Such equations are very important in e.g. linear quadratic regulator (LQR) problems, where they provide a link between the state of the system and the optimal input. The methods can also be extended to generalized Riccati equations, e.g. arising from LQR problems given in implicit form. In contrast to previously proposed schemes such as BDF or Rosenbrock methods, the splitting schemes exploit the fact that the nonlinear and affine parts of the problem, when considered in isolation, have closed-form solutions. We show that if the solution possesses low-rank structure, which is frequently the case, then this is preserved by the method. This feature is used to implement the methods efficiently for large-scale problems. The proposed methods are expected to be competitive, as they at most require the solution of a small number of linear equation systems per time step. Finally, we apply our low-rank implementations to the Riccati equations arising from two LQR problems. The results show that the rank of the solutions stay low, and the expected orders of convergence are observed

Convergence analysis for splitting of the abstract differential Riccati equation

We consider a splitting-based approximation of the abstract differential Riccati equation in the setting of Hilbert--Schmidt operators. The Riccati equation arises in many different areas and is important within the field of optimal control. In this paper we conduct a temporal error analysis and prove that the splitting method converges with the same order as the implicit Euler scheme, under the same low regularity requirements on the initial values. For a subsequent spatial discretization, the abstract setting also yields uniform temporal error bounds with respect to the spatial discretization parameter. The spatial discretizations commonly lead to large-scale problems, where the use of structural properties of the solution is essential. We therefore conclude by proving that the splitting method preserves low-rank structure in the matrix-valued case. Numerical results demonstrate the validity of the convergence analysis