First order sensitivity analysis of symplectic eigenvalues

For every $2n \times 2n$ positive definite matrix $A$ there are $n$ positive numbers $d_1(A) \leq \ldots \leq d_n(A)$ associated with $A$ called the symplectic eigenvalues of $A.$ It is known that $d_m$ are continuous functions of $A$ but are not differentiable in general. In this paper, we show that the directional derivative of $d_m$ exists and derive its expression. We also discuss various subdifferential properties of $d_m$ such as Clarke and Michel-Penot subdifferentials.Comment: 24 page

Spectrally Constrained Optimization

We investigate how to solve smooth matrix optimization problems with general linear inequality constraints on the eigenvalues of a symmetric matrix. We present solution methods to obtain exact global minima for linear objective functions, i.e., $F(X) = \langle C, X \rangle$, and perform exact projections onto the eigenvalue constraint set. Two first-order algorithms are developed to obtain first-order stationary points for general non-convex objective functions. Both methods are proven to converge sublinearly when the constraint set is convex. Numerical experiments demonstrate the applicability of both the model and the methods.Comment: 32 pages, 2 figures, 2 table

Subdiferencijal svojstvene vrijednosti simetrične matrice

U ovom radu proširen je pojam diferencijabilnosti lokalno Lipschitzovih funkcija sa $\mathbb{R}^{n}$ u $\mathbb{R}$. Prvo promatramo derivaciju u smjeru poznatu iz klasične analize i navodimo neka njena svojstva, kao što su na primjer, ograničenost, Lipschitzovost i sublinearnost. Nakon toga definiramo subdiferencijal konveksne funkcije u točki $x \in \mathbb{R}^n$. Taj skup je neprazan, konveksan i kompaktan, a u slučaju da funkcija nije konveksna, može biti i prazan. U nastavku promatramo općenite lokalno Lipschitzove, ne nužno konveksne funkcije i definiramo generaliziranu Clarkeovu derivaciju u smjeru i Clarkeov subdiferencijal. Navodimo neka njihova svojstva i uspoređujemo s običnom derivacijom u smjeru. Kao i u klasičnoj analizi za diferencijabilne funkcije, i za generaliziranu derivaciju u smjeru postoje pravila računanja za linearnu kombinaciju, produkt, kvocijent i kompoziciju. No, u općenitom slučaju vrijedi samo inkluzija medu pripadnim subdiferencijalima. Zatim navodimo teorem o subdiferencijalu max funkcije i generalizirani teorem srednje vrijednosti. Koristeći rezultate iz prethodnih poglavlja, računamo subdiferencijal i derivaciju najveće svojstvene vrijednosti realne simetrične matrice. Zatim uvodimo Michel-Penotovu derivaciju u smjeru i pripadni Michel-Penotov subdiferencijal i pomoću nekoliko rezultata dolazimo do zaključka da se Clarkeov i Michel-Penotov subdiferencijal podudaraju za funkciju m-te najveće svojstvene vrijednosti realne simetrične matrice.In this master thesis we study the extended notion of diferentiability of real locally Lipschitz functions defined on space $\mathbb{R}^{n}$. In the first part we study the well known directional derivative of convex functions and its properties, some of which are, for instance, boundness, locally Lipschitz continuity and sublinearity. After that, the subdifferential of convex function in $x \in \mathbb{R}^n$ is defined, and we show that it is a nonempty, convex and compact set, which can be empty if the function f is not convex. Next, we study the Clarke’s directional derivative and Clarke’s subdifferential. We list some of their properties and compare them to the usual directional derivative. Similarly to classical analysis, there are calculus rules in nonsmooth analysis for Clarke’s subdifferentials, such as for linear combination of functions, product, quotient and chain rule, but in the case of Clarke subdifferential, only one inclusion can be showed. We also generalize some important results, such as mean value theorem and max-function theorem. In the last section we calculate the subdifferential and directional derivative of the largest eigenvalue of the real symmetric matrix with the help of the results from the previous chapters. Another generalized derivative is defined, called the Michel-Penot’s directional derivative. After defining the Michel-Penot’s subdifferential we show that Clarke’s and Michel-Penot’s subdifferentials coincide for mth largest eigenvalue of a real symmetric matrix

Robust network formation with biological applications

We provide new results on the structure of optimal transportation networks obtained as minimizers of an energy cost functional consisting of a kinetic (pumping) and material (metabolic) cost terms, constrained by a local mass conservation law. In particular, we prove that every tree (i.e., graph without loops) represents a local minimizer of the energy with concave metabolic cost. For the linear metabolic cost, we prove that the set of minimizers contains a loop-free structure. Moreover, we enrich the energy functional such that it accounts also for robustness of the network, measured in terms of the Fiedler number of the graph with edge weights given by their conductivities. We examine fundamental properties of the modified functional, in particular, its convexity and differentiability. We provide analytical insights into the new model by considering two simple examples. Subsequently, we employ the projected subgradient method to find global minimizers of the modified functional numerically. We then present two numerical examples, illustrating how the optimal graph's structure and energy expenditure depend on the required robustness of the network.Comment: 26 pages, 7 figures, 1 tabl

Subdiferencijal svojstvene vrijednosti simetrične matrice

U ovom radu proširen je pojam diferencijabilnosti lokalno Lipschitzovih funkcija sa $\mathbb{R}^{n}$ u $\mathbb{R}$. Prvo promatramo derivaciju u smjeru poznatu iz klasične analize i navodimo neka njena svojstva, kao što su na primjer, ograničenost, Lipschitzovost i sublinearnost. Nakon toga definiramo subdiferencijal konveksne funkcije u točki $x \in \mathbb{R}^n$. Taj skup je neprazan, konveksan i kompaktan, a u slučaju da funkcija nije konveksna, može biti i prazan. U nastavku promatramo općenite lokalno Lipschitzove, ne nužno konveksne funkcije i definiramo generaliziranu Clarkeovu derivaciju u smjeru i Clarkeov subdiferencijal. Navodimo neka njihova svojstva i uspoređujemo s običnom derivacijom u smjeru. Kao i u klasičnoj analizi za diferencijabilne funkcije, i za generaliziranu derivaciju u smjeru postoje pravila računanja za linearnu kombinaciju, produkt, kvocijent i kompoziciju. No, u općenitom slučaju vrijedi samo inkluzija medu pripadnim subdiferencijalima. Zatim navodimo teorem o subdiferencijalu max funkcije i generalizirani teorem srednje vrijednosti. Koristeći rezultate iz prethodnih poglavlja, računamo subdiferencijal i derivaciju najveće svojstvene vrijednosti realne simetrične matrice. Zatim uvodimo Michel-Penotovu derivaciju u smjeru i pripadni Michel-Penotov subdiferencijal i pomoću nekoliko rezultata dolazimo do zaključka da se Clarkeov i Michel-Penotov subdiferencijal podudaraju za funkciju m-te najveće svojstvene vrijednosti realne simetrične matrice.In this master thesis we study the extended notion of diferentiability of real locally Lipschitz functions defined on space $\mathbb{R}^{n}$. In the first part we study the well known directional derivative of convex functions and its properties, some of which are, for instance, boundness, locally Lipschitz continuity and sublinearity. After that, the subdifferential of convex function in $x \in \mathbb{R}^n$ is defined, and we show that it is a nonempty, convex and compact set, which can be empty if the function f is not convex. Next, we study the Clarke’s directional derivative and Clarke’s subdifferential. We list some of their properties and compare them to the usual directional derivative. Similarly to classical analysis, there are calculus rules in nonsmooth analysis for Clarke’s subdifferentials, such as for linear combination of functions, product, quotient and chain rule, but in the case of Clarke subdifferential, only one inclusion can be showed. We also generalize some important results, such as mean value theorem and max-function theorem. In the last section we calculate the subdifferential and directional derivative of the largest eigenvalue of the real symmetric matrix with the help of the results from the previous chapters. Another generalized derivative is defined, called the Michel-Penot’s directional derivative. After defining the Michel-Penot’s subdifferential we show that Clarke’s and Michel-Penot’s subdifferentials coincide for mth largest eigenvalue of a real symmetric matrix

Subdiferencijal svojstvene vrijednosti simetrične matrice

U ovom radu proširen je pojam diferencijabilnosti lokalno Lipschitzovih funkcija sa $\mathbb{R}^{n}$ u $\mathbb{R}$. Prvo promatramo derivaciju u smjeru poznatu iz klasične analize i navodimo neka njena svojstva, kao što su na primjer, ograničenost, Lipschitzovost i sublinearnost. Nakon toga definiramo subdiferencijal konveksne funkcije u točki $x \in \mathbb{R}^n$. Taj skup je neprazan, konveksan i kompaktan, a u slučaju da funkcija nije konveksna, može biti i prazan. U nastavku promatramo općenite lokalno Lipschitzove, ne nužno konveksne funkcije i definiramo generaliziranu Clarkeovu derivaciju u smjeru i Clarkeov subdiferencijal. Navodimo neka njihova svojstva i uspoređujemo s običnom derivacijom u smjeru. Kao i u klasičnoj analizi za diferencijabilne funkcije, i za generaliziranu derivaciju u smjeru postoje pravila računanja za linearnu kombinaciju, produkt, kvocijent i kompoziciju. No, u općenitom slučaju vrijedi samo inkluzija medu pripadnim subdiferencijalima. Zatim navodimo teorem o subdiferencijalu max funkcije i generalizirani teorem srednje vrijednosti. Koristeći rezultate iz prethodnih poglavlja, računamo subdiferencijal i derivaciju najveće svojstvene vrijednosti realne simetrične matrice. Zatim uvodimo Michel-Penotovu derivaciju u smjeru i pripadni Michel-Penotov subdiferencijal i pomoću nekoliko rezultata dolazimo do zaključka da se Clarkeov i Michel-Penotov subdiferencijal podudaraju za funkciju m-te najveće svojstvene vrijednosti realne simetrične matrice.In this master thesis we study the extended notion of diferentiability of real locally Lipschitz functions defined on space $\mathbb{R}^{n}$. In the first part we study the well known directional derivative of convex functions and its properties, some of which are, for instance, boundness, locally Lipschitz continuity and sublinearity. After that, the subdifferential of convex function in $x \in \mathbb{R}^n$ is defined, and we show that it is a nonempty, convex and compact set, which can be empty if the function f is not convex. Next, we study the Clarke’s directional derivative and Clarke’s subdifferential. We list some of their properties and compare them to the usual directional derivative. Similarly to classical analysis, there are calculus rules in nonsmooth analysis for Clarke’s subdifferentials, such as for linear combination of functions, product, quotient and chain rule, but in the case of Clarke subdifferential, only one inclusion can be showed. We also generalize some important results, such as mean value theorem and max-function theorem. In the last section we calculate the subdifferential and directional derivative of the largest eigenvalue of the real symmetric matrix with the help of the results from the previous chapters. Another generalized derivative is defined, called the Michel-Penot’s directional derivative. After defining the Michel-Penot’s subdifferential we show that Clarke’s and Michel-Penot’s subdifferentials coincide for mth largest eigenvalue of a real symmetric matrix

Morse inequalities for ordered eigenvalues of generic families of self-adjoint matrices

In many applied problems one seeks to identify and count the critical points of a particular eigenvalue of a smooth parametric family of self-adjoint matrices, with the parameter space often being known and simple, such as a torus. Among particular settings where such a question arises are the Floquet--Bloch decomposition of periodic Schroedinger operators, topology of potential energy surfaces in quantum chemistry, spectral optimization problems such as minimal spectral partitions of manifolds, as well as nodal statistics of graph eigenfunctions. In contrast to the classical Morse theory dealing with smooth functions, the eigenvalues of families of self-adjoint matrices are not smooth at the points corresponding to repeated eigenvalues (called, depending on the application and on the dimension of the parameter space, the diabolical/Dirac/Weyl points or the conical intersections). This work develops a procedure for associating a Morse polynomial to a point of eigenvalue multiplicity; it utilizes the assumptions of smoothness and self-adjointness of the family to provide concrete answers. In particular, we define the notions of non-degenerate topologically critical point and generalized Morse family, establish that generalized Morse families are generic in an appropriate sense, establish a differential first-order conditions for criticality, as well as compute the local contribution of a topologically critical point to the Morse polynomial. Remarkably, the non-smooth contribution to the Morse polynomial turns out to be universal: it depends only on the size of the eigenvalue multiplicity and the relative position of the eigenvalue of interest and not on the particulars of the operator family; it is expressed in terms of the homologies of Grassmannians.Comment: 38 page

Geometric numerical integration for optimisation

In this thesis, we study geometric numerical integration for the optimisation of various classes of functionals. Numerical integration and the study of systems of differential equations have received increased attention within the optimisation community in the last decade, as a means for devising new optimisation schemes as well as to improve our understanding of the dynamics of existing schemes. Discrete gradient methods from geometric numerical integration preserve structures of first-order gradient systems, including the dissipative structure of schemes such as gradient flows, and thus yield iterative methods that are unconditionally dissipative, i.e. decrease the objective function value for all time steps. We look at discrete gradient methods for optimisation in several settings. First, we provide a comprehensive study of discrete gradient methods for optimisation of continuously differentiable functions. In particular, we prove properties such as well-posedness of the discrete gradient update equation, convergence rates, convergence of the iterates, and propose methods for solving the discrete gradient update equation with superior stability and convergence rates. Furthermore, we present results from numerical experiments which support the theory. Second, motivated by the existence of derivative-free discrete gradients, and seeking to solve nonsmooth optimisation problems and more generally black-box problems, including for parameter optimisation problems, we propose methods based on the Itoh--Abe discrete gradient method for solving nonconvex, nonsmooth optimisation problems with derivative-free methods. In this setting, we prove well-posedness of the method, and convergence guarantees within the nonsmooth, nonconvex Clarke subdifferential framework for locally Lipschitz continuous functions. The analysis is shown to hold in various settings, namely in the unconstrained and constrained setting, including epi-Lipschitzian constraints, and for stochastic and deterministic optimisation methods. Building on the work of derivative-free discrete gradient methods and the concept of structure preservation in geometric numerical integration, we consider discrete gradient methods applied to other differential systems with dissipative structures. In particular, we study the inverse scale space flow, linked to the well-known Bregman methods, which are central to variational optimisation problems and regularisation methods for inverse problems. In this setting, we propose and implement derivative-free schemes that exploit structures such as sparsity to achieve superior convergence rates in numerical experiments, and prove convergence guarantees for these methods in the nonsmooth, nonconvex setting. Furthermore, these schemes can be seen as generalisations of the Gauss-Seidel method and successive-over-relaxation. Finally, we return to parameter optimisation problems, namely nonsmooth bilevel optimisation problems, and propose a framework to employ first-order methods for these problems, when the underlying variational optimisation problem admits a nonsmooth structure in the partial smoothness framework. In this setting, we prove piecewise differentiability of the parameter-dependent solution mapping, and study algorithmic differentiation approaches to evaluating the derivatives. Furthermore, we prove that the algorithmic derivatives converge to the implicit derivatives. Thus we demonstrate that, although some parameter tuning problems must inevitably be treated as black-box optimisation problems, for a large number of variational problems one can exploit the structure of nonsmoothness to perform gradient-based bilevel optimisation