A Riemannian Fletcher-Reeves Conjugate Gradient Method for Doubly Stochastic Inverse Eigenvalue Problems

published_or_final_versio

Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold

The conjugate gradient method is a crucial first-order optimization method that generally converges faster than the steepest descent method, and its computational cost is much lower than the second-order methods. However, while various types of conjugate gradient methods have been studied in Euclidean spaces and on Riemannian manifolds, there has little study for those in distributed scenarios. This paper proposes a decentralized Riemannian conjugate gradient descent (DRCGD) method that aims at minimizing a global function over the Stiefel manifold. The optimization problem is distributed among a network of agents, where each agent is associated with a local function, and communication between agents occurs over an undirected connected graph. Since the Stiefel manifold is a non-convex set, a global function is represented as a finite sum of possibly non-convex (but smooth) local functions. The proposed method is free from expensive Riemannian geometric operations such as retractions, exponential maps, and vector transports, thereby reducing the computational complexity required by each agent. To the best of our knowledge, DRCGD is the first decentralized Riemannian conjugate gradient algorithm to achieve global convergence over the Stiefel manifold

Riemannian Optimization for Convex and Non-Convex Signal Processing and Machine Learning Applications

The performance of most algorithms for signal processing and machine learning applications highly depends on the underlying optimization algorithms. Multiple techniques have been proposed for solving convex and non-convex problems such as interior-point methods and semidefinite programming. However, it is well known that these algorithms are not ideally suited for large-scale optimization with a high number of variables and/or constraints. This thesis exploits a novel optimization method, known as Riemannian optimization, for efficiently solving convex and non-convex problems with signal processing and machine learning applications. Unlike most optimization techniques whose complexities increase with the number of constraints, Riemannian methods smartly exploit the structure of the search space, a.k.a., the set of feasible solutions, to reduce the embedded dimension and efficiently solve optimization problems in a reasonable time. However, such efficiency comes at the expense of universality as the geometry of each manifold needs to be investigated individually. This thesis explains the steps of designing first and second-order Riemannian optimization methods for smooth matrix manifolds through the study and design of optimization algorithms for various applications. In particular, the paper is interested in contemporary applications in signal processing and machine learning, such as community detection, graph-based clustering, phase retrieval, and indoor and outdoor location determination. Simulation results are provided to attest to the efficiency of the proposed methods against popular generic and specialized solvers for each of the above applications

The Modelling of Biological Growth: a Pattern Theoretic Approach

Mathematical and statistical modeling and analysis of biological growth using images collected over time are important for understanding of normal and abnormal development. In computational anatomy, changes in the shape of a growing anatomical structure have been modeled by means of diffeomorphic transformations in the background coordinate space. Various image and landmark matching algorithms have been developed for inference of large transformations that perform image registration consistent with the material properties of brain anatomy under study. However, from a biological perspective, it is not material constants that regulate growth, it is the genetic control system. A pattern theoretic model called the Growth as Random Iterated Diffeomorphisims (GRID) introduced by Ulf Grenander (Brown University) constructs growth-induced transformations according to fundamental biological principles of growth. They are governed by an underlying genetic control that is expressed in terms of probability laws governing the spatial-temporal patterns of elementary cell decisions (e.g., cell division/death). This thesis addresses computational and stochastic aspects of the GRID model and develops its application to image analysis of growth. The first part of the thesis introduces the original GRID view of growth-induced deformation on a fine time scale as a composition of several, elementary, local deformations each resulting from a random cell decision, a highly localized event in space-time called a seed. A formalization of the proposed model using theory of stochastic processes is presented, namely, an approximation of the GRID model by the diffusion process and the Fokker-Planck equation describing the evolution of the probability density of seed trajectories in space-time. Its time-dependent and stationary numerical solutions reveal bimodal distribution of a random seed trajectory in space-time. The second part of the thesis considers the growth pattern on a coarse time scale which underlies visible shape changes seen in images. It is shown that such a "macroscopic" growth pattern is a solution to a deterministic integro-differential equation in the form of a diffeomorphic flow dependent on the GRID growth variables such as the probability density of cell decisions and the rate of contraction/expansion. Since the GRID variables are unobserved, they have to be estimated from image data. Using the GRID macroscopic growth equation such an estimation problem is formulated as an optimal control problem. The estimated GRID variables are optimal controls that force the image of an initial organism to be continuously transformed into the image of a grown organism. The GRID-based inference method is implemented for inference of growth properties of the Drosophila wing disc directly from confocal micrographs of Wingless gene expression patterns