Dictionary Learning under Symmetries via Group Representations

The dictionary learning problem can be viewed as a data-driven process to learn a suitable transformation so that data is sparsely represented directly from example data. In this paper, we examine the problem of learning a dictionary that is invariant under a pre-specified group of transformations. Natural settings include Cryo-EM, multi-object tracking, synchronization, pose estimation, etc. We specifically study this problem under the lens of mathematical representation theory. Leveraging the power of non-abelian Fourier analysis for functions over compact groups, we prescribe an algorithmic recipe for learning dictionaries that obey such invariances. We relate the dictionary learning problem in the physical domain, which is naturally modelled as being infinite dimensional, with the associated computational problem, which is necessarily finite dimensional. We establish that the dictionary learning problem can be effectively understood as an optimization instance over certain matrix orbitopes having a particular block-diagonal structure governed by the irreducible representations of the group of symmetries. This perspective enables us to introduce a band-limiting procedure which obtains dimensionality reduction in applications. We provide guarantees for our computational ansatz to provide a desirable dictionary learning outcome. We apply our paradigm to investigate the dictionary learning problem for the groups SO(2) and SO(3). While the SO(2)-orbitope admits an exact spectrahedral description, substantially less is understood about the SO(3)-orbitope. We describe a tractable spectrahedral outer approximation of the SO(3)-orbitope, and contribute an alternating minimization paradigm to perform optimization in this setting. We provide numerical experiments to highlight the efficacy of our approach in learning SO(3)-invariant dictionaries, both on synthetic and on real world data.Comment: 29 pages, 2 figure

Variational data assimilation for two interface problems

“Variational data assimilation (VDA) is a process that uses optimization techniques to determine an initial condition of a dynamical system such that its evolution best fits the observed data. In this dissertation, we develop and analyze the variational data assimilation method with finite element discretization for two interface problems, including the Parabolic Interface equation and the Stokes-Darcy equation with the Beavers-Joseph interface condition. By using Tikhonov regularization and formulating the VDA into an optimization problem, we establish the existence, uniqueness and stability of the optimal solution for each concerned case. Based on weak formulations of the Parabolic Interface equation and Stokes-Darcy equation, the dual method and Lagrange multiplier rule are utilized to derive the first order optimality system (OptS) for both the continuous and discrete VDA problems, where the discrete data assimilations are built on certain finite element discretization in space and the backward Euler scheme in time. By introducing auxiliary equations, rescaling the optimality system, and employing other subtle analysis skills, we present the finite element convergence estimation for each case with special attention paid to recovering the properties missed in between the continuous and discrete OptS. Moreover, to efficiently solve the OptS, we present two classical gradient methods, the steepest descent method and the conjugate gradient method, to reduce the computational cost for well-stabilized and ill-stabilized VDA problems, respectively. Furthermore, we propose the time parallel algorithm and proper orthogonal decomposition method to further optimize the computing efficiency. Finally, numerical results are provided to validate the proposed methods”--Abstract, page iii

Sensor tasking utilizing deep reinforcement learning in a random finite set framework

There is a growing need to increase the capabilities of existing sensor arrays to monitor a large amount of space objects orbiting the Earth with a limited number of opportunities to observe these objects. Due to geopolitical considerations and financial cost, it is infeasible to create an array of sensors that can monitor each space object and accurately describe its state. Instead of brute force techniques by increasing the number of sensors worldwide, the current advancements in computational capability along with new algorithms for multi-target filtering and reinforcement learning has allowed a pathway to begin solving the non-myopic, heterogenous sensor tasking problem. This work employs the labeled multi-Bernoulli filter in conjunction with advanced, deep reinforcement learning techniques such as the policy gradient Q-learning algorithm and deep Q-networks. The filter and reinforcement learning techniqures are used together to track ten targets in geosynchronous orbit, while a linear Kalman filter and the reinforcement learning techniques are used to evaluate their effectiveness in multi-agent learning scenarios. The future deployment of these algorithms and their specific logistical considerations are also discussed with potential solutions.Aerospace Engineerin

When Are Nonconvex Optimization Problems Not Scary?

Nonconvex optimization is NP-hard, even the goal is to compute a local minimizer. In applied disciplines, however, nonconvex problems abound, and simple algorithms, such as gradient descent and alternating direction, are often surprisingly effective. The ability of simple algorithms to find high-quality solutions for practical nonconvex problems remains largely mysterious. This thesis focuses on a class of nonconvex optimization problems which CAN be solved to global optimality with polynomial-time algorithms. This class covers natural nonconvex formulations of central problems in signal processing, machine learning, and statistical estimation, such as sparse dictionary learning (DL), generalized phase retrieval (GPR), and orthogonal tensor decomposition. For each of the listed problems, the nonconvex formulation and optimization lead to novel and often improved computational guarantees. This class of nonconvex problems has two distinctive features: (i) All local minimizer are also global. Thus obtaining any local minimizer solves the optimization problem; (ii) Around each saddle point or local maximizer, the function has a negative directional curvature. In other words, around these points, the Hessian matrices have negative eigenvalues. We call smooth functions with these two properties (qualitative) X functions, and derive concrete quantities and strategy to help verify the properties, particularly for functions with random inputs or parameters. As practical examples, we establish that certain natural nonconvex formulations for complete DL and GPR are X functions with concrete parameters. Optimizing X functions amounts to finding any local minimizer. With generic initializations, typical iterative methods at best only guarantee to converge to a critical point that might be a saddle point or local maximizer. Interestingly, the X structure allows a number of iterative methods to escape from saddle points and local maximizers and efficiently find a local minimizer, without special initializations. We choose to describe and analyze the second-order trust-region method (TRM) that seems to yield the strongest computational guarantees. Intuitively, second-order methods can exploit Hessian to extract negative curvature directions around saddle points and local maximizers, and hence are able to successfully escape from the saddles and local maximizers of X functions. We state the TRM in a Riemannian optimization framework to cater to practical manifold-constrained problems. For DL and GPR, we show that under technical conditions, the TRM algorithm finds a global minimizer in a polynomial number of steps, from arbitrary initializations

Active Learning in Cognitive Radio Networks

In this thesis, numerous Machine Learning (ML) applications for Cognitive Radios Networks (CRNs) are developed and presented which facilitate the e cient spectral coexistence of a legacy system, the Primary Users (PUs), and a CRN, the Secondary Users (SUs). One way to better exploit the capacity of the legacy system frequency band is to consider a coexistence scenario using underlay Cognitive Radio (CR) techniques, where SUs may transmit in the frequency band of the PU system as long as the induced to the PU interference is under a certain limit and thus does not harmfully a ect the legacy system operability

Variational Methods for the Estimation of Transport Fields with Application to the Recovery of Physics-Based Optical Flows Across Boundaries

In this thesis we develop a method for the estimation of the flow behaviour of an incom- pressible fluid based on observations of the brightness intensity of a transported visible substance which does not influence the flow. The observations are given in a subregion of the flow as a sequence of discrete images with in- and outflow across the image boundaries. The resulting mathematical problem is ill-posed and has to be regularised with information of the underlying fluid flow model. We consider a constrained optimisation problem, namely the minimisation of a tracking type data term for the brightness distribution and a regularisation term subject to a system of weakly coupled partial differential equations. The system consists of the time- dependent incompressible Navier-Stokes equations coupled by the velocity vector field to a convection-diffusion equation, which describes the transport of brightness patterns in the image sequence. Due to the flow across the boundaries of the computational domain we solve a boundary identification problem. The usage of (strong) Dirichlet boundary controls for this purpose leads to theoretical and numerical complications, so that we will instead use Robin-type controls, which allow for a more convenient theoretical and numerical framework. We will prove well-posedness and investigate the functionality of the proposed approach by means of numerical examples. Furthermore, we discuss the connection to Dirichlet-control problems, e. g. the approximation of Dirichlet-controls by the so-called penalised Neumann method, which is based on the Robin-type controls for a varying penalty parameter. We will show via numerical tests that Robin-type controls are suitable for the identifi- cation of the correct fluid flow. Moreover, the examples indicate that the underlying physical model used for the regularisation influences the flow reconstruction process. Thus appropriate knowledge of the model is essential, e. g. the viscosity parameter. For a time- independent example we will present a heuristic, which, beside the boundary identification, automatically evaluates the viscosity in case the parameter is unknown. The developed physics-based optical flow estimation approach is finally used for the data set of a prototypical application. The background of the application is the approximation of horizontal wind fields in sparsely populated areas like desert regions. A sequence of satellite images documenting the brightness intensity of an observable substance distributed by the wind (e. g. dust plumes) is thereby assumed to be the only available data. Wind field information is for example needed to simulate the distribution of other, not directly observ- able, substances in the lower atmosphere. For the prototypical example we compute a high quality reconstruction of the underlying fluid flow by a (discrete) sequence of consecutive spatially distributed brightness intensities. Thereby, we compare three different models (heat equation, Stokes system and the original fluid flow model) in the reconstruction process and show that using as much model knowledge as possible is essential for a good reconstruction result

A Themed Issue Dedicated to Professor John B. Goodenough on the Occasion of His 100th Birthday Anniversary

This book of Molecules is dedicated to Professor John B. Goodenough (born July 25, 1922, Jena, Germany), an American physicist, who won the 2019 Nobel Prize for Chemistry for his work on developing lithium-ion batteries

Shell nouns : in a systemic functional linguistics perspective

Tese de doutoramento, Linguística (Análise do Discurso), Universidade de Lisboa, Faculdade de Letras, 2015Shell nouns in a Systemic Functional Linguistics perspective. The aim of this thesis is to develop an account of shell nouns (Schmid, 2000) in a Systemic Functional Linguistics (SFL) perspective. Using a parallel corpus comprising five article submissions by Portuguese academics in the field of economics and five published articles on comparable topics, the ideational, interpersonal and textual functions of shell nouns are tagged at the strata of the lexicogrammar and discourse semantics using Corpus Tool version 2.7.4 (O’Donnell, 2008). The systems networks used to tag the corpus are grounded in SFL theory. The analysis shows that shell nouns constitute an important systemic resource for the writers of research articles, who need to build an argument, positioning themselves and their study to convince the discourse community that their paper makes a contribution to knowledge in their disciplinary field. They enable a text to unfold by compacting information realised as a clause or more elsewhere in the text. Thus they can help scaffold a text through hyper-Themes, hyper-News and internal conjunction. At the stratum of the lexicogrammar, anaphorically referring nominal groups with a shell noun as Head often compose Theme, where they constitute a shared point of departure for the clause. In a decoding relational clause whose Process is realised by a verb such as reveal, confirm, or suggest, an anaphorically referring shell noun that construes Token helps to explicitly build the writer’s argument. Shell nouns that construe the field of research, such as results and findings are common in this function. Mental, linguistic and factual shell nouns contribute to construing dialogic position, and coupling between interpersonal systems and textual systems enables the writer to align the reader with certain positions and disalign with others. Although most shell nouns are not field specific, because they can project a figure that instantiates an entity, they contribute to construing field, for example instantiating entities as the object of study of the empirical research. The capacity of shell nouns to function as described above derives from their status as semiotic abstractions, which can refer to text as fact or report and are grammatical metaphors. They can be seen as lying at the intersection of modality and the logico-semantic relations of projection and expansion, brought into being by the semogenic process of nominalisation. The writers of the published articles and article submissions are found to use shell nouns in all of the functions above, but there are differences in the relative shares of the functions, which may affect reader reactions to the text