The dictionary learning problem can be viewed as a data-driven process to
learn a suitable transformation so that data is sparsely represented directly
from example data. In this paper, we examine the problem of learning a
dictionary that is invariant under a pre-specified group of transformations.
Natural settings include Cryo-EM, multi-object tracking, synchronization, pose
estimation, etc. We specifically study this problem under the lens of
mathematical representation theory. Leveraging the power of non-abelian Fourier
analysis for functions over compact groups, we prescribe an algorithmic recipe
for learning dictionaries that obey such invariances. We relate the dictionary
learning problem in the physical domain, which is naturally modelled as being
infinite dimensional, with the associated computational problem, which is
necessarily finite dimensional. We establish that the dictionary learning
problem can be effectively understood as an optimization instance over certain
matrix orbitopes having a particular block-diagonal structure governed by the
irreducible representations of the group of symmetries. This perspective
enables us to introduce a band-limiting procedure which obtains dimensionality
reduction in applications. We provide guarantees for our computational ansatz
to provide a desirable dictionary learning outcome. We apply our paradigm to
investigate the dictionary learning problem for the groups SO(2) and SO(3).
While the SO(2)-orbitope admits an exact spectrahedral description,
substantially less is understood about the SO(3)-orbitope. We describe a
tractable spectrahedral outer approximation of the SO(3)-orbitope, and
contribute an alternating minimization paradigm to perform optimization in this
setting. We provide numerical experiments to highlight the efficacy of our
approach in learning SO(3)-invariant dictionaries, both on synthetic and on
real world data.Comment: 29 pages, 2 figure