69 research outputs found
Convolutional Analysis Operator Learning: Acceleration and Convergence
Convolutional operator learning is gaining attention in many signal
processing and computer vision applications. Learning kernels has mostly relied
on so-called patch-domain approaches that extract and store many overlapping
patches across training signals. Due to memory demands, patch-domain methods
have limitations when learning kernels from large datasets -- particularly with
multi-layered structures, e.g., convolutional neural networks -- or when
applying the learned kernels to high-dimensional signal recovery problems. The
so-called convolution approach does not store many overlapping patches, and
thus overcomes the memory problems particularly with careful algorithmic
designs; it has been studied within the "synthesis" signal model, e.g.,
convolutional dictionary learning. This paper proposes a new convolutional
analysis operator learning (CAOL) framework that learns an analysis sparsifying
regularizer with the convolution perspective, and develops a new convergent
Block Proximal Extrapolated Gradient method using a Majorizer (BPEG-M) to solve
the corresponding block multi-nonconvex problems. To learn diverse filters
within the CAOL framework, this paper introduces an orthogonality constraint
that enforces a tight-frame filter condition, and a regularizer that promotes
diversity between filters. Numerical experiments show that, with sharp
majorizers, BPEG-M significantly accelerates the CAOL convergence rate compared
to the state-of-the-art block proximal gradient (BPG) method. Numerical
experiments for sparse-view computational tomography show that a convolutional
sparsifying regularizer learned via CAOL significantly improves reconstruction
quality compared to a conventional edge-preserving regularizer. Using more and
wider kernels in a learned regularizer better preserves edges in reconstructed
images.Comment: 22 pages, 11 figures, fixed incorrect math theorem numbers in fig.
A Matrix Factorization Approach for Learning Semidefinite-Representable Regularizers
Regularization techniques are widely employed in optimization-based approaches for solving ill-posed inverse problems in data analysis and scientific computing. These methods are based on augmenting the objective with a penalty function, which is specified based on prior domain-specific expertise to induce a desired structure in the solution. We consider the problem of learning suitable regularization functions from data in settings in which precise domain knowledge is not directly available. Previous work under the title of `dictionary learning' or `sparse coding' may be viewed as learning a regularization function that can be computed via linear programming. We describe generalizations of these methods to learn regularizers that can be computed and optimized via semidefinite programming. Our framework for learning such semidefinite regularizers is based on obtaining structured factorizations of data matrices, and our algorithmic approach for computing these factorizations combines recent techniques for rank minimization problems along with an operator analog of Sinkhorn scaling. Under suitable conditions on the input data, our algorithm provides a locally linearly convergent method for identifying the correct regularizer that promotes the type of structure contained in the data. Our analysis is based on the stability properties of Operator Sinkhorn scaling and their relation to geometric aspects of determinantal varieties (in particular tangent spaces with respect to these varieties). The regularizers obtained using our framework can be employed effectively in semidefinite programming relaxations for solving inverse problems
- …