Recovering Structured Low-rank Operators Using Nuclear Norms

This work considers the problem of recovering matrices and operators from limited and/or noisy observations. Whereas matrices result from summing tensor products of vectors, operators result from summing tensor products of matrices. These constructions lead to viewing both matrices and operators as the sum of "simple" rank-1 factors. A popular line of work in this direction is low-rank matrix recovery, i.e., using linear measurements of a matrix to reconstruct it as the sum of few rank-1 factors. Rank minimization problems are hard in general, and a popular approach to avoid them is convex relaxation. Using the trace norm as a surrogate for rank, the low-rank matrix recovery problem becomes convex. While the trace norm has received much attention in the literature, other convexifications are possible. This thesis focuses on the class of nuclear norms—a class that includes the trace norm itself. Much as the trace norm is a convex surrogate for the matrix rank, other nuclear norms provide convex complexity measures for additional matrix structure. Namely, nuclear norms measure the structure of the factors used to construct the matrix. Transitioning to the operator framework allows for novel uses of nuclear norms in recovering these structured matrices. In particular, this thesis shows how to lift structured matrix factorization problems to rank-1 operator recovery problems. This new viewpoint allows nuclear norms to measure richer types of structures present in matrix factorizations. This work also includes a Python software package to model and solve structured operator recovery problems. Systematic numerical experiments in operator denoising demonstrate the effectiveness of nuclear norms in recovering structured operators. In particular, choosing a specific nuclear norm that corresponds to the underlying factor structure of the operator improves the performance of the recovery procedures when compared, for instance, to the trace norm. Applications in hyperspectral imaging and self-calibration demonstrate the additional flexibility gained by utilizing operator (as opposed to matrix) factorization models.</p

Design of Scalable Hardware-Efficient Compressive Sensing Image Sensors

This work presents a new compressive sensing (CS) measurement method for image sensors, which limits pixel summation within neighbor pixels and follows regular summation patterns. Simulations with a large set of benchmark images show that the proposed method leads to improved image quality. Circuit implementation for the proposed CS measurement method is presented with the use of current mode pixel cells; and the resultant CS image sensor circuit is significantly simpler than existing designs. With compression rates of 4 and 8, the developed CS image sensors can achieve 34.2 dB and 29.6 dB PSNR values with energy consumption of 1.4 mJ and 0.73 mJ per frame, respectively

Estimation and Calibration Algorithms for Distributed Sampling Systems

Thesis Supervisor: Gregory W. Wornell Title: Professor of Electrical Engineering and Computer ScienceTraditionally, the sampling of a signal is performed using a single component such as an analog-to-digital converter. However, many new technologies are motivating the use of multiple sampling components to capture a signal. In some cases such as sensor networks, multiple components are naturally found in the physical layout; while in other cases like time-interleaved analog-to-digital converters, additional components are added to increase the sampling rate. Although distributing the sampling load across multiple channels can provide large benefits in terms of speed, power, and resolution, a variety mismatch errors arise that require calibration in order to prevent a degradation in system performance. In this thesis, we develop low-complexity, blind algorithms for the calibration of distributed sampling systems. In particular, we focus on recovery from timing skews that cause deviations from uniform timing. Methods for bandlimited input reconstruction from nonuniform recurrent samples are presented for both the small-mismatch and the low-SNR domains. Alternate iterative reconstruction methods are developed to give insight into the geometry of the problem. From these reconstruction methods, we develop time-skew estimation algorithms that have high performance and low complexity even for large numbers of components. We also extend these algorithms to compensate for gain mismatch between sampling components. To understand the feasibility of implementation, analysis is also presented for a sequential implementation of the estimation algorithm. In distributed sampling systems, the minimum input reconstruction error is dependent upon the number of sampling components as well as the sample times of the components. We develop bounds on the expected reconstruction error when the time-skews are distributed uniformly. Performance is compared to systems where input measurements are made via projections onto random bases, an alternative to the sinc basis of time-domain sampling. From these results, we provide a framework on which to compare the effectiveness of any calibration algorithm. Finally, we address the topic of extreme oversampling, which pertains to systems with large amounts of oversampling due to redundant sampling components. Calibration algorithms are developed for ordering the components and for estimating the input from ordered components. The algorithms exploit the extra samples in the system to increase estimation performance and decrease computational complexity

Estimating Sparse Representations from Dictionaries With Uncertainty

In the last two decades, sparse representations have gained increasing attention in a variety of engineering applications. A sparse representation of a signal requires a dictionary of basic elements that describe salient and discriminant features of that signal. When the dictionary is created from a mathematical model, its expressiveness depends on the quality of this model. In this dissertation, the problem of estimating sparse representations in the presence of errors and uncertainty in the dictionary is addressed. In the first part, a statistical framework for sparse regularization is introduced. The second part is concerned with the development of methodologies for estimating sparse representations from highly redundant dictionaries along with unknown dictionary parameters. The presented methods are illustrated using applications in direction finding and fiber-optic sensing. They serve as illustrative examples for investigating the abstract problems in the theory of sparse representations. Estimating a sparse representation often involves the solution of a regularized optimization problem. The presented regularization framework offers a systematic procedure for the determination of a regularization parameter that accounts for the joint effects of model errors and measurement noise. It is determined as an upper bound of the mean-squared error between the corrupted data and the ideal model. Despite proper regularization, the quality and accuracy of the obtained sparse representation remains affected by model errors and is indeed sensitive to changes in the regularization parameter. To alleviate this problem, dictionary calibration is performed. The framework is applied to the problem of direction finding. Redundancy enables the dictionary to describe a broader class of observations but also increases the similarity between different entries, which leads to ambiguous representations. To address the problem of redundancy and additional uncertainty in the dictionary parameters, two strategies are pursued. Firstly, an alternating estimation method for iteratively determining the underlying sparse representation and the dictionary parameters is presented. Also, theoretical bounds for the estimation errors are derived. Secondly, a Bayesian framework for estimating sparse representations and dictionary learning is developed. A hierarchical structure is considered to account for uncertainty in prior assumptions. The considered model for the coefficients of the sparse representation is particularly designed to handle high redundancy in the dictionary. Approximate inference is accomplished using a hybrid Markov Chain Monte Carlo algorithm. The performance and practical applicability of both methodologies is evaluated for a problem in fiber-optic sensing, where a mathematical model for the sensor signal is compiled. This model is used to generate a suitable parametric dictionary