Continuous Hands-off Control by CLOT Norm Minimization

In this paper, we consider hands-off control via minimization of the C LOT (Combined L -One and Two) norm. The maximum hands-off control is the L 0 -optimal (or the sparsest) control among all feasible controls that are bounded b y a specified value and transfer the state from a given initial state to the origin within a fixed time dura tion. In general, the maximum hands-off control is a bang-off-bang control taking value s of ± 1 and 0. For many real applications, such discontinuity in the control is not desirable. To ob tain a continuous but still relatively sparse control, we propose to use the CLOT norm, a conv ex combination of L 1 and L 2 norms. We show by numerical simulation that the CLOT control is con tinuous and much sparser (i.e. has longer time duration on which the control takes 0) than the conventional EN (elastic net) control, which is a convex combination of L 1 and squared L 2 norms

A Practical Study of Longitudinal Reference Based Compressed Sensing for MRI

Compressed sensing (CS) is a new signal acquisition paradigm that enables the reconstruction of signals and images from a low number of samples. A particularly exciting application of CS is Magnetic Resonance Imaging (MRI), where CS significantly speeds up scan time by requiring far fewer measurements than standard MRI techniques. Such a reduction in sampling time leads to less power consumption, less need for patient sedation, and more accurate images. This accuracy increase is especially pronounced in pediatric MRI where patients have trouble being still for long scan periods. Although such gains are already significant, even further improvements can be made by utilizing past MRI scans of the same patient. Many patients require repeated scans over a period of time in order to track illnesses and the prior scans can be used as references for the current image. This allows samples to be taken adaptively, based on both the prior scan and the current measurements. Work by Weizman [20] has shown that so-called reference based adaptive-weighted temporal Compressed Sensing MRI (LACS-MRI) requires far fewer samples than standard Compressed Sensing (CS) to achieve the same reconstruction signal-to-noise ratio (RSNR). The method uses a mixture of reference-based and adaptive-sampling. In this work, we test this methodology by using various adaptive sensing schemes, reconstruction methods, and image types. We create a thorough catalog of reconstruction behavior and success rates that is interesting from a mathematical point of view and is useful for practitioners. We also solve a grayscale compensation toy problem that supports the insensitivity of LACS-MRI to changes in MRI acquisition parameters and thus showcases the reliability of LACS-MRI in possible clinical situations

A Practical Study of Longitudinal Reference Based Compressed Sensing for MRI

Compressed sensing (CS) is a new signal acquisition paradigm that enables the reconstruction of signals and images from a low number of samples. A particularly exciting application of CS is Magnetic Resonance Imaging (MRI), where CS significantly speeds up scan time by requiring far fewer measurements than standard MRI techniques. Such a reduction in sampling time leads to less power consumption, less need for patient sedation, and more accurate images. This accuracy increase is especially pronounced in pediatric MRI where patients have trouble being still for long scan periods. Although such gains are already significant, even further improvements can be made by utilizing past MRI scans of the same patient. Many patients require repeated scans over a period of time in order to track illnesses and the prior scans can be used as references for the current image. This allows samples to be taken adaptively, based on both the prior scan and the current measurements. Work by Weizman has shown that so-called reference based adaptive-weighted temporal Compressed Sensing MRI (LACS-MRI) requires far fewer samples than standard Compressed Sensing (CS) to achieve the same reconstruction signal-to-noise ratio (RSNR). The method uses a mixture of reference-based and adaptive-sampling. In this work, we test this methodology by using various adaptive sensing schemes, reconstruction methods, and image types. We create a thorough catalog of reconstruction behavior and success rates that is interesting from a mathematical point of view and is useful for practitioners. We also solve a grayscale compensation toy problem that supports the insensitivity of LACS-MRI to changes in MRI acquisition parameters and thus showcases the reliability of LACS-MRI in possible clinical situations

Numerical Characterization of Support Recovery in Sparse Regression with Correlated Design

Sparse regression is frequently employed in diverse scientific settings as a feature selection method. A pervasive aspect of scientific data that hampers both feature selection and estimation is the presence of strong correlations between predictive features. These fundamental issues are often not appreciated by practitioners, and jeapordize conclusions drawn from estimated models. On the other hand, theoretical results on sparsity-inducing regularized regression such as the Lasso have largely addressed conditions for selection consistency via asymptotics, and disregard the problem of model selection, whereby regularization parameters are chosen. In this numerical study, we address these issues through exhaustive characterization of the performance of several regression estimators, coupled with a range of model selection strategies. These estimators and selection criteria were examined across correlated regression problems with varying degrees of signal to noise, distribution of the non-zero model coefficients, and model sparsity. Our results reveal a fundamental tradeoff between false positive and false negative control in all regression estimators and model selection criteria examined. Additionally, we are able to numerically explore a transition point modulated by the signal-to-noise ratio and spectral properties of the design covariance matrix at which the selection accuracy of all considered algorithms degrades. Overall, we find that SCAD coupled with BIC or empirical Bayes model selection performs the best feature selection across the regression problems considered

A compressed sensing approach to block-iterative equalization: connections and applications to radar imaging reconstruction

The widespread of underdetermined systems has brought forth a variety of new algorithmic solutions, which capitalize on the Compressed Sensing (CS) of sparse data. While well known greedy or iterative threshold type of CS recursions take the form of an adaptive filter followed by a proximal operator, this is no different in spirit from the role of block iterative decision-feedback equalizers (BI-DFE), where structure is roughly exploited by the signal constellation slicer. By taking advantage of the intrinsic sparsity of signal modulations in a communications scenario, the concept of interblock interference (IBI) can be approached more cunningly in light of CS concepts, whereby the optimal feedback of detected symbols is devised adaptively. The new DFE takes the form of a more efficient re-estimation scheme, proposed under recursive-least-squares based adaptations. Whenever suitable, these recursions are derived under a reduced-complexity, widely-linear formulation, which further reduces the minimum-mean-square-error (MMSE) in comparison with traditional strictly-linear approaches. Besides maximizing system throughput, the new algorithms exhibit significantly higher performance when compared to existing methods. Our reasoning will also show that a properly formulated BI-DFE turns out to be a powerful CS algorithm itself. A new algorithm, referred to as CS-Block DFE (CS-BDFE) exhibits improved convergence and detection when compared to first order methods, thus outperforming the state-of-the-art Complex Approximate Message Passing (CAMP) recursions. The merits of the new recursions are illustrated under a novel 3D MIMO Radar formulation, where the CAMP algorithm is shown to fail with respect to important performance measures.A proliferação de sistemas sub-determinados trouxe a tona uma gama de novas soluções algorítmicas, baseadas no sensoriamento compressivo (CS) de dados esparsos. As recursões do tipo greedy e de limitação iterativa para CS se apresentam comumente como um filtro adaptativo seguido de um operador proximal, não muito diferente dos equalizadores de realimentação de decisão iterativos em blocos (BI-DFE), em que um decisor explora a estrutura do sinal de constelação. A partir da esparsidade intrínseca presente na modulação de sinais no contexto de comunicações, a interferência entre blocos (IBI) pode ser abordada utilizando-se o conceito de CS, onde a realimentação ótima de símbolos detectados é realizada de forma adaptativa. O novo DFE se apresenta como um esquema mais eficiente de reestimação, baseado na atualização por mínimos quadrados recursivos (RLS). Sempre que possível estas recursões são propostas via formulação linear no sentido amplo, o que reduz ainda mais o erro médio quadrático mínimo (MMSE) em comparação com abordagens tradicionais. Além de maximizar a taxa de transferência de informação, o novo algoritmo exibe um desempenho significativamente superior quando comparado aos métodos existentes. Também mostraremos que um equalizador BI-DFE formulado adequadamente se torna um poderoso algoritmo de CS. O novo algoritmo CS-BDFE apresenta convergência e detecção aprimoradas, quando comparado a métodos de primeira ordem, superando as recursões de Passagem de Mensagem Aproximada para Complexos (CAMP). Os méritos das novas recursões são ilustrados através de um modelo tridimensional para radares MIMO recentemente proposto, onde o algoritmo CAMP falha em aspectos importantes de medidas de desempenho

Toy Models of Superposition

Neural networks often pack many unrelated concepts into a single neuron - a puzzling phenomenon known as 'polysemanticity' which makes interpretability much more challenging. This paper provides a toy model where polysemanticity can be fully understood, arising as a result of models storing additional sparse features in "superposition." We demonstrate the existence of a phase change, a surprising connection to the geometry of uniform polytopes, and evidence of a link to adversarial examples. We also discuss potential implications for mechanistic interpretability.Comment: Also available at https://transformer-circuits.pub/2022/toy_model/index.htm

Recent Progress in Image Deblurring

This paper comprehensively reviews the recent development of image deblurring, including non-blind/blind, spatially invariant/variant deblurring techniques. Indeed, these techniques share the same objective of inferring a latent sharp image from one or several corresponding blurry images, while the blind deblurring techniques are also required to derive an accurate blur kernel. Considering the critical role of image restoration in modern imaging systems to provide high-quality images under complex environments such as motion, undesirable lighting conditions, and imperfect system components, image deblurring has attracted growing attention in recent years. From the viewpoint of how to handle the ill-posedness which is a crucial issue in deblurring tasks, existing methods can be grouped into five categories: Bayesian inference framework, variational methods, sparse representation-based methods, homography-based modeling, and region-based methods. In spite of achieving a certain level of development, image deblurring, especially the blind case, is limited in its success by complex application conditions which make the blur kernel hard to obtain and be spatially variant. We provide a holistic understanding and deep insight into image deblurring in this review. An analysis of the empirical evidence for representative methods, practical issues, as well as a discussion of promising future directions are also presented.Comment: 53 pages, 17 figure