On the incomplete oblique projections method for solving box constrained least squares problems

The aim of this paper is to extend the applicability of the incomplete oblique projections method (IOP) previously introduced by the authors for solving inconsistent linear systems to the box constrained case. The new algorithm employs incomplete projections onto the set of solutions of the augmented system Ax − r = b, together with the box constraints, based on a scheme similar to the one of IOP, adding the conditions for accepting an approximate solution in the box. The theoretical properties of the new algorithm are analyzed, and numerical experiences are presented comparing its performance with some well-known methods.Facultad de IngenieríaFacultad de Ciencias Exacta

Factor Analysis of Data Matrices: New Theoretical and Computational Aspects With Applications

The classical fitting problem in exploratory factor analysis (EFA) is to find estimates for the factor loadings matrix and the matrix of unique factor variances which give the best fit to the sample covariance or correlation matrix with respect to some goodness-of-fit criterion. Predicted factor scores can be obtained as a function of these estimates and the data. In this thesis, the EFA model is considered as a specific data matrix decomposition with fixed unknown matrix parameters. Fitting the EFA model directly to the data yields simultaneous solutions for both loadings and factor scores. Several new algorithms are introduced for the least squares and weighted least squares estimation of all EFA model unknowns. The numerical procedures are based on the singular value decomposition, facilitate the estimation of both common and unique factor scores, and work equally well when the number of variables exceeds the number of available observations. Like EFA, noisy independent component analysis (ICA) is a technique for reduction of the data dimensionality in which the interrelationships among the observed variables are explained in terms of a much smaller number of latent factors. The key difference between EFA and noisy ICA is that in the latter model the common factors are assumed to be both independent and non-normal. In contrast to EFA, there is no rotational indeterminacy in noisy ICA. In this thesis, noisy ICA is viewed as a method of factor rotation in EFA. Starting from an initial EFA solution, an orthogonal rotation matrix is sought that minimizes the dependence between the common factors. The idea of rotating the scores towards independence is also employed in three-mode factor analysis to analyze data sets having a three-way structure. The new theoretical and computational aspects contained in this thesis are illustrated by means of several examples with real and artificial data

Euclidean Distance Matrices: Essential Theory, Algorithms and Applications

Euclidean distance matrices (EDM) are matrices of squared distances between points. The definition is deceivingly simple: thanks to their many useful properties they have found applications in psychometrics, crystallography, machine learning, wireless sensor networks, acoustics, and more. Despite the usefulness of EDMs, they seem to be insufficiently known in the signal processing community. Our goal is to rectify this mishap in a concise tutorial. We review the fundamental properties of EDMs, such as rank or (non)definiteness. We show how various EDM properties can be used to design algorithms for completing and denoising distance data. Along the way, we demonstrate applications to microphone position calibration, ultrasound tomography, room reconstruction from echoes and phase retrieval. By spelling out the essential algorithms, we hope to fast-track the readers in applying EDMs to their own problems. Matlab code for all the described algorithms, and to generate the figures in the paper, is available online. Finally, we suggest directions for further research.Comment: - 17 pages, 12 figures, to appear in IEEE Signal Processing Magazine - change of title in the last revisio

Two derivative-free methods for solving underdetermined nonlinear systems of equations

In this paper, two different approaches to solve underdetermined nonlinear system of equations are proposed. In one of them, the derivative-free method defined by La Cruz, Martínez and Raydan for solving square nonlinear systems is modified and extended to cope with the underdetermined case. The other approach is a Quasi-Newton method that uses the Broyden update formula and the globalized line search that combines the strategy of Grippo, Lampariello and Lucidi with the Li and Fukushima one. Global convergence results for both methods are proved and numerical experiments are presented.Facultad de Ciencias Exacta

Model-based X-ray CT Image and Light Field Reconstruction Using Variable Splitting Methods.

Model-based image reconstruction (MBIR) is a powerful technique for solving ill-posed inverse problems. Compared with direct methods, it can provide better estimates from noisy measurements and from incomplete data, at the cost of much longer computation time. In this work, we focus on accelerating and applying MBIR for solving reconstruction problems, including X-ray computed tomography (CT) image reconstruction and light field reconstruction, using variable splitting based on the augmented Lagrangian (AL) methods. For X-ray CT image reconstruction, we combine the AL method and ordered subsets (OS), a well-known technique in the medical imaging literature for accelerating tomographic reconstruction, by considering a linearized variant of the AL method and propose a fast splitting-based ordered-subset algorithm, OS-LALM, for solving X-ray CT image reconstruction problems with penalized weighted least-squares (PWLS) criterion. Practical issues such as the non-trivial parameter selection of AL methods and remarkable memory overhead when considering the finite difference image variable splitting are carefully studied, and several variants of the proposed algorithm are investigated for solving practical model-based X-ray CT image reconstruction problems. Experimental results show that the proposed algorithm significantly accelerates the convergence of X-ray CT image reconstruction with negligible overhead and greatly reduces the noise-like OS artifacts in the reconstructed image when using many subsets for OS acceleration. For light field reconstruction, considering decomposing the camera imaging process into a linear convolution and a non-linear slicing operations for faster forward projection, we propose to reconstruct light field from a sequence of photos taken with different focus settings, i.e., a focal stack, using an alternating direction method of multipliers (ADMM). To improve the quality of the reconstructed light field, we also propose a signal-independent sparsifying transform by considering the elongated structure of light fields. Flatland simulation results show that our proposed sparse light field prior produces high resolution light field with fine details compared with other existing sparse priors for natural images.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/108981/1/hungnien_1.pd

Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)

The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in Belgium, from Wednesday August 27th till Friday August 29th, 2014. The workshop was conveniently located in "The Arsenal" building within walking distance of both hotels and town center. iTWIST'14 has gathered about 70 international participants and has featured 9 invited talks, 10 oral presentations, and 14 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing; Union of low dimensional subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph sensing/processing; Blind inverse problems and dictionary learning; Sparsity and computational neuroscience; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?; Sparse machine learning and inference.Comment: 69 pages, 24 extended abstracts, iTWIST'14 website: http://sites.google.com/site/itwist1

Magnetic Doppler imaging of alpha^2 Canum Venaticorum in all four Stokes parameters. Unveiling the hidden complexity of stellar magnetic fields

Strong organized magnetic fields have been studied in the upper main sequence chemically peculiar stars for more than half a century. However, only recently have observational methods and numerical techniques become sufficiently mature to allow us to record and interpret high-resolution four Stokes parameter spectra, leading to the first assumption-free magnetic field models of these stars. Here we present a detailed magnetic Doppler imaging analysis of the spectropolarimetric observations of the prototypical magnetic Ap star alpha^2 CVn. The surface abundance distributions of Fe and Cr and a full vector map of the stellar magnetic field are reconstructed in a self-consistent inversion using our state-of-the-art magnetic Doppler imaging code Invers10. We succeeded in reproducing most of the details of the available spectropolarimetric observations of alpha^2 CVn with a magnetic map which combines a global dipolar-like field topology with localized spots of higher field intensity. We demonstrate that these small-scale magnetic structures are inevitably required to fit the linear polarization spectra; however, their presence cannot be inferred from the Stokes I and V observations alone. Our magnetic Doppler imaging analysis of alpha^2 CVn and previous results for 53 Cam support the view that the upper main sequence stars can harbour fairly complex surface magnetic fields which resemble oblique dipoles only at the largest spatial scales. Spectra in all four Stokes parameters are absolutely essential to unveil and meaningfully characterize this field complexity in Ap stars. We therefore suggest that understanding magnetism of stars in other parts of the H-R diagram is similarly incomplete without investigation of their linear polarization spectra.Comment: 16 pages, 12 figures; Accepted for publication by Astronomy & Astrophysic

Learning understandable classifier models.

The topic of this dissertation is the automation of the process of extracting understandable patterns and rules from data. An unprecedented amount of data is available to anyone with a computer connected to the Internet. The disciplines of Data Mining and Machine Learning have emerged over the last two decades to face this challenge. This has led to the development of many tools and methods. These tools often produce models that make very accurate predictions about previously unseen data. However, models built by the most accurate methods are usually hard to understand or interpret by humans. In consequence, they deliver only decisions, and are short of any explanations. Hence they do not directly lead to the acquisition of new knowledge. This dissertation contributes to bridging the gap between the accurate opaque models and those less accurate but more transparent for humans. This dissertation first defines the problem of learning from data. It surveys the state-of-the-art methods for supervised learning of both understandable and opaque models from data, as well as unsupervised methods that detect features present in the data. It describes popular methods of rule extraction from unintelligible models which rewrite them into an understandable form. Limitations of rule extraction are described. A novel definition of understandability which ties computational complexity and learning is provided to show that rule extraction is an NP-hard problem. Next, a discussion whether one can expect that even an accurate classifier has learned new knowledge. The survey ends with a presentation of two approaches to building of understandable classifiers. On the one hand, understandable models must be able to accurately describe relations in the data. On the other hand, often a description of the output of a system in terms of its input requires the introduction of intermediate concepts, called features. Therefore it is crucial to develop methods that describe the data with understandable features and are able to use those features to present the relation that describes the data. Novel contributions of this thesis follow the survey. Two families of rule extraction algorithms are considered. First, a method that can work with any opaque classifier is introduced. Artificial training patterns are generated in a mathematically sound way and used to train more accurate understandable models. Subsequently, two novel algorithms that require that the opaque model is a Neural Network are presented. They rely on access to the network\u27s weights and biases to induce rules encoded as Decision Diagrams. Finally, the topic of feature extraction is considered. The impact on imposing non-negativity constraints on the weights of a neural network is considered. It is proved that a three layer network with non-negative weights can shatter any given set of points and experiments are conducted to assess the accuracy and interpretability of such networks. Then, a novel path-following algorithm that finds robust sparse encodings of data is presented. In summary, this dissertation contributes to improved understandability of classifiers in several tangible and original ways. It introduces three distinct aspects of achieving this goal: infusion of additional patterns from the underlying pattern distribution into rule learners, the derivation of decision diagrams from neural networks, and achieving sparse coding with neural networks with non-negative weights

Sparse MRI and CT Reconstruction

Sparse signal reconstruction is of the utmost importance for efficient medical imaging, conducting accurate screening for security and inspection, and for non-destructive testing. The sparsity of the signal is dictated by either feasibility, or the cost and the screening time constraints of the system. In this work, two major sparse signal reconstruction systems such as compressed sensing magnetic resonance imaging (MRI) and sparse-view computed tomography (CT) are investigated. For medical CT, a limited number of views (sparse-view) is an option for whether reducing the amount of ionizing radiation or the screening time and the cost of the procedure. In applications such as non-destructive testing or inspection of large objects, like a cargo container, one angular view can take up to a few minutes for only one slice. On the other hand, some views can be unavailable due to the configuration of the system. A problem of data sufficiency and on how to estimate a tomographic image when the projection data are not ideally sufficient for precise reconstruction is one of two major objectives of this work. Three CT reconstruction methods are proposed: algebraic iterative reconstruction-reprojection (AIRR), sparse-view CT reconstruction based on curvelet and total variation regularization (CTV), and sparse-view CT reconstruction based on nonconvex L1-L2 regularization. The experimental results confirm a high performance based on subjective and objective quality metrics. Additionally, sparse-view neutron-photon tomography is studied based on Monte-Carlo modelling to demonstrate shape reconstruction, material discrimination and visualization based on the proposed 3D object reconstruction method and material discrimination signatures. One of the methods for efficient acquisition of multidimensional signals is the compressed sensing (CS). A significantly low number of measurements can be obtained in different ways, and one is undersampling, that is sampling below the Shannon-Nyquist limit. Magnetic resonance imaging (MRI) suffers inherently from its slow data acquisition. The compressed sensing MRI (CSMRI) offers significant scan time reduction with advantages for patients and health care economics. In this work, three frameworks are proposed and evaluated, i.e., CSMRI based on curvelet transform and total generalized variation (CT-TGV), CSMRI using curvelet sparsity and nonlocal total variation: CS-NLTV, CSMRI that explores shearlet sparsity and nonlocal total variation: SS-NLTV. The proposed methods are evaluated experimentally and compared to the previously reported state-of-the-art methods. Results demonstrate a significant improvement of image reconstruction quality on different medical MRI datasets