Contributions to Medical Image Segmentation and Signal Analysis Utilizing Model Selection Methods

This thesis presents contributions to model selection techniques, especially based on information theoretic criteria, with the goal of solving problems appearing in signal analysis and in medical image representation, segmentation, and compression.The field of medical image segmentation is wide and is quickly developing to make use of higher available computational power. This thesis concentrates on several applications that allow the utilization of parametric models for image and signal representation. One important application is cell nuclei segmentation from histological images. We model nuclei contours by ellipses and thus the complicated problem of separating overlapping nuclei can be rephrased as a model selection problem, where the number of nuclei, their shapes, and their locations define one segmentation. In this thesis, we present methods for model selection in this parametric setting, where the intuitive algorithms are combined with more principled ones, namely those based on the minimum description length (MDL) principle. The results of the introduced unsupervised segmentation algorithm are compared with human subject segmentations, and are also evaluated with the help of a pathology expert.Another considered medical image application is lossless compression. The objective has been to add the task of image segmentation to that of image compression such that the image regions can be transmitted separately, depending on the region of interest for diagnosis. The experiments performed on retinal color images show that our modeling, in which the MDL criterion selects the structure of the linear predictive models, outperforms publicly available image compressors such as the lossless version of JPEG 2000.For time series modeling, the thesis presents an algorithm which allows detection of changes in time series signals. The algorithm is based on one of the most recent implementations of the MDL principle, the sequentially normalized maximum likelihood (SNML) models.This thesis produces contributions in the form of new methods and algorithms, where the simplicity of information theoretic principles are combined with a rather complex and problem dependent modeling formulation, resulting in both heuristically motivated and principled algorithmic solutions

A Better Looking Brain: Image Pre-Processing Approaches for fMRI Data

Researchers in the field of functional neuroimaging have faced a long standing problem in pre-processing low spatial resolution data without losing meaningful details within. Commonly, the brain function is recorded by a technique known as echo-planar imaging that represents the measure of blood flow (BOLD signal) through a particular location in the brain as an array of intensity values changing over time. This approach to record a movie of blood flow in the brain is known as fMRI. The neural activity is then studied from the temporal correlation patterns existing within the fMRI time series. However, the resulting images are noisy and contain low spatial detail, thus making it imperative to pre-process them appropriately to derive meaningful activation patterns. Two of the several standard preprocessing steps employed just before the analysis stage are denoising and normalization. Fundamentally, it is difficult to perfectly remove noise from an image without making assumptions about signal and noise distributions. A convenient and commonly used alternative is to smooth the image with a Gaussian filter, but this method suffers from various obvious drawbacks, primarily loss of spatial detail. A greater challenge arises when we attempt to derive average activation patterns from fMRI images acquired from a group of individuals. The brain of one individual differs from others in a structural sense as well as in a functional sense. Commonly, the inter-individual differences in anatomical structures are compensated for by co-registering each subject\u27s data to a common normalization space, known as spatial normalization. However, there are no existing methods to compensate for the differences in functional organization of the brain. This work presents first steps towards data-driven robust algorithms for fMRI image denoising and multi-subject image normalization by utilizing inherent information within fMRI data. In addition, a new validation approach based on spatial shape of the activation regions is presented to quantify the effects of preprocessing and also as a tool to record the differences in activation patterns between individual subjects or within two groups such as healthy controls and patients with mental illness. Qualititative and quantitative results of the proposed framework compare favorably against existing and widely used model-driven approaches such as Gaussian smoothing and structure-based spatial normalization. This work is intended to provide neuroscience researchers tools to derive more meaningful activation patterns to accurately identify imaging biomarkers for various neurodevelopmental diseases and also maximize the specificity of a diagnosis

Second generation sparse models

Sparse data models, where data is assumed to be well represented as a linear combination of a few elements from a learned dictionary, have gained considerable attention in recent years, and their use has led to state-of-the-art results in many applications. The success of these models is largely attributed to two critical features: the use of sparsity as a robust mechanism for regularizing the linear coefficients that represent the data, and the flexibility provided by overcomplete dictionaries that are learned from the data. These features are controlled by two critical hyper-parameters: the desired sparsity of the coefficients, and the size of the dictionaries to be learned. However, lacking theoretical guidelines for selecting these critical parameters, applications based on sparse models often require hand-tuning and cross-validation to select them, for each application, and each data set. This can be both inefficient and ineffective. On the other hand, there are multiple scenarios in which imposing additional constraints to the produced representations, including the sparse codes and the dictionary itself, can result in further improvements. This thesis is about improving and/or extending current sparse models by addressing the two issues discussed above, providing the elements for a new generation of more powerful and flexible sparse models. First, we seek to gain a better understanding of sparse models as data modeling tools, so that critical parameters can be selected automatically, efficiently, and in a principled way. Secondly, we explore new sparse modeling formulations for effectively exploiting the prior information present in different scenarios. In order to achieve these goals, we combine ideas and tools from information theory, statistics, machine learning, and optimization theory. The theoretical contributions are complemented with applications in audio, image and video processing

Subspace-based order estimation techniques in massive MIMO

Order estimation, also known as source enumeration, is a classical problem in array signal processing which consists in estimating the number of signals received by an array of sensors. In the last decades, numerous approaches to this problem have been proposed. However, the need of working with large-scale arrays (like in massive MIMO systems), low signal-to-noise- ratios, and poor sample regime scenarios, introduce new challenges to order estimation problems. For instance, most of the classical approaches are based on information theoretic criteria, which usually require a large sample size, typically several times larger than the number of sensors. Obtaining a number of samples several times larger than the number of sensors is not always possible with large-scale arrays. In addition, most of the methods found in literature assume that the noise is spatially white, which is very restrictive for many practical scenarios. This dissertation deals with the problem of source enumeration for large-scale arrays, and proposes solutions that work robustly in the small sample regime under various noise models. The first part of the dissertation solves the problem by applying the idea of subspace averaging. The input data are modelled as subspaces, and an average or central subspace is computed. The source enumeration problem can be seen as an estimation of the dimension of the central subspace. A key element of the proposed method is to construct a bootstrap procedure, based on a newly proposed discrete distribution on the manifold of projection matrices, for stochastically generating subspaces from a function of experimentally determined eigenvalues. In this way, the proposed subspace averaging (SA) technique determines the order based on the eigenvalues of an average projection matrix, rather than on the likelihood of a covariance model, penalized by functions of the model order. The proposed SA criterion is especially effective in high-dimensional scenarios with low sample support for uniform linear arrays in the presence of white noise. Further, the proposed SA method is extended for: i) non-white noises, and ii) non-uniform linear arrays. The SA criterion is sensitive with the chosen dimension of extracted subspaces. To solve this problem, we combine the SA technique with a majority vote approach. The number of sources is detected for increasing dimensions of the SA technique and then a majority vote is applied to determine the final estimate. Further, to extend SA for arrays with arbitrary geometries, the SA is combined with a sparse reconstruction (SR) step. In the first step, each received snapshot is approximated by a sparse linear combination of the rest of snapshots. The SR problem is regularized by the logarithm-based surrogate of the l-0 norm and solved using a majorization-minimization approach. Based on the SR solution, a sampling mechanism is proposed in the second step to generate a collection of subspaces, all of which approximately span the same signal subspace. Finally, the dimension of the average of this collection of subspaces provides a robust estimate for the number of sources. The second half of the dissertation introduces a completely different approach to the order estimation for uniform linear arrays, which is based on matrix completion algorithms. This part first discusses the problem of order estimation in the presence of noise whose spatial covariance structure is a diagonal matrix with possibly different variances. The diagonal terms of the sample covariance matrix are removed and, after applying Toeplitz rectification as a denoising step, the signal covariance matrix is reconstructed by using a low-rank matrix completion method adapted to enforce the Toeplitz structure of the sought solution. The proposed source enumeration criterion is based on the Frobenius norm of the reconstructed signal covariance matrix obtained for increasing rank values. The proposed method performs robustly for both small and large-scale arrays with few snapshots. Finally, an approach to work with a reduced number of radio–frequency (RF) chains is proposed. The receiving array relies on antenna switching so that at every time instant only the signals received by a randomly selected subset of antennas are downconverted to baseband and sampled. Low-rank matrix completion (MC) techniques are then used to reconstruct the missing entries of the signal data matrix to keep the angular resolution of the original large-scale array. The proposed MC algorithm exploits not only the low- rank structure of the signal subspace, but also the shift-invariance property of uniform linear arrays, which results in a better estimation of the signal subspace. In addition, the effect of MC on DOA estimation is discussed under the perturbation theory framework. Further, this approach is extended to devise a novel order estimation criterion for missing data scenario. The proposed source enumeration criterion is based on the chordal subspace distance between two sub-matrices extracted from the reconstructed matrix after using MC for increasing rank values. We show that the proposed order estimation criterion performs consistently with a very few available entries in the data matrix.This work was supported by the Ministerio de Ciencia e Innovación (MICINN) of Spain, under grants TEC2016-75067-C4-4-R (CARMEN) and BES-2017-080542

Information Extraction and Modeling from Remote Sensing Images: Application to the Enhancement of Digital Elevation Models

To deal with high complexity data such as remote sensing images presenting metric resolution over large areas, an innovative, fast and robust image processing system is presented. The modeling of increasing level of information is used to extract, represent and link image features to semantic content. The potential of the proposed techniques is demonstrated with an application to enhance and regularize digital elevation models based on information collected from RS images

Técnicas baseadas em subespaços e aplicações

Doutoramento em Engenharia ElectrónicaEste trabalho focou-se no estudo de técnicas de sub-espaço tendo em vista as aplicações seguintes: eliminação de ruído em séries temporais e extracção de características para problemas de classificação supervisionada. Foram estudadas as vertentes lineares e não-lineares das referidas técnicas tendo como ponto de partida os algoritmos SSA e KPCA. No trabalho apresentam-se propostas para optimizar os algoritmos, bem como uma descrição dos mesmos numa abordagem diferente daquela que é feita na literatura. Em qualquer das vertentes, linear ou não-linear, os métodos são apresentados utilizando uma formulação algébrica consistente. O modelo de subespaço é obtido calculando a decomposição em valores e vectores próprios das matrizes de kernel ou de correlação/covariância calculadas com um conjunto de dados multidimensional. A complexidade das técnicas não lineares de subespaço é discutida, nomeadamente, o problema da pre-imagem e a decomposição em valores e vectores próprios de matrizes de dimensão elevada. Diferentes algoritmos de préimagem são apresentados bem como propostas alternativas para a sua optimização. A decomposição em vectores próprios da matriz de kernel baseada em aproximações low-rank da matriz conduz a um algoritmo mais eficiente- o Greedy KPCA. Os algoritmos são aplicados a sinais artificiais de modo a estudar a influência dos vários parâmetros na sua performance. Para além disso, a exploração destas técnicas é extendida à eliminação de artefactos em séries temporais biomédicas univariáveis, nomeadamente, sinais EEG.This work focuses on the study of linear and non-linear subspace projective techniques with two intents: noise elimination and feature extraction. The conducted study is based on the SSA, and Kernel PCA algorithms. Several approaches to optimize the algorithms are addressed along with a description of those algorithms in a distinct approach from the one made in the literature. All methods presented here follow a consistent algebraic formulation to manipulate the data. The subspace model is formed using the elements from the eigendecomposition of kernel or correlation/covariance matrices computed on multidimensional data sets. The complexity of non-linear subspace techniques is exploited, namely the preimage problem and the kernel matrix dimensionality. Different pre-image algorithms are presented together with alternative proposals to optimize them. In this work some approximations to the kernel matrix based on its low rank approximation are discussed and the Greedy KPCA algorithm is introduced. Throughout this thesis, the algorithms are applied to artificial signals in order to study the influence of the several parameters in their performance. Furthermore, the exploitation of these techniques is extended to artefact removal in univariate biomedical time series, namely, EEG signals.FCT - SFRH/BD/28404/200

Fourth Workshop on Information Theoretic Methods in Science and Engineering : Proceedings

Peer reviewe

Sparse Modeling for Image and Vision Processing

In recent years, a large amount of multi-disciplinary research has been conducted on sparse models and their applications. In statistics and machine learning, the sparsity principle is used to perform model selection---that is, automatically selecting a simple model among a large collection of them. In signal processing, sparse coding consists of representing data with linear combinations of a few dictionary elements. Subsequently, the corresponding tools have been widely adopted by several scientific communities such as neuroscience, bioinformatics, or computer vision. The goal of this monograph is to offer a self-contained view of sparse modeling for visual recognition and image processing. More specifically, we focus on applications where the dictionary is learned and adapted to data, yielding a compact representation that has been successful in various contexts.Comment: 205 pages, to appear in Foundations and Trends in Computer Graphics and Visio