Sinusoidal frequency estimation with applications to ultrasound

This thesis comprises two parts. The first part deals with single carrier and multiple-carrier based frequency estimation. The second part is concerned with the application of ultrasound using the proposed estimators and introduces a novel efficient implementation of a subspace tracking technique. In the first part, the problem of single frequency estimation is initially examined, and a hybrid single tone estimator is proposed, comprising both coarse and refined estimates. The coarse estimate of the unknown frequency is obtained using the unweighted linear prediction method, and is used to remove the frequency dependence of the signal-to-noise ratio (SNR) threshold. The SNR threshold is then further reduced via a combination of using an aver aging filter and an outlier removal scheme. Finally, a refined frequency estimate is formed using a weighted phase average technique. The hybrid estimator outperforms other recently developed estimators and is found to be independent of the underlying frequency. A second topic considered in the first part of this thesis is multiple-carrier based frequency estimation. Based on this idea, three novel velocity estimators are proposed by exploiting the velocity dependence of the backscattered carriers using synthetic data, all three proposed estimators are found to exhibit the capability of mitigating the poor high velocity performance of the conventional correlation based techniques and thereby provide usable performance beyond the conventional Nyquist velocity limit. To evaluate these methods statistically, the Cramer-Rao lower bound for the velocity estimation is derived. In the second part, the fundamentals of ultrasound are briefly reviewed. An efficient subspace tracking technique is introduced as a way to implement clutter eigenfilters, greatly reducing the computation complexity as compared to conventional eigenfilters which are based on the evaluation of the block singular value decomposition technique. Finally, the hybrid estimator and the multiple-carrier based velocity estimators proposed in the first part of the thesis are examined with realistic radio frequency data, illustrating the usefulness of these methods in solving practical problems

Automatic machine learning:methods, systems, challenges

This open access book presents the first comprehensive overview of general methods in Automatic Machine Learning (AutoML), collects descriptions of existing systems based on these methods, and discusses the first international challenge of AutoML systems. The book serves as a point of entry into this quickly-developing field for researchers and advanced students alike, as well as providing a reference for practitioners aiming to use AutoML in their work. The recent success of commercial ML applications and the rapid growth of the field has created a high demand for off-the-shelf ML methods that can be used easily and without expert knowledge. Many of the recent machine learning successes crucially rely on human experts, who select appropriate ML architectures (deep learning architectures or more traditional ML workflows) and their hyperparameters; however the field of AutoML targets a progressive automation of machine learning, based on principles from optimization and machine learning itself

On scalable inference and learning in spike-and-slab sparse coding

Sparse coding is a widely applied latent variable analysis technique. The standard formulation of sparse coding assumes Laplace as a prior distribution for modeling the activations of latent components. In this work we study sparse coding with spike-and-slab distribution as a prior for latent activity. A spike-and-slab distribution has its probability mass distributed across a ’spike’ at zero and a ’slab’ spreading over a continuous range. For its capacity to induce exact zeros with a higher likelihood, a spike-and-slab prior distribution constitutes a more accurate model of sparse coding. The distribution as a prior also allows for the sparseness of latent activity to be directly inferred from observed data, which essentially makes spike-and-slab sparse coding more flexible and self-adaptive to a wide range of data distributions. By modeling the slab with a Gaussian distribution, we furthermore show that in contrast to the standard approach to sparse coding, we can indeed derive closed-form analytical expressions for exact inference and learning in linear spike-and-slab sparse coding. However, as the posterior landscape of a spike-and-slab prior turns out to be highly multi-modal with a prohibitive exploration cost, in addition to the exact method, we also develop subspace and Gibbs sampling based approximate inference techniques for scalable applications of the linear model. We contrast our approximation methods with variational approximation for scalable posterior inference in linear spike-and-slab sparse coding. We further combine the Gaussian spike-and-slab prior with a nonlinear generative model, which assumes a point-wise maximum combination rule for the generation of observed data. We analyze the model as a precise encoder of low-level features such as edges and their occlusions in visual data. We again combine subspace selection with Gibbs sampling to overcome the analytical intractability of performing exact inference in the model. We numerically analyze our methods on both synthetic and real data for their verification and comparison with other approaches. We assess the linear spike-and-slab approach on source separation and image denoising benchmarks. In most experiments we obtain competitive or state-of-the-art results, while we find that spike-and-slab sparse coding overall outperforms other comparable approaches. By extracting thousands of latent components from a large amount of training data we further demonstrate that our subspace Gibbs sampler is among the most scalable posterior inference methods for a linear sparse coding approach. For the nonlinear model we experiment with artificial and real images to demonstrate that the components learned by the model lie closer to the ground-truth and are easily interpretable as the underlying generative causes of the input. We find that in comparison to standard sparse coding, the nonlinear spike-and-slab approach can compressively encode images using naturally sparse and discernible compositions of latent components. We also demonstrate that the components inferred by the model from natural image patches are statistically more consistent with respect to their structure and distribution to the response patterns of simple cells in the primary visual cortex of the brain. This work thereby contributes novel methods for sophisticated inference and learning in spike-and-slab sparse coding, while it also empirically showcases their functional efficacy through a variety of applications.Sparse Coding ist eine weit verbreitete Technik der latenten Variablenanalyse. Die Standardformulierung von Sparse Coding setzt a priori eine Laplace-Verteilung zur Modellierung der Aktivierung von latenten Komponenten voraus. In dieser Arbeit untersuchen wir Sparse Coding mit einer a priori Spike-and-Slab-Verteilung für latente Aktivität. Eine Spike-and-Slab-Verteilung verteilt ihre Wahrscheinlichkeitsmasse um ein Aktionspotential (“Spike”) um Null und eine dicke Verteilung (“slab”) über einen kontinuierlichen Wertebereich. Durch die Induktion von exakten Nullen mit einer höheren Wahrscheinlichkeit erzeugt eine Apriori-Spike-and-Slab-Verteilung ein genaueres Modell von Sparse Coding. Als A-priori-Verteilung erlaubt sie es uns die Seltenheit von latenten Komponenten direkt von Daten abzuleiten, sodass ein Spike-and-Slab-getriebenes Modell von Sparse Coding sich besser verschiedensten Verteilungen von Daten anpasst. Durch das Modellieren des Slab mittels einer Gauß-Verteilung zeigen wir, dass – im Gegensatz zur Standardformulierung von Sparse Coding – wir in der Tat geschlossene analytische Ausdrücke ableiten können, um eine exakte Ableitung und das Lernen eines linearen Spike-and-Slab-Sparse-Coding-Modell durchzuführen. Weil eine Spike-and-Slab-A-priori-Verteilung zu einer hoch multimodalen A-posteriori-Landschaft mit viel zu hohen Suchkosten führt, entwickeln wir zusätzlich zur exakten Methode Näherungslösungen basierend auf einem Teilraum und Gibbs-Sampling für skalierbare Anwendungen des Modells. Wir vergleichen unseren Ansatz der näherungsweisen Inferenz mit näherungsweiser Variationsrechnung des linearen Spike-and-Slab-Sparse Coding. Des Weiteren kombinieren wir die Spike-and-Slab-A-priori-Verteilung mit einem nicht-linearen Sparse-Coding-Modell, das eine punktweise Maximum-Kombinationsregel zur Datengenerierung voraussetzt. Wir analysieren das Modell als genauen Kodierer von untergeordneten Merkmalen in Bildern wie z.B. Kanten und deren Okklusionen. Wir lösen die analytische Ausweglosigkeit, eine Ableitung von multimodalen A-posteriori-Verteilungen im Modell durchzuführen, durch die Kombination von Gibbs-Sampling und der Auswahl eines Teilraums, um eine skalierbare Prozedur für die approximative Inferenz des Modells zu entwickeln. Wir analysieren unsere Methode numerisch durch synthetische und wirkliche Daten zum Nachweis und Vergleich mit anderen Ansätzen. Wir bewerten den linearen Spike-and-Slab-Ansatz mittels Maßstäben für die Quellentrennung und zur Rauschunterdrückung in Bildern. In den meisten Experimenten erhalten wir vergleichsweise oder die beste Resultate. Gleichzeitig finden wir, dass Spike-and-Slab-Sparse-Coding insgesamt andere vergleichbare Ansätze übertrifft. Durch die Extraktion von Tausenden von latenten Komponenten aus einer riesigen Menge an Trainingsdaten zeigen wir des Weiteren, dass unserer Teilraum Gibbs-Sampler zu den skalierbarsten Inferenzmethoden der linearen Sparse-Coding-Modelle gehört. Für das nichtlineare Modell experimentieren wir mit künstlichen und echten Bildern zur Demonstration, dass die von dem Modell gelernten Komponenten näher an der “Ground Truth” liegen und leichter zu interpretieren sind als die zugrundeliegenden generierenden Einflüsse der Eingabe. Wir finden, dass – im Vergleich zu Standard-Sparse-Coding – der nichtlineare Spike-and-Slab-Ansatz Bilder komprimierend kodieren kann durch natürliche dünnbesetzte und klar erkennbare Kompositionen von latenten Komponenten. Wir zeigen auch, dass die vom Modell abgeleiteten Komponenten von natürlichen Bildern statistisch konsistenter sind in ihrer Struktur und Verteilung mit dem Antwortmuster von einfachen Zellen im primären visuellen Kortex. Diese Arbeit leistet durch neue Methoden zur komplexen Inferenz und zum Erlernen ivvon Spike-and-Slab-Sparse-Coding einen Beitrag und demonstriert deren praktikable Wirksamkeit durch einen Vielzahl von Anwendungen

Generalizing, Decoding, and Optimizing Support Vector Machine Classification

The classification of complex data usually requires the composition of processing steps. Here, a major challenge is the selection of optimal algorithms for preprocessing and classification. Nowadays, parts of the optimization process are automized but expert knowledge and manual work are still required. We present three steps to face this process and ease the optimization. Namely, we take a theoretical view on classical classifiers, provide an approach to interpret the classifier together with the preprocessing, and integrate both into one framework which enables a semiautomatic optimization of the processing chain and which interfaces numerous algorithms

Getting the most from medical VOC data using Bayesian feature learning

The metabolic processes in the body naturally produce a diverse set of Volatile Organic Compounds (VOCs), which are excreted in breath, urine, stool and other biological samples. The VOCs produced are odorous and influenced by disease, meaning olfaction can provide information on a person’s disease state. A variety of instruments exist for performing “artificial olfaction”: measuring a sample, such as patient breath, and producing a high dimensional output representing the odour. Such instruments may be paired with machine learning techniques to identify properties of interest, such as the presence of a given disease. Research shows good disease-predictive ability of artificial olfaction instrumentation. However, the statistical methods employed are typically off-the-shelf, and do not take advantage of prior knowledge of the structure of the high dimensional data. Since sample sizes are also typically small, this can lead to suboptimal results due to a poorly-learned model. In this thesis we explore ways to get more out of artificial olfaction data. We perform statistical analyses in a medical setting, investigating disease diagnosis from breath, urine and vaginal swab measurements, and illustrating both successful identification and failure cases. We then introduce two new latent variable models constructed for dimension reduction of artificial olfaction data, but which are widely applicable. These models place a Gaussian Process (GP) prior on the mapping from latent variables to observations. Specifying a covariance function for the GP prior is an intuitive way for a user to describe their prior knowledge of the data covariance structure. We also enable an approximate posterior and marginal likelihood to be computed, and introduce a sparse variant. Both models have been made available in the R package stpca hosted at https://github.com/JimSkinner/stpca. In experiments with artificial olfaction data, these models outperform standard feature learning methods in a predictive pipeline

Functional data analysis approaches for 3-dimensional brain images

Functional data analysis approaches for 3-dimensional brain images The analysis of brain images poses many challenges from a statistical perspective. First, these images are usually high-dimensional (sometimes millions of data points for each image), therefore a statistical analysis based on scalable techniques is often required. Second, these data exhibit clear spatial dependence due to the differences in structures and functions of the brain regions. Functional data analysis is a modern branch of statistics aimed at analysing data that are in the form of functions. Many tools from multivariate analysis and nonparametric smoothing are used in functional data analysis to reduce noise and perform dimension reduction. This thesis shows three applications of functional data analysis for large-scale 3-dimensional brain images, mainly focusing on prediction of scalar and imaging outcomes. A workflow for building prediction intervals for scalar outcomes from 3D covariates is devised and applied for the prediction of individual chronological age from brain anatomical images. Then, a framework for the analysis of functional data with spatially-dependent mean-variance relationship and skewness is described, with an application to structural imaging. At last, a functional imaging problem is studied: the prediction of a task-evoked response image from resting-state data is achieved through an image-on-image regression model. The results discussed in this thesis are mostly comparable with more complicated machine-learning approaches available in the literature, while being more easily interpretable and often more computationally appealing. Functional data analysis might represent a valid option for the statistical analysis of brain images even in high-dimensional setting

ERP source tracking and localization from single trial EEG MEG signals

Electroencephalography (EEG) and magnetoencephalography (MEG), which are two of a number of neuroimaging techniques, are scalp recordings of the electrical activity of the brain. EEG and MEG (E/MEG) have excellent temporal resolution, they are easy to acquire, and have a wide range of applications in science, medicine and engineering. These valuable signals, however, suffer from poor spatial resolution and in many cases from very low signal to noise ratios. In this study, new computational methods for analyzing and improving the quality of E/MEG signals are presented. We mainly focus on single trial event-related potential (ERP) estimation and E/MEG dipole source localization. Several methods basically based on particle filtering (PF) are proposed. First, a method using PF for single trial estimation of ERP signals is considered. In this method, the wavelet coefficients of each ERP are assumed to be a Markovian process and do not change extensively across trials. The wavelet coefficients are then estimated recursively using PF. The results both for simulations and real data are compared with those of the well known Kalman Filtering (KF) approach. In the next method we move from single trial estimation to source localization of E/MEG signals. The beamforming (BF) approach for dipole source localization is generalized based on prior information about the noise. BF is in fact a spatial filter that minimizes the power of all the signals at the output of the filter except those that come from the locations of interest. In the proposed method, using two more constraints than in the classical BF formulation, the output noise powers are minimized and the interference activities are stopped