Sparse Modeling for Image and Vision Processing

In recent years, a large amount of multi-disciplinary research has been conducted on sparse models and their applications. In statistics and machine learning, the sparsity principle is used to perform model selection---that is, automatically selecting a simple model among a large collection of them. In signal processing, sparse coding consists of representing data with linear combinations of a few dictionary elements. Subsequently, the corresponding tools have been widely adopted by several scientific communities such as neuroscience, bioinformatics, or computer vision. The goal of this monograph is to offer a self-contained view of sparse modeling for visual recognition and image processing. More specifically, we focus on applications where the dictionary is learned and adapted to data, yielding a compact representation that has been successful in various contexts.Comment: 205 pages, to appear in Foundations and Trends in Computer Graphics and Visio

Fast high-dimensional Bayesian classification and clustering

We introduce a fast approach to classification and clustering applicable to high-dimensional continuous data, based on Bayesian mixture models for which explicit computations are available. This permits us to treat classification and clustering in a single framework, and allows calculation of unobserved class probability. The new classifier is robust to adding noise variables as a drawback of the built-in spike-and-slab structure of the proposed Bayesian model. The usefulness of classification using our method is shown on metabololomic example, and on the Iris data with and without noise variables. Agglomerative hierarchical clustering is used to construct a dendrogram based on the posterior probabilities of particular partitions, to provide a dendrogram with a probabilistic interpretation. An extension to variable selection is proposed which summarises the importance of variables for classification or clustering and has probabilistic interpretation. Having a simple model provides estimation of the model parameters using maximum likelihood and therefore yields a fully automatic algorithm. The new clustering method is applied to metabolomic, microarray, and image data and is studied using simulated data motivated by real datasets. The computational difficulties of the new approach are discussed, solutions for algorithm acceleration are proposed, and the written computer code is briefly analysed. Simulations shows that the quality of the estimated model parameters depends on the parametric distribution assumed for effects, but after fixing the model parameters to reasonable values, the distribution of the effects influences clustering very little. Simulations confirms that the clustering algorithm and the proposed variable selection method is reliable when the model assumptions are wrong. The new approach is compared with the popular Bayesian clustering alternative, MCLUST, fitted on the principal components using two loss functions in which our proposed approach is found to be more efficient in almost every situation

Fail-Safe Vehicle Pose Estimation in Lane-Level Maps Using Pose Graph Optimization

Die hochgenaue Posenschätzung autonomer Fahrzeuge sowohl in HD-Karten als auch spurrelativ ist unerlässlich um eine sichere Fahrzeugführung zu gewährleisten. Für die Serienfertigung wird aus Kosten- und Platzgründen bewusst auf hochgenaue, teure Einzelsensorik verzichtet und stattdessen auf eine Vielzahl von Sensoren, die neben der Posenschätzung auch von anderen Modulen verwendet werden können, zurückgegriffen. Im Fokus dieser Arbeit steht die Unsicherheitsschätzung, Bewertung und Fusion dieser Sensordaten. Die Optimierung von Posengraphen zur Fusion von Sensordaten zeichnet sich, im Gegensatz zu klassischen Filterverfahren, wie Kalman oder Partikelfilter, durch seine Robustheit gegenüber Fehlmessungen und der Flexibilität in der Modellierung aus. Die Optimierung eines Posengraphen wurde erstmalig auf mobilen Roboterplattformen zur Lösung sogenannter SLAM-Probleme angewendet. Diese Verfahren wurden immer weiter entwickelt und im speziellen auch zur rein kamerabasierten Lokalisierung autonomer Fahrzeuge in 3D-Punktwolken erfolgreich emonstriert. Für die Entwicklung und Freigabe sicherheitsrelevanter Systeme nach ISO 26262 wird neben der Genauigkeit jedoch auch eine Aussage über die Qualität und Ausfallsicherheit dieser Systeme gefordert. Diese Arbeit befasst sich, neben der Schätzung der karten- und spurrelativen Pose, auch mit der Schätzung der Posenunsicherheit und der Integrität der Sensordaten zueinander. Auf Grundlage dieser Arbeit wird eine Abschätzung der Ausfallsicherheit des Lokalisierungsmoduls ermöglicht. Motiviert durch das Projekt Ko-HAF werden zur Lokalisierung in HD-Karten lediglich Spurmarkierungen verwendet. Die speichereffiziente Darstellung dieser Karten ermöglicht eine hochfrequente Aktualisierung der Karteninhalte durch eine Fahrzeugflotte. Der vorgestellte Ansatz wurde prototypisch auf einem Opel Insignia umgesetzt. Der Testträger wurde um eine Front- und Heckkamera sowie einen GNSS-Empfänger erweitert. Zunächst werden die Schätzung der karten-und spurrelativen Fahrzeugpose, der GNSS-Signalauswertung sowie der Bewegungsschätzung des Fahrzeugs vorgestellt. Durch einen Vergleich der Schätzungen zueinander werden die Unsicherheiten der einzelnen Module berechnet. Das Lokalisierungsproblem wird dann durch einen Optimierer gelöst. Mithilfe der berechneten Unsicherheiten wird in einem nachgelagerten Schritt eine Bewertung der einzelnen Module durchgeführt. Zur Bewertung des Ansatzes wurden sowohl hochdynamische Manöver auf einer Teststrecke als auch Fahrten auf öffentlichen Autobahnen ausgewertet

Information Theory and Machine Learning

The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be distributed, have transferable learning results, use computation resources efficiently, convergence quickly on online settings, have performance guarantees, satisfy fairness or privacy constraints, incorporate domain knowledge on model structures, etc. A new wave of developments in statistical learning theory and information theory has set out to address these challenges. This Special Issue, "Machine Learning and Information Theory", aims to collect recent results in this direction reflecting a diverse spectrum of visions and efforts to extend conventional theories and develop analysis tools for these complex machine learning systems

Advanced analyses of physiological signals and their role in Neonatal Intensive Care

Preterm infants admitted to the neonatal intensive care unit (NICU) face an array of life-threatening diseases requiring procedures such as resuscitation and invasive monitoring, and other risks related to exposure to the hospital environment, all of which may have lifelong implications. This thesis examined a range of applications for advanced signal analyses in the NICU, from identifying of physiological patterns associated with neonatal outcomes, to evaluating the impact of certain treatments on physiological variability. Firstly, the thesis examined the potential to identify infants at risk of developing intraventricular haemorrhage, often interrelated with factors leading to preterm birth, mechanical ventilation, hypoxia and prolonged apnoeas. This thesis then characterised the cardiovascular impact of caffeine therapy which is often administered to prevent and treat apnoea of prematurity, finding greater pulse pressure variability and enhanced responsiveness of the autonomic nervous system. Cerebral autoregulation maintains cerebral blood flow despite fluctuations in arterial blood pressure and is an important consideration for preterm infants who are especially vulnerable to brain injury. Using various time and frequency domain correlation techniques, the thesis found acute changes in cerebral autoregulation of preterm infants following caffeine therapy. Nutrition in early life may also affect neurodevelopment and morbidity in later life. This thesis developed models for identifying malnutrition risk using anthropometry and near-infrared interactance features. This thesis has presented a range of ways in which advanced analyses including time series analysis, feature selection and model development can be applied to neonatal intensive care. There is a clear role for such analyses in early detection of clinical outcomes, characterising the effects of relevant treatments or pathologies and identifying infants at risk of later morbidity

CELLULAR-ENABLED MACHINE TYPE COMMUNICATIONS: RECENT TECHNOLOGIES AND COGNITIVE RADIO APPROACHES

The scarcity of bandwidth has always been the main obstacle for providing reliable high data-rate wireless links, which are in great demand to accommodate nowadays and immediate future wireless applications. In addition, recent reports have showed inefficient usage and under-utilization of the available bandwidth. Cognitive radio (CR) has recently emerged as a promising solution to enhance the spectrum utilization, where it offers the ability for unlicensed users to access the licensed spectrum opportunistically. By allowing opportunistic spectrum access which is the main concept for the interweave network model, the overall spectrum utilization can be improved. This requires cognitive radio networks (CRNs) to consider the spectrum sensing and monitoring as an essential enabling process for the interweave network model. Machine-to-machine (M2M) communication, which is the basic enabler for the Internet-of-Things (IoT), has emerged to be a key element in future networks. Machines are expected to communicate with each other exchanging information and data without human intervention. The ultimate objective of M2M communications is to construct comprehensive connections among all machines distributed over an extensive coverage area. Due to the radical change in the number of users, the network has to carefully utilize the available resources in order to maintain reasonable quality-of-service (QoS). Generally, one of the most important resources in wireless communications is the frequency spectrum. To utilize the frequency spectrum in IoT environment, it can be argued that cognitive radio concept is a possible solution from the cost and performance perspectives. Thus, supporting numerous number of machines is possible by employing dual-mode base stations which can apply cognitive radio concept in addition to the legacy licensed frequency assignment. In this thesis, a detailed review of the state of the art related to the application of spectrum sensing in CR communications is considered. We present the latest advances related to the implementation of the legacy spectrum sensing approaches. We also address the implementation challenges for cognitive radios in the direction of spectrum sensing and monitoring. We propose a novel algorithm to solve the reduced throughput issue due to the scheduled spectrum sensing and monitoring. Further, two new architectures are considered to significantly reduce the power consumption required by the CR to enable wideband sensing. Both systems rely on the 1-bit quantization at the receiver side. The system performance is analytically investigated and simulated. Also, complexity and power consumption are investigated and studied. Furthermore, we address the challenges that are expected from the next generation M2M network as an integral part of the future IoT. This mainly includes the design of low-power low-cost machine with reduced bandwidth. The trade-off between cost, feasibility, and performance are also discussed. Because of the relaxation of the frequency and spatial diversities, in addition, to enabling the extended coverage mode, initial synchronization and cell search have new challenges for cellular-enabled M2M systems. We study conventional solutions with their pros and cons including timing acquisition, cell detection, and frequency offset estimation algorithms. We provide a technique to enhance the performance in the presence of the harsh detection environment for LTE-based machines. Furthermore, we present a frequency tracking algorithm for cellular M2M systems that utilizes the new repetitive feature of the broadcast channel symbols in next generation Long Term Evolution (LTE) systems. In the direction of narrowband IoT support, we propose a cell search and initial synchronization algorithm that utilizes the new set of narrowband synchronization signals. The proposed algorithms have been simulated at very low signal to noise ratios and in different fading environments

New perspectives and methods for stream learning in the presence of concept drift.

153 p.Applications that generate data in the form of fast streams from non-stationary environments, that is,those where the underlying phenomena change over time, are becoming increasingly prevalent. In thiskind of environments the probability density function of the data-generating process may change overtime, producing a drift. This causes that predictive models trained over these stream data become obsoleteand do not adapt suitably to the new distribution. Specially in online learning scenarios, there is apressing need for new algorithms that adapt to this change as fast as possible, while maintaining goodperformance scores. Examples of these applications include making inferences or predictions based onfinancial data, energy demand and climate data analysis, web usage or sensor network monitoring, andmalware/spam detection, among many others.Online learning and concept drift are two of the most hot topics in the recent literature due to theirrelevance for the so-called Big Data paradigm, where nowadays we can find an increasing number ofapplications based on training data continuously available, named as data streams. Thus, learning in nonstationaryenvironments requires adaptive or evolving approaches that can monitor and track theunderlying changes, and adapt a model to accommodate those changes accordingly. In this effort, Iprovide in this thesis a comprehensive state-of-the-art approaches as well as I identify the most relevantopen challenges in the literature, while focusing on addressing three of them by providing innovativeperspectives and methods.This thesis provides with a complete overview of several related fields, and tackles several openchallenges that have been identified in the very recent state of the art. Concretely, it presents aninnovative way to generate artificial diversity in ensembles, a set of necessary adaptations andimprovements for spiking neural networks in order to be used in online learning scenarios, and finally, adrift detector based on this former algorithm. All of these approaches together constitute an innovativework aimed at presenting new perspectives and methods for the field