Towards Learning Representations in Visual Computing Tasks

abstract: The performance of most of the visual computing tasks depends on the quality of the features extracted from the raw data. Insightful feature representation increases the performance of many learning algorithms by exposing the underlying explanatory factors of the output for the unobserved input. A good representation should also handle anomalies in the data such as missing samples and noisy input caused by the undesired, external factors of variation. It should also reduce the data redundancy. Over the years, many feature extraction processes have been invented to produce good representations of raw images and videos. The feature extraction processes can be categorized into three groups. The first group contains processes that are hand-crafted for a specific task. Hand-engineering features requires the knowledge of domain experts and manual labor. However, the feature extraction process is interpretable and explainable. Next group contains the latent-feature extraction processes. While the original feature lies in a high-dimensional space, the relevant factors for a task often lie on a lower dimensional manifold. The latent-feature extraction employs hidden variables to expose the underlying data properties that cannot be directly measured from the input. Latent features seek a specific structure such as sparsity or low-rank into the derived representation through sophisticated optimization techniques. The last category is that of deep features. These are obtained by passing raw input data with minimal pre-processing through a deep network. Its parameters are computed by iteratively minimizing a task-based loss. In this dissertation, I present four pieces of work where I create and learn suitable data representations. The first task employs hand-crafted features to perform clinically-relevant retrieval of diabetic retinopathy images. The second task uses latent features to perform content-adaptive image enhancement. The third task ranks a pair of images based on their aestheticism. The goal of the last task is to capture localized image artifacts in small datasets with patch-level labels. For both these tasks, I propose novel deep architectures and show significant improvement over the previous state-of-art approaches. A suitable combination of feature representations augmented with an appropriate learning approach can increase performance for most visual computing tasks.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Automated Design Simplification of Quantum Program Synthesis

Quantum computers are a new and emerging technology that offer promises of being able to outperform classical machines. However, they differ from classical machines so much that they provide unique challenges to development. Working on quantum machines is currently very difficult, requiring a large amount of expertise in a great deal of areas. In order to facilitate practical software engineering methods it will be necessary to greatly simplify this process. To provide this process simplification we identify automation methods and approaches that can perform steps of quantum program compilation to greatly reduce the need for human expertise. The first contribution looks at integrating an existing classical method into the quantum model. This is done through the application of a Genetic Improvement algorithm. The second contribution looks at modelling the quantum compilation problem in a way compatible with a classical model. This is done through the generation of a Planning Domain Definition Language (PDDL) model. The third and final contribution looks at simplifying the building of a compilation stack. This is done by using a neural network to make decisions about what steps to add to the compilation stack. The results of this show a set of automated methods that produce error rates competitive with the standard quantum compilation methods. In addition, these methods require much less expertise about specific quantum hardware or the quantum compilation stack and are built to be compatible with the current IBM Quantum Experience software stack

Neural networks-on-chip for hybrid bio-electronic systems

PhD ThesisBy modelling the brains computation we can further our understanding of its function and develop novel treatments for neurological disorders. The brain is incredibly powerful and energy e cient, but its computation does not t well with the traditional computer architecture developed over the previous 70 years. Therefore, there is growing research focus in developing alternative computing technologies to enhance our neural modelling capability, with the expectation that the technology in itself will also bene t from increased awareness of neural computational paradigms. This thesis focuses upon developing a methodology to study the design of neural computing systems, with an emphasis on studying systems suitable for biomedical experiments. The methodology allows for the design to be optimized according to the application. For example, di erent case studies highlight how to reduce energy consumption, reduce silicon area, or to increase network throughput. High performance processing cores are presented for both Hodgkin-Huxley and Izhikevich neurons incorporating novel design features. Further, a complete energy/area model for a neural-network-on-chip is derived, which is used in two exemplar case-studies: a cortical neural circuit to benchmark typical system performance, illustrating how a 65,000 neuron network could be processed in real-time within a 100mW power budget; and a scalable highperformance processing platform for a cerebellar neural prosthesis. From these case-studies, the contribution of network granularity towards optimal neural-network-on-chip performance is explored

Machine learning as a service for high energy physics (MLaaS4HEP): a service for ML-based data analyses

With the CERN LHC program underway, there has been an acceleration of data growth in the High Energy Physics (HEP) field and the usage of Machine Learning (ML) in HEP will be critical during the HL-LHC program when the data that will be produced will reach the exascale. ML techniques have been successfully used in many areas of HEP nevertheless, the development of a ML project and its implementation for production use is a highly time-consuming task and requires specific skills. Complicating this scenario is the fact that HEP data is stored in ROOT data format, which is mostly unknown outside of the HEP community. The work presented in this thesis is focused on the development of a ML as a Service (MLaaS) solution for HEP, aiming to provide a cloud service that allows HEP users to run ML pipelines via HTTP calls. These pipelines are executed by using the MLaaS4HEP framework, which allows reading data, processing data, and training ML models directly using ROOT files of arbitrary size from local or distributed data sources. Such a solution provides HEP users non-expert in ML with a tool that allows them to apply ML techniques in their analyses in a streamlined manner. Over the years the MLaaS4HEP framework has been developed, validated, and tested and new features have been added. A first MLaaS solution has been developed by automatizing the deployment of a platform equipped with the MLaaS4HEP framework. Then, a service with APIs has been developed, so that a user after being authenticated and authorized can submit MLaaS4HEP workflows producing trained ML models ready for the inference phase. A working prototype of this service is currently running on a virtual machine of INFN-Cloud and is compliant to be added to the INFN Cloud portfolio of services

Probabilistic models for human behavior learning

The problem of human behavior learning is a popular interdisciplinary research topic that has been explored from multiple perspectives, with a principal branch of study in the context of computer vision systems and activity recognition. However, the statistical methods used in these frameworks typically assume short time scales, usually of minutes or even seconds. The emergence of mobile electronic devices, such as smartphones and wearables, has changed this paradigm as long as we are now able to massively collect digital records from users. This collection of smartphone-generated data, whose attributes are obtained in an unobtrusive manner from the devices via multiple sensors and apps, shape the behavioral footprint that is unique for everyone of us. At an individual level, the data projection also di ers from person to person, as not all sensors are equal, neither the apps installed, or the devices used in the real life. This point actually reflects that learning the human behavior from the digital signature of users is an arduous task, that requires to fuse irregular data. For instance, collections of samples that are corrupted, heterogeneous, outliers or have shortterm correlations. The statistical modelling of this sort of objects is one of the principal contributions of this thesis, that we study from the perspective of Gaussian processes (gp). In the particular case of humans, as well as many other life species in our world, we are inherently conditioned to the diurnal and nocturnal cycles that everyday shape our behavior, and hence, our data. We can study these cycles in our behavioral representation to see that there exists a perpetual circadian rhytm in everyone of us. This tempo is the 24h periodic component that shapes the baseline temporal structure of our behavior, not the particular patterns that change for every person. Looking to the trajectories and variabilities that our behavior may take in the data, we can appreciate that there is not a single repetitive behavior. Instead, there are typically several patterns or routines, sampled from our own dictionary, that we choose for every special situation. At the same time, these routines are arbitrary combinations of di erents timescales, correlations, levels of mobility, social interaction, sleep quality or will for working during the same hours on weekdays. Together, the properties of human behavior already indicate to us how we shall proceed to model its structure, not as unique functions, but as a dictionary of latent behavioral profiles. To discover them, we have considered latent variable models. The main application of the statistical methods developed for human behavior learning appears as we look to medicine. Having a personalized model that is accurately fitted to the behavioral patterns of some patient of interest, sudden changes in them could be early indicators of future relapses. From a technical point of view, the traditional question use to be if newer observations conform or not to the expected behavior indicated by the already fitted model. The problem can be analyzed from two perspectives that are interrelated, one more oriented to the characterization of that single object as outlier, typically named as anomaly detection, and another focused in refreshing the learning model if no longer fits to the new sequential data. This last problem, widely known as change-point detection (cpd) is another pillar of this thesis. These methods are oriented to mental health applications, and particularly to the passive detection of crisis events. The final goal is to provide an early detection methodology based on probabilistic modeling for early intervention, e.g. prevent suicide attempts, on psychiatric outpatients with severe a ective disorders of higher prevalence, such as depression or bipolar diseases.El problema de aprendizaje del comportamiento humano es un tema de investigación interdisciplinar que ha sido explorado desde múltiples perspectivas, con una línea de estudio principal en torno a los sistemas de visión por ordenador y el reconocimiento de actividades. Sin embargo, los métodos estadísticos usados en estos casos suelen asumir escalas de tiempo cortas, generalmente de minutos o incluso segundos. La aparición de tecnologías móviles, tales como teléfonos o relojes inteligentes, ha cambiado este paradigma, dado que ahora es posible recolectar ingentes colecciones de datos a partir de los usuarios. Este conjunto de datos generados a partir de nuestro teléfono, cuyos atributos se obtienen de manera no invasiva desde múltiples sensores y apps, conforman la huella de comportamiento que es única para cada uno de nosotros. A nivel individual, la proyección sobre los datos difiere de persona a persona, dado que no todos los sensores son iguales, ni las apps instaladas así como los dispositivos utilizados en la vida real. Esto precisamente refleja que el aprendizaje del comportamiento humano a partir de la huella digital de los usuarios es una ardua tarea, que requiere principalmente fusionar datos irregulares. Por ejemplo, colecciones de muestras corruptas, heterogéneas, con outliers o poseedoras de correlaciones cortas. El modelado estadístico de este tipo de objetos es una de las contribuciones principales de esta tesis, que estudiamos desde la perspectiva de los procesos Gaussianos (gp). En el caso particular de los humanos, así como para muchas otras especies en nuestro planeta, estamos inherentemente condicionados a los ciclos diurnos y nocturnos que cada día dan forma a nuestro comportamiento, y por tanto, a nuestros datos. Podemos estudiar estos ciclos en la representación del comportamiento que obtenemos y ver que realmente existe un ritmo circadiano perpetuo en cada uno de nosotros. Este tempo es en realidad la componente periódica de 24 horas que construye la base sobre la que se asienta nuestro comportamiento, no únicamente los patrones que cambian para cada persona. Mirando a las trayectorias y variabilidades que nuestro comportamiento puede plasmar en los datos, podemos apreciar que no existe un comportamiento único y repetitivo. En su lugar, hay varios patrones o rutinas, obtenidas de nuestro propio diccionario, que elegimos para cada situación especial. Al mismo tiempo, estas rutinas son combinaciones arbitrarias de diferentes escalas de tiempo, correlaciones, niveles de movilidad, interacción social, calidad del sueño o iniciativa para trabajar durante las mismas horas cada día laborable. Juntas, estas propiedades del comportamiento humano nos indican como debemos proceder a modelar su estructura, no como funciones únicas, sino como un diccionario de perfiles ocultos de comportamiento, Para descubrirlos, hemos considerado modelos de variables latentes. La aplicación principal de los modelos estadísticos desarrollados para el aprendizaje de comportamiento humano aparece en cuanto miramos a la medicina. Teniendo un modelo personalizado que está ajustado de una manera precisa a los patrones de comportamiento de un paciente, los cambios espontáneos en ellos pueden ser indicadores de futuras recaídas. Desde un punto de vista técnico, la pregunta clásica suele ser si nuevas observaciones encajan o no con lo indicado por el modelo. Este problema se puede enfocar desde dos perspectivas que están interrelacionadas, una más orientada a la caracterización de aquellos objetos como outliers, que usualmente se conoce como detección de anomalías, y otro enfocado en refrescar el modelo de aprendizaje si este deja de ajustarse debidamente a los nuevos datos secuenciales. Este último problema, ampliamente conocido como detección de puntos de cambio (cpd) es otro de los pilares de esta tesis. Estos métodos se han orientado a aplicaciones de salud mental, y particularmente, a la detección pasiva de eventos críticos. El objetivo final es proveer de una metodología de detección temprana basada en el modelado probabilístico para intervenciones rápidas. Por ejemplo, de cara a prever intentos de suicidio en pacientes fuera de hospitales con trastornos afectivos severos de gran prevalencia, como depresión o síndrome bipolar.Programa de Doctorado en Multimedia y Comunicaciones por la Universidad Carlos III de Madrid y la Universidad Rey Juan CarlosPresidente: Pablo Martínez Olmos.- Secretario: Daniel Hernández Lobato.- Vocal: Javier González Hernánde

Safety and Reliability - Safe Societies in a Changing World

The contributions cover a wide range of methodologies and application areas for safety and reliability that contribute to safe societies in a changing world. These methodologies and applications include: - foundations of risk and reliability assessment and management - mathematical methods in reliability and safety - risk assessment - risk management - system reliability - uncertainty analysis - digitalization and big data - prognostics and system health management - occupational safety - accident and incident modeling - maintenance modeling and applications - simulation for safety and reliability analysis - dynamic risk and barrier management - organizational factors and safety culture - human factors and human reliability - resilience engineering - structural reliability - natural hazards - security - economic analysis in risk managemen

Compositionality, stability and robustness in probabilistic machine learning

Probability theory plays an integral part in the field of machine learning. Its use has been advocated by many [MacKay, 2002; Jaynes, 2003] as it allows for the quantification of uncertainty and the incorporation of prior knowledge by simply applying the rules of probability [Kolmogorov, 1950]. While probabilistic machine learning has been originally restricted to simple models, the advent of new computational technologies, such as automatic differentiation, and advances in approximate inference, such as Variational Inference [Blei et al., 2017], has made it more viable in complex settings. Despite this progress, there remain many challenges to its application to real-world tasks. Among those are questions about the ability of probabilistic models to model complex tasks and their reliability both in training and in the face of unexpected data perturbation. These three issues can be addressed by examining the three properties of compositionality, stability and robustness in these models. Hence, this thesis explores these three key properties and their application to probabilistic models, while validating their importance on a range of applications. The first contribution in this thesis studies compositionality. Compositionality enables the construction of complex and expressive probabilistic models from simple components. This increases the types of phenomena that one can model and provides the modeller with a wide array of modelling options. This thesis examines this property through the lens of Gaussian processes [Rasmussen and Williams, 2006]. It proposes a generic compositional Gaussian process model to address the problem of multi-task learning in the non-linear setting. Additionally, this thesis contributes two methods addressing the issue of stability. Stability determines the reliability of inference algorithms in the presence of noise. More stable training procedures lead to faster, more reliable inferences, especially for complex models. The two proposed methods aim at stabilising stochastic gradient estimation in Variational Inference using the method of control variates [Owen, 2013]. Finally, the last contribution of this thesis considers robustness. Robust machine learning methods are unaffected by unaccounted-for phenomena in the data. This makes such methods essential in deploying machine learning on real-world datasets. This thesis examines the problem of robust inference in sequential probabilistic models by combining the ideas of Generalised Bayesian Inference [Bissiri et al., 2016] and Sequential Monte Carlo sampling [Doucet and Johansen, 2011]