Uncertainty Estimation, Explanation and Reduction with Insufficient Data

Human beings have been juggling making smart decisions under uncertainties, where we manage to trade off between swift actions and collecting sufficient evidence. It is naturally expected that a generalized artificial intelligence (GAI) to navigate through uncertainties meanwhile predicting precisely. In this thesis, we aim to propose strategies that underpin machine learning with uncertainties from three perspectives: uncertainty estimation, explanation and reduction. Estimation quantifies the variability in the model inputs and outputs. It can endow us to evaluate the model predictive confidence. Explanation provides a tool to interpret the mechanism of uncertainties and to pinpoint the potentials for uncertainty reduction, which focuses on stabilizing model training, especially when the data is insufficient. We hope that this thesis can motivate related studies on quantifying predictive uncertainties in deep learning. It also aims to raise awareness for other stakeholders in the fields of smart transportation and automated medical diagnosis where data insufficiency induces high uncertainty. The thesis is dissected into the following sections: Introduction. we justify the necessity to investigate AI uncertainties and clarify the challenges existed in the latest studies, followed by our research objective. Literature review. We break down the the review of the state-of-the-art methods into uncertainty estimation, explanation and reduction. We make comparisons with the related fields encompassing meta learning, anomaly detection, continual learning as well. Uncertainty estimation. We introduce a variational framework, neural process that approximates Gaussian processes to handle uncertainty estimation. Two variants from the neural process families are proposed to enhance neural processes with scalability and continual learning. Uncertainty explanation. We inspect the functional distribution of neural processes to discover the global and local factors that affect the degree of predictive uncertainties. Uncertainty reduction. We validate the proposed uncertainty framework on two scenarios: urban irregular behaviour detection and neurological disorder diagnosis, where the intrinsic data insufficiency undermines the performance of existing deep learning models. Conclusion. We provide promising directions for future works and conclude the thesis

Mapping Dark Matter in Galaxy Clusters with Gravitational Lensing

This thesis uses gravitational lensing to map the distribution of dark matter around galaxy clusters, and to infer their formation history. Galaxy clusters are the oldest and most massive gravitationally-bound objects in the Universe, exploited in the most discriminating tests of cosmology. It is therefore essential to understand the astrophysics of their formation. Indeed, clusters grow through filamentary connections with surrounding large-scale structures - and to chart their history is to trace the evolution and trajectory of the Universe itself. Gravitational lensing is the apparent distortion in the shapes of distant galaxies due to foreground mass, such as a galaxy cluster. Many software algorithms have been developed to measure gravitational lensing and to reconstruct the distribution of foreground mass. In this thesis, we assess the performance of two mass-mapping techniques, using mock images of the BAHAMAS simulation, where the true distribution of mass is known. We find the methods suitable for different applications: MRLens suppresses noise without bias, while Lenstool suppresses noise further, but at a cost of over-estimating the mass in cluster outskirts (R>1Mpc) by up to a factor 2. We also develop a filter to search for large-scale filaments connected to galaxy clusters. We then use these calibrated techniques, and the largest ever mosaic of Hubble Space Telescope imaging, to study galaxy cluster MS 0451-03 (z=0.54). We map the distribution of its dark matter, and discover six group-scale substructures, linked to the cluster halo by three possible filaments. By comparing lensing results with analyses of X-ray emission and optical spectroscopy, we conclude that the cluster collided with another 2--7 Gyr ago. Its star formation was quenched and its gas was heated; its gas has still not yet relaxed, and the dark matter halos are approaching second apocentre. In the next decade, space-based telescopes will reveal this richness of detail about tens of thousands of galaxy clusters. If these observations are properly calibrated, via studies like this thesis, they will bring a new era of precision cosmology. As a final step towards this future, we present preliminary results from two ongoing projects: using deep learning to further suppress noise in lensing mass reconstruction, and the first successful measurement of gravitational lensing from a balloon-borne telescope at the edge of space

Application of deep learning methods in materials microscopy for the quality assessment of lithium-ion batteries and sintered NdFeB magnets

Die Qualitätskontrolle konzentriert sich auf die Erkennung von Produktfehlern und die Überwachung von Aktivitäten, um zu überprüfen, ob die Produkte den gewünschten Qualitätsstandard erfüllen. Viele Ansätze für die Qualitätskontrolle verwenden spezialisierte Bildverarbeitungssoftware, die auf manuell entwickelten Merkmalen basiert, die von Fachleuten entwickelt wurden, um Objekte zu erkennen und Bilder zu analysieren. Diese Modelle sind jedoch mühsam, kostspielig in der Entwicklung und schwer zu pflegen, während die erstellte Lösung oft spröde ist und für leicht unterschiedliche Anwendungsfälle erhebliche Anpassungen erfordert. Aus diesen Gründen wird die Qualitätskontrolle in der Industrie immer noch häufig manuell durchgeführt, was zeitaufwändig und fehleranfällig ist. Daher schlagen wir einen allgemeineren datengesteuerten Ansatz vor, der auf den jüngsten Fortschritten in der Computer-Vision-Technologie basiert und Faltungsneuronale Netze verwendet, um repräsentative Merkmale direkt aus den Daten zu lernen. Während herkömmliche Methoden handgefertigte Merkmale verwenden, um einzelne Objekte zu erkennen, lernen Deep-Learning-Ansätze verallgemeinerbare Merkmale direkt aus den Trainingsproben, um verschiedene Objekte zu erkennen. In dieser Dissertation werden Modelle und Techniken für die automatisierte Erkennung von Defekten in lichtmikroskopischen Bildern von materialografisch präparierten Schnitten entwickelt. Wir entwickeln Modelle zur Defekterkennung, die sich grob in überwachte und unüberwachte Deep-Learning-Techniken einteilen lassen. Insbesondere werden verschiedene überwachte Deep-Learning-Modelle zur Erkennung von Defekten in der Mikrostruktur von Lithium-Ionen-Batterien entwickelt, von binären Klassifizierungsmodellen, die auf einem Sliding-Window-Ansatz mit begrenzten Trainingsdaten basieren, bis hin zu komplexen Defekterkennungs- und Lokalisierungsmodellen, die auf ein- und zweistufigen Detektoren basieren. Unser endgültiges Modell kann mehrere Klassen von Defekten in großen Mikroskopiebildern mit hoher Genauigkeit und nahezu in Echtzeit erkennen und lokalisieren. Das erfolgreiche Trainieren von überwachten Deep-Learning-Modellen erfordert jedoch in der Regel eine ausreichend große Menge an markierten Trainingsbeispielen, die oft nicht ohne weiteres verfügbar sind und deren Beschaffung sehr kostspielig sein kann. Daher schlagen wir zwei Ansätze vor, die auf unbeaufsichtigtem Deep Learning zur Erkennung von Anomalien in der Mikrostruktur von gesinterten NdFeB-Magneten basieren, ohne dass markierte Trainingsdaten benötigt werden. Die Modelle sind in der Lage, Defekte zu erkennen, indem sie aus den Trainingsdaten indikative Merkmale von nur "normalen" Mikrostrukturmustern lernen. Wir zeigen experimentelle Ergebnisse der vorgeschlagenen Fehlererkennungssysteme, indem wir eine Qualitätsbewertung an kommerziellen Proben von Lithium-Ionen-Batterien und gesinterten NdFeB-Magneten durchführen

Advances in Image Processing, Analysis and Recognition Technology

For many decades, researchers have been trying to make computers’ analysis of images as effective as the system of human vision is. For this purpose, many algorithms and systems have previously been created. The whole process covers various stages, including image processing, representation and recognition. The results of this work can be applied to many computer-assisted areas of everyday life. They improve particular activities and provide handy tools, which are sometimes only for entertainment, but quite often, they significantly increase our safety. In fact, the practical implementation of image processing algorithms is particularly wide. Moreover, the rapid growth of computational complexity and computer efficiency has allowed for the development of more sophisticated and effective algorithms and tools. Although significant progress has been made so far, many issues still remain, resulting in the need for the development of novel approaches

Egocentric vision-based passive dietary intake monitoring

Egocentric (first-person) perception captures and reveals how people perceive their surroundings. This unique perceptual view enables passive and objective monitoring of human-centric activities and behaviours. In capturing egocentric visual data, wearable cameras are used. Recent advances in wearable technologies have enabled wearable cameras to be lightweight, accurate, and with long battery life, making long-term passive monitoring a promising solution for healthcare and human behaviour understanding. In addition, recent progress in deep learning has provided an opportunity to accelerate the development of passive methods to enable pervasive and accurate monitoring, as well as comprehensive modelling of human-centric behaviours. This thesis investigates and proposes innovative egocentric technologies for passive dietary intake monitoring and human behaviour analysis. Compared to conventional dietary assessment methods in nutritional epidemiology, such as 24-hour dietary recall (24HR) and food frequency questionnaires (FFQs), which heavily rely on subjects’ memory to recall the dietary intake, and trained dietitians to collect, interpret, and analyse the dietary data, passive dietary intake monitoring can ease such burden and provide more accurate and objective assessment of dietary intake. Egocentric vision-based passive monitoring uses wearable cameras to continuously record human-centric activities with a close-up view. This passive way of monitoring does not require active participation from the subject, and records rich spatiotemporal details for fine-grained analysis. Based on egocentric vision and passive dietary intake monitoring, this thesis proposes: 1) a novel network structure called PAR-Net to achieve accurate food recognition by mining discriminative food regions. PAR-Net has been evaluated with food intake images captured by wearable cameras as well as those non-egocentric food images to validate its effectiveness for food recognition; 2) a deep learning-based solution for recognising consumed food items as well as counting the number of bites taken by the subjects from egocentric videos in an end-to-end manner; 3) in light of privacy concerns in egocentric data, this thesis also proposes a privacy-preserved solution for passive dietary intake monitoring, which uses image captioning techniques to summarise the image content and subsequently combines image captioning with 3D container reconstruction to report the actual food volume consumed. Furthermore, a novel framework that integrates food recognition, hand tracking and face recognition has also been developed to tackle the challenge of assessing individual dietary intake in food sharing scenarios with the use of a panoramic camera. Extensive experiments have been conducted. Tested with both laboratory (captured in London) and field study data (captured in Africa), the above proposed solutions have proven the feasibility and accuracy of using the egocentric camera technologies with deep learning methods for individual dietary assessment and human behaviour analysis.Open Acces

Deep Visual Learning with Less Labeled Data

The rapid development of deep learning has revolutionized various vision tasks, but the success relies heavily on supervised training with large-scale labeled datasets, which can be costly and laborious to acquire. In this context, semi-supervised learning (SSL) has emerged as a promising approach to facilitating deep visual learning with less labeled data. Despite numerous research endeavours in SSL, some technical issues, e.g., the low unlabeled utilization and instance-discriminating, have not been well studied. This thesis emphasizes the cruciality of these issues and proposes new methods for semi-supervised classification (SSC) and semantic segmentation (SSS). In SSC, recent studies are limited in excluding samples with low-confidence predictions and underutilization of label information. Hence, we propose a Label-guided Self-training approach to SSL, which exploits label information to employ a class-aware contrastive loss and buffer-aided label propagation algorithm to fully utilize all unlabeled data. Furthermore, most SSC assumes labeled and unlabeled datasets share an identical class distribution, which is hard to meet in practice. The distribution mismatch between the two sets causes severe bias and performance degradation. We thus propose the Distribution Consistency SSL to address the mismatch from a distribution perspective. In SSS, most studies treat all unlabeled data equally and barely consider different training difficulties among unlabeled instances. We highlight instance differences and propose instance-specific and model-adaptive supervision for SSS. We also study semi-supervised medical image segmentation, where labeled data is scarce. Unlike current increasingly complicated methods, we propose a simple yet effective approach that applies data perturbation and model stabilization strategies to boost performance. Extensive experiments and ablation studies are conducted to verify the superiority of proposed methods on SSC and SSS benchmarks

Microstructure Optimization and Melt Pool Control in Powder Bed Fusion towards the Development of Architected Lattice Materials

Powder bed fusion (PBF) metal additive manufacturing (AM) techniques can shape materials from the micron to centimeter scale and presents the ability to manufacture parts with greater topological complexity than previously allowed by subtractive techniques. The solidification environment varies between electron beam and laser PBF (PBF-EB and PBF-LB, respectively) but is generally characterized by anisotropy, high thermal gradients, and inconsistency in material properties across the scale of the part. This work describes the unique thermal processing conditions of these methods and the microstructures they produce, showing that this anisotropy can manifest through different mechanisms (e.g., residual strains, grain sizes, microstructural morphology) and is driven by part geometry in addition to PBF processing parameters or scan strategy. The relationships between residual strains, microstructural morphology, and laser processing conditions are explored and this enables methods for controlling and influencing the microstructure of a part in real time as it is being manufactured. This has implications towards the application of fine-featured, highly architected, and topologically optimized metallic micro lattices (MML) materials; with the desirable feature resolution (100-500um) overlapping with the length scale of the PBF melt pool. PBF enables the manufacture of fine features, but the melt pool environment is shown to vary across the build volume of a MML cell which leads to microstructural anisotropy across it. Therefore, the solution proposed and described utilizes control of common laser processing parameters (e.g., power or speed) to control the microstructure across the volume of a given part. The topology of a given MML will vary significantly across its volume, and this research indicates that the microstructure of the lattice will as well; unless a variable scan strategy is adopted which can allow for a more real-time control of microstructure. Using in-situ melt pool sensing, a digital twin of the part is created, and provides a method of mapping how changes to laser process variables can influence the solidification environment and microstructure as a function of this changing topology. This has implications for the functional grading, control, validation, and prediction of microstructures within both MML materials and traditionally engineered parts