EL-GAN: Embedding Loss Driven Generative Adversarial Networks for Lane Detection

Convolutional neural networks have been successfully applied to semantic segmentation problems. However, there are many problems that are inherently not pixel-wise classification problems but are nevertheless frequently formulated as semantic segmentation. This ill-posed formulation consequently necessitates hand-crafted scenario-specific and computationally expensive post-processing methods to convert the per pixel probability maps to final desired outputs. Generative adversarial networks (GANs) can be used to make the semantic segmentation network output to be more realistic or better structure-preserving, decreasing the dependency on potentially complex post-processing. In this work, we propose EL-GAN: a GAN framework to mitigate the discussed problem using an embedding loss. With EL-GAN, we discriminate based on learned embeddings of both the labels and the prediction at the same time. This results in more stable training due to having better discriminative information, benefiting from seeing both `fake' and `real' predictions at the same time. This substantially stabilizes the adversarial training process. We use the TuSimple lane marking challenge to demonstrate that with our proposed framework it is viable to overcome the inherent anomalies of posing it as a semantic segmentation problem. Not only is the output considerably more similar to the labels when compared to conventional methods, the subsequent post-processing is also simpler and crosses the competitive 96% accuracy threshold.Comment: 14 pages, 7 figure

I Bet You Are Wrong: Gambling Adversarial Networks for Structured Semantic Segmentation

Adversarial training has been recently employed for realizing structured semantic segmentation, in which the aim is to preserve higher-level scene structural consistencies in dense predictions. However, as we show, value-based discrimination between the predictions from the segmentation network and ground-truth annotations can hinder the training process from learning to improve structural qualities as well as disabling the network from properly expressing uncertainties. In this paper, we rethink adversarial training for semantic segmentation and propose to formulate the fake/real discrimination framework with a correct/incorrect training objective. More specifically, we replace the discriminator with a "gambler" network that learns to spot and distribute its budget in areas where the predictions are clearly wrong, while the segmenter network tries to leave no clear clues for the gambler where to bet. Empirical evaluation on two road-scene semantic segmentation tasks shows that not only does the proposed method re-enable expressing uncertainties, it also improves pixel-wise and structure-based metrics.Comment: 13 pages, 8 figure

Nucleus segmentation : towards automated solutions

Single nucleus segmentation is a frequent challenge of microscopy image processing, since it is the first step of many quantitative data analysis pipelines. The quality of tracking single cells, extracting features or classifying cellular phenotypes strongly depends on segmentation accuracy. Worldwide competitions have been held, aiming to improve segmentation, and recent years have definitely brought significant improvements: large annotated datasets are now freely available, several 2D segmentation strategies have been extended to 3D, and deep learning approaches have increased accuracy. However, even today, no generally accepted solution and benchmarking platform exist. We review the most recent single-cell segmentation tools, and provide an interactive method browser to select the most appropriate solution.Peer reviewe

Automatic Object Detection and Categorisation in Deep Astronomical Imaging Surveys Using Unsupervised Machine Learning

I present an unsupervised machine learning technique that automatically segments and labels galaxies in astronomical imaging surveys using only pixel data. Distinct from previous unsupervised machine learning approaches used in astronomy the technique uses no pre-selection or pre-filtering of target galaxy type to identify galaxies that are similar. I demonstrate the technique on the Hubble Space Telescope (HST) Frontier Fields. By training the algorithm using galaxies from one field (Abell 2744) and applying the result to another (MACS0416.1-2403), I show how the algorithm can cleanly separate early and late type galaxies without any form of pre-directed training for what an ‘early’ or ‘late’ type galaxy is. I present the results of testing the technique for generalisation and to identify its optimal configuration. I then apply the technique to the HST Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS) fields, creating a catalogue of 60000 labelled galaxies, grouped by their similarity. I show how the automatically identified groups contain galaxies with similar morphological (and photometric) type. I compare the catalogue to human-classifications from the Galaxy Zoo: CANDELS project. Although there is not a direct mapping, I demonstrate a good level of concordance between them. I publicly release the catalogue and a corresponding visual catalogue and galaxy similarity search facility at www.galaxyml.uk. I show how the technique can be used to identify rarer objects and present lensed galaxy candidates from the CANDELS imaging. Finally, I consider how the technique can be improved and applied to future surveys to identify transient objects

Gradient light interference microscopy for imaging strongly scattering samples

A growing interest in three-dimensional cellular systems has raised new challenges for light microscopy. The fundamental difficulty is the tendency for the optical field to scramble when interacting with turbid media, leading to contrast images. In this work, we outline the development of an instrument that uses broadband optical fields in conjunction with phase-shifting interferometry to extract high-resolution and high-contrast structures from otherwise cloudy images. We construct our system from a differential interference contrast microscope, demonstrating our new modality in transmission and reflection geometries. We call this modality Gradient Light Interference Microscopy (GLIM) as the image measures the gradient of the object’s scattering potential. To facilitate complex experiments, we develop a high-throughput acquisition software and propose several ways to analyze this new kind of data using deep convolutional neural networks. This new proposal, termed phase imaging with computational specificity (PICS), allows for non-destructive yet chemically motivated annotation of microscopy images. The results presented in this dissertation provide templates that are readily extendible to other quantitative phase imaging modalities

Application of Machine Learning Techniques to Classify and Identify Galaxy Merger Events in the Candels Field

Galaxy mergers are dynamic systems that offer us a glimpse into the evolution of the cosmos and the galaxies that constitute it. However, with the advent of large astronomical surveys, it is becoming increasingly difficult to rely on humans to classify the vast number of astronomical images collected every year and find the images that capture these systems. In recent years, researchers have increasingly relied on machine learning and computer vision classifiers, and while these techniques have proven useful for classifying broad galaxy morphologies, they have struggled to identify galaxy mergers. A random forest classifier was applied to a subset of galaxies from the Cosmic Assembly Near-infrared Extragalactic Legacy Survey (CANDELS) to classify merger and non-merger events. 283 merging and 283 non-merging galaxies were selected from the five CANDELS fields, totaling a combined 566 galaxies for training and validation. The classifier was trained on a set of parameters measured for each galaxy, including mass, star formation rate, galactic half-light radius, as well as Concentration and Asymmetry measurements. The classifier performed with a mean accuracy of 92.31% and a precision of 0.9332 on the validation dataset. Additionally, a computer vision convolutional neural network was trained to analyze and classify images of merger and non-merger events in the same fields. Due to the small number of merger events present in the CANDELS fields, data augmentation was utilized to increase the dataset significantly and boost performance. The computer vision classifier performed with an accuracy of 87.87% and a precision of 0.8683 on validation data. The pre-trained convolutional neural network was then used to predicted classes for a dataset containing active galactic nuclei (AGN) hosting galaxies and a control sample, although no correlation was found between predicted classes and whether the galaxy hosts an AGN

Machine learning aided bioimpedance tomography for tissue engineering

In tissue engineering, miniature Electrical Impedance Tomography (mEIT) (or bioimpedance tomography), is an emerging tomographic modality that contributes to non-destructive and label-free imaging and monitoring of 3-D cellular dynamics. The main challenge of mEIT comes from the nonlinear and ill-posed image reconstruction problem, leading to the increased sensitivity to imperfect measurement signals. Physical model-based image reconstruction methods have been successfully applied to conventional setups, but are less satisfying for the mEIT setup regarding image quality, conductivity retrieval and computational efficiency. Data-driven or learning-based methods have recently become a new frontier for tomographic image reconstruction, particularly for medical imaging modalities, e.g., Computed Tomography (CT), Positron Emission Tomography (PET), and Magnetic Resonance Imaging (MRI). However, the study of learning-based image reconstruction methods in challenging micro-scale sensor setups and the seamless integration of such algorithms with the tomography instrument remains a gap. This thesis aims to develop 2-D and 3-D imaging platforms integrating multi-frequency EIT and machine learning-based image reconstruction algorithms to extract spectroscopic electrical properties of 3-D cultivated cells under in vitro conditions, in a non-destructive, robust, and computation-efficient manner. Recent advances in deep learning have pointed out a promising alternative for EIT image reconstruction. However, it is still challenging to image multiple objects with varying conductivity levels with a single neural network. A deep learning and group sparsity regularization-based hybrid image reconstruction framework was proposed to enable high-quality cell culture imaging with mEIT. A deep neural network was proposed to estimate the structural information in binary masks, given the limited number of data sets. Then the structural information is encoded in group sparsity regularization to obtain the final conductivity estimation. We validated our approach by imaging 3D cancer cell spheroids (MCF-7). Our method can be readily translated to spheroids, organoids, and cell culture in scaffolds of biomaterials. As the measured conductivity is a proxy for cell viability, mEIT has excellent potential to enable non-invasive, real-time, long-term monitoring of 3D cell growth, opening new avenues in regenerative medicine and drug testing. Deep learning provides binary structural information in the above-mentioned hybrid learning approach, whereas the regularization algorithm determines conductivity contrasts. Despite the advancement of structure distribution, the exact conductivity values of different objects are less accurately estimated by the regularization-based framework, which essentially prevents EIT’s transition from generating qualitative images to quantitative images. A structure-aware dual-branch deep learning method was proposed to further tackle this issue to predict structure distribution and conductivity values. The proposed network comprises two independent branches to encode the structure and conductivity features, respectively, and the two branches are joined later to make final predictions of conductivity distributions. Numerical and experimental evaluation results on MCF-7 human breast cancer cell spheroids demonstrate the superior performance of the proposed method in dealing with the multi-level, continuous conductivity reconstruction problem. Multi-frequency Electrical Impedance Tomography (mfEIT) is an emerging biomedical imaging modality to reveal frequency-dependent conductivity distributions in biomedical applications. Conventional model-based image reconstruction methods suffer from low spatial resolution, unconstrained frequency correlation and high computational cost. Most existing learning-based approaches deal with the single-frequency setup, which is inefficient and ineffective when extended to the multi-frequency setup. A Multiple Measurement Vector (MMV) model-based learning algorithm named MMV-Net was proposed to solve the mfEIT image reconstruction problem. MMV-Net considers the correlations between mfEIT images and unfolds the update steps of the Alternating Direction Method of Multipliers for the MMV problem (MMV-ADMM). The nonlinear shrinkage operator associated with the weighted l_{1,2} regularization term of MMV-ADMM is generalized in MMV-Net with a cascade of a Spatial Self-Attention module and a Convolutional Long Short-Term Memory (ConvLSTM) module to capture intra- and inter-frequency dependencies better. The proposed MMV-Net was validated on our Edinburgh mfEIT Dataset and a series of comprehensive experiments. The results show superior image quality, convergence performance, noise robustness and computational efficiency against the conventional MMV-ADMM and the state-of-the-art deep learning methods. Finally, few work on image reconstruction for Electrical Impedance Tomography (EIT) focuses on 3D geometries. Existing reconstruction algorithms adopt voxel grids for representation, which typically results in low image quality and considerable computational cost, and limits their applicability to real-time applications. In contrast, point clouds are a more efficient format for 3D surfaces, and such representation can naturally handle 3D shapes of arbitrary topologies with fine-grained details. Therefore, a learning-based 3D EIT reconstruction algorithm with efficient 3D representations (i.e., point cloud) was proposed to achieve higher image accuracy, spatial resolution and computational efficiency. A transformer-like point cloud network is adopted for 3D image reconstruction. This network simultaneously recovers the 3D coordinates of points to adaptively portray the objects' surface and predicts each point's conductivity. The results show that point cloud provides more efficient fine-shape descriptions and effectively alleviates computational costs. In summary, the work demonstrated in this thesis addressed the research void in tissue imaging with bioimpedance tomography by developing learning-based imaging approaches. The results achieved in this thesis could promote bioimpedance tomography as a robust, intelligent imaging technique for tissue engineering applications