Generalizable automated pixel-level structural segmentation of medical and biological data

Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution. This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D structural segmentation in a more generalizable manner, yet has enough adaptability to address a number of specific image modalities, spanning retinal funduscopy, sequential fluorescein angiography and two-photon microscopy. The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D. To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-) pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations. Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this into consideration, we introduce a 5D orientation mapping to capture these orientation properties. This mapping is incorporated into the local feature map description prior to a learning machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods. For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces

ON SYMMETRY: A FRAMEWORK FOR AUTOMATED SYMMETRY DETECTION

Symmetry has weaved itself into almost all fabrics of science, as well as in arts, and has left an indelible imprint on our everyday lives. And, in the same manner, it has pervaded a wide range of areas of computer science, especially computer vision area, and a copious amount of literature has been produced to seek an algorithmic way to identify symmetry in digital data. Notwithstanding decades of endeavor and attempt to have an efficient system that can locate and recover symmetry embedded in real-world images, it is still challenging to fully automate such tasks while maintaining a high level of efficiency. The subject of this thesis is symmetry of imaged objects. Symmetry is one of the non-accidental features of shapes and has long been (maybe mistakenly) speculated as a pre-attentive feature, which improves recognition of quickly presented objects and reconstruction of shapes from incomplete set of measurements. While symmetry is known to provide rich and useful geometric cues to computer vision, it has been barely used as a principal feature for applications because figuring out how to represent and recognize symmetries embedded in objects is a singularly difficult task, both for computer vision and for perceptual psychology. The three main problems addressed in the dissertation are: (i) finding approximate symmetry by identifying the most prominent axis of symmetry out of an entire region, (ii) locating bilaterally symmetrical areas from a scene, and (iii) automating the process of symmetry recovery by solving the problems mentioned above. Perfect symmetries are rare in the extreme in natural images and symmetry perception in humans allows for qualification so that symmetry can be graduated based on the degree of structural deformation or replacement error. There have been many approaches to detect approximate symmetry by searching an optimal solution in a form of an exhaustive exploration of the parameter space or surmising the center of mass. The algorithm set out in this thesis circumvents the computationally intensive operations by using geometric constraints of symmetric images, and assumes no prerequisite knowledge of the barycenter. The results from an extensive set of evaluation experiments on metrics for symmetry distance and a comparison of the performance between the method presented in this thesis and the state of the art approach are demonstrated as well. Many biological vision systems employ a special computational strategy to locate regions of interest based on local image cues while viewing a compound visual scene. The method taken in this thesis is a bottom-up approach that causes the observer favors stimuli based on their saliency, and creates a feature map contingent on symmetry. With the help of summed area tables, the time complexity of the proposed algorithm is linear in the size of the image. The distinguished regions are then delivered to the algorithm described above to uncover approximate symmetry

Automated Remote Sensing Image Interpretation with Limited Labeled Training Data

Automated remote sensing image interpretation has been investigated for more than a decade. In early years, most work was based on the assumption that there are sufficient labeled samples to be used for training. However, ground-truth collection is a very tedious and time-consuming task and sometimes very expensive, especially in the field of remote sensing that usually relies on field surveys to collect ground truth. In recent years, as the development of advanced machine learning techniques, remote sensing image interpretation with limited ground-truth has caught the attention of researchers in the fields of both remote sensing and computer science. Three approaches that focus on different aspects of the interpretation process, i.e., feature extraction, classification, and segmentation, are proposed to deal with the limited ground truth problem. First, feature extraction techniques, which usually serve as a pre-processing step for remote sensing image classification are explored. Instead of only focusing on feature extraction, a joint feature extraction and classification framework is proposed based on ensemble local manifold learning. Second, classifiers in the case of limited labeled training data are investigated, and an enhanced ensemble learning method that outperforms state-of-the-art classification methods is proposed. Third, image segmentation techniques are investigated, with the aid of unlabeled samples and spatial information. A semi-supervised self-training method is proposed, which is capable of expanding the number of training samples by its own and hence improving classification performance iteratively. Experiments show that the proposed approaches outperform state-of-the-art techniques in terms of classification accuracy on benchmark remote sensing datasets.4 month

Motion Learning for Dynamic Scene Understanding

An important goal of computer vision is to automatically understand the visual world. With the introduction of deep networks, we see huge progress in static image understanding. However, we live in a dynamic world, so it is far from enough to merely understand static images. Motion plays a key role in analyzing dynamic scenes and has been one of the fundamental research topics in computer vision. It has wide applications in many fields, including video analysis, socially-aware robotics, autonomous driving, etc. In this dissertation, we study motion from two perspectives: geometric and semantic. From the geometric perspective, we aim to accurately estimate the 3D motion (or scene flow) and 3D structure of the scene. Since manually annotating motion is difficult, we propose self-supervised models for scene flow estimation from image and point cloud sequences. From the semantic perspective, we aim to understand the meanings of different motion patterns and first show that motion benefits detecting and tracking objects from videos. Then we propose a framework to understand the intentions and predict the future locations of agents in a scene. Finally, we study the role of motion information in action recognition

Reconstruction and motion estimation of sparsely sampled ionospheric data

COPYRIGHT Attention is drawn to the fact that copyright of this thesis rests with its author. A copy of this thesis has been supplied on condition that anyone who consults it is understood to recognise that its copyright rests with the author and they must not copy it or use material from it except as permitted by law or with the consent of the author. This thesis may be made available for consultation within the University Library and may be photocopied or lent to other libraries for the purposes of consultation

Computational strategies for understanding underwater optical image datasets

Thesis: Ph. D. in Mechanical and Oceanographic Engineering, Joint Program in Oceanography/Applied Ocean Science and Engineering (Massachusetts Institute of Technology, Department of Mechanical Engineering; and the Woods Hole Oceanographic Institution), 2013.Cataloged from PDF version of thesis.Includes bibliographical references (pages 117-135).A fundamental problem in autonomous underwater robotics is the high latency between the capture of image data and the time at which operators are able to gain a visual understanding of the survey environment. Typical missions can generate imagery at rates hundreds of times greater than highly compressed images can be transmitted acoustically, delaying that understanding until after the vehicle has been recovered and the data analyzed. While automated classification algorithms can lessen the burden on human annotators after a mission, most are too computationally expensive or lack the robustness to run in situ on a vehicle. Fast algorithms designed for mission-time performance could lessen the latency of understanding by producing low-bandwidth semantic maps of the survey area that can then be telemetered back to operators during a mission. This thesis presents a lightweight framework for processing imagery in real time aboard a robotic vehicle. We begin with a review of pre-processing techniques for correcting illumination and attenuation artifacts in underwater images, presenting our own approach based on multi-sensor fusion and a strong physical model. Next, we construct a novel image pyramid structure that can reduce the complexity necessary to compute features across multiple scales by an order of magnitude and recommend features which are fast to compute and invariant to underwater artifacts. Finally, we implement our framework on real underwater datasets and demonstrate how it can be used to select summary images for the purpose of creating low-bandwidth semantic maps capable of being transmitted acoustically.by Jeffrey W. Kaeli.Ph. D. in Mechanical and Oceanographic Engineerin

Deep visual generation for automotive design upgrading and market optimising

The rising levels of homogeneity of modern cars in terms of price and functions has made exterior styling increasingly vital for market success. Recently, researchers have attempted to apply deep learning, especially deep generative models, to automotive exterior design, which has enabled machines to deliver diverse novel designs from large-scale data. In this thesis, we argue that recent advancements in deep learning techniques, particularly in deep generation, can be utilised to facilitate different aspects of automotive exterior design, including design generation, evaluation, and market profit predicting. We conducted three independent studies, each providing tailored solutions to specific automotive design scenarios. These include: a study focused on adapting the latest deep generative model to achieve regional modifications in existing designs, and evaluating these adjustments in terms of design aesthetic and prospective profit changes; another study dedicated to developing a predictive model to assess the modernity of existing designs regarding the future fashion trends; and a final study aiming to incorporate the distinctive shape characteristics of a cheetah into the side view designs of cars. This thesis has four main contributions. First, the developed DVM-CAR dataset is the first large-scale automotive dataset containing designs and marketing data over 10 years. It can be used for different types of research needs from multiple disciplines. Second, given the inherent constraints in automotive design, such as the need to maintain “family face”, and the fact that unconstrained design generation can be seen as a special form of regional modification, our research distinctively focuses on the regional modifications to existing designs, a departure from existing studies. Third, our studies are the first works that integrate the design modules with market profit optimisation. This reforms the traditional product design optimisation frameworks by replacing the abridged design profiles with graphical designs. Finally, the proposed data-driven measures offer effective approaches for automotive aesthetic evaluation and market forecasting, including approaches that can make assessments from a dynamic perspective by examining the evolving fashion trends

Proceedings of the First International Workshop on Mathematical Foundations of Computational Anatomy (MFCA'06) - Geometrical and Statistical Methods for Modelling Biological Shape Variability

International audienceNon-linear registration and shape analysis are well developed research topic in the medical image analysis community. There is nowadays a growing number of methods that can faithfully deal with the underlying biomechanical behaviour of intra-subject shape deformations. However, it is more difficult to relate the anatomical shape of different subjects. The goal of computational anatomy is to analyse and to statistically model this specific type of geometrical information. In the absence of any justified physical model, a natural attitude is to explore very general mathematical methods, for instance diffeomorphisms. However, working with such infinite dimensional space raises some deep computational and mathematical problems. In particular, one of the key problem is to do statistics. Likewise, modelling the variability of surfaces leads to rely on shape spaces that are much more complex than for curves. To cope with these, different methodological and computational frameworks have been proposed. The goal of the workshop was to foster interactions between researchers investigating the combination of geometry and statistics for modelling biological shape variability from image and surfaces. A special emphasis was put on theoretical developments, applications and results being welcomed as illustrations. Contributions were solicited in the following areas: * Riemannian and group theoretical methods on non-linear transformation spaces * Advanced statistics on deformations and shapes * Metrics for computational anatomy * Geometry and statistics of surfaces 26 submissions of very high quality were recieved and were reviewed by two members of the programm committee. 12 papers were finally selected for oral presentations and 8 for poster presentations. 16 of these papers are published in these proceedings, and 4 papers are published in the proceedings of MICCAI'06 (for copyright reasons, only extended abstracts are provided here)

Vision-based representation and recognition of human activities in image sequences

Magdeburg, Univ., Fak. für Elektrotechnik und Informationstechnik, Diss., 2013von Samy Sadek Mohamed Bakhee