New Methods to Improve Large-Scale Microscopy Image Analysis with Prior Knowledge and Uncertainty

Multidimensional imaging techniques provide powerful ways to examine various kinds of scientific questions. The routinely produced data sets in the terabyte-range, however, can hardly be analyzed manually and require an extensive use of automated image analysis. The present work introduces a new concept for the estimation and propagation of uncertainty involved in image analysis operators and new segmentation algorithms that are suitable for terabyte-scale analyses of 3D+t microscopy images

New Methods to Improve Large-Scale Microscopy Image Analysis with Prior Knowledge and Uncertainty

Multidimensional imaging techniques provide powerful ways to examine various kinds of scientific questions. The routinely produced datasets in the terabyte-range, however, can hardly be analyzed manually and require an extensive use of automated image analysis. The present thesis introduces a new concept for the estimation and propagation of uncertainty involved in image analysis operators and new segmentation algorithms that are suitable for terabyte-scale analyses of 3D+t microscopy images.Comment: 218 pages, 58 figures, PhD thesis, Department of Mechanical Engineering, Karlsruhe Institute of Technology, published online with KITopen (License: CC BY-SA 3.0, http://dx.doi.org/10.5445/IR/1000057821

Detecting, segmenting and tracking bio-medical objects

Studying the behavior patterns of biomedical objects helps scientists understand the underlying mechanisms. With computer vision techniques, automated monitoring can be implemented for efficient and effective analysis in biomedical studies. Promising applications have been carried out in various research topics, including insect group monitoring, malignant cell detection and segmentation, human organ segmentation and nano-particle tracking. In general, applications of computer vision techniques in monitoring biomedical objects include the following stages: detection, segmentation and tracking. Challenges in each stage will potentially lead to unsatisfactory results of automated monitoring. These challenges include different foreground-background contrast, fast motion blur, clutter, object overlap and etc. In this thesis, we investigate the challenges in each stage, and we propose novel solutions with computer vision methods to overcome these challenges and help automatically monitor biomedical objects with high accuracy in different cases --Abstract, page iii

Multiple-view microscopy with light-sheet based fluorescence microscope

The axial resolution of any standard single-lens light microscope is lower than its lateral resolution. The ratio is approximately 3-4 when high numerical aperture objective lenses are used (NA 1.2 -1.4) and more than 10 with low numerical apertures (NA 0.2 and below). In biological imaging, the axial resolution is normally insufficient to resolve subcellular phenomena. Furthermore, parts of the images of opaque specimens are often highly degraded or obscured. Multiple-view fluorescence microscopy overcomes both problems simultaneously by recording multiple images of the same specimen along different directions. The images are digitally fused into a single high-quality image. Multiple-view imaging was developed as an extension to the light-sheet based fluorescence microscope (LSFM), a novel technique that seems to be better suited for multiple-view imaging than any other fluorescence microscopy method to date. In this contribution, the LSFM properties, which are important for multiple-view imaging, are characterized and the implementation of LSFM based multiple-view microscopy is described. The important aspects of multiple-view image alignment and fusion are discussed, the published algorithms are reviewed and original solutions are proposed. The advantages and limitations of multiple-view imaging with LSFM are demonstrated using a number of specimens, which range in size from a single yeast cell to an adult fruit fly and to Medaka fish

Computational neuroanatomy of the central complex of drosophila melanogaster

In many different insect species the highly conserved neuropil regions known as the central complex or central body complex have been shown to be important in behaviours such as locomotion, visual memory and courtship conditioning. The aim of this project is to generate accurate quantitative neuroanatomy of the central complex in the fruit fly Drosophila melanogaster. Much of the authoritative neuroanatomy of the fruit fly from past literature has been derived using Golgi stains, and in important cases these data are available only from 2D camera lucida drawings of the neurons and linguistic descriptions of connectivity. These cannot easily be mapped onto 3D template brains or compared directly to our own data. Using GAL4 driver and reporter constructs, some of the findings within these studies could be visualized using immunohistochemistry and confocal microscopy. A range of GAL4 driver lines were selected that particularly had prominent expression in the fan-shaped body. Images of brains from these lines were archived using a web-based 3D image stack archive developed for the sharing and backup of large confocal stacks. This is also the platform which we use to publish the data, so that other researchers can reuse this catalogue and compare their results directly. Each brain was annotated using desktop-based tools for labelling neuropil regions, locating landmarks in image stacks and tracing fine neuronal processes both manually and automatically. The development of the tracing and landmark annotation tools is described, and all of the tools used in this work are available as free software. In order to compare and aggregate these data, which are from many different brains, it is necessary to register each image stack onto some standard template brain. Although this is a well-studied problem in medical imaging, these high resolution scans of the central fly brain are unusual in a number of respects. The relative effectiveness of various methods currently available were tested on this data set. The best registrations were produced by a method that generates free-form deformations based on B-splines (the Computational Morphometry Toolkit), but for much faster registrations, the thin plate spline method based on manual landmarks may be sufficient. The annotated and registered data allows us to produce central complex template images and also files that accurately represent the possible central complex connectivity apparent in these images. One interesting result to arise from these efforts was evidence for a possible connection between the inferior region of the fan-shaped body and the beta lobe of the mushroom body which had previously been missed in these GAL4 lines. In addition, we can identify several connections which appear to be similar to those described in [Hanesch et al., 1989], the canonical paper on the architecture of the Drosophila melanogaster central complex, and describe for the first time their variation statistically. This registered data was also used to suggest a method for classifying layers of expression within the fan-shaped body

Model and Appearance Based Analysis of Neuronal Morphology from Different Microscopy Imaging Modalities

The neuronal morphology analysis is key for understanding how a brain works. This process requires the neuron imaging system with single-cell resolution; however, there is no feasible system for the human brain. Fortunately, the knowledge can be inferred from the model organism, Drosophila melanogaster, to the human system. This dissertation explores the morphology analysis of Drosophila larvae at single-cell resolution in static images and image sequences, as well as multiple microscopy imaging modalities. Our contributions are on both computational methods for morphology quantification and analysis of the influence of the anatomical aspect. We develop novel model-and-appearance-based methods for morphology quantification and illustrate their significance in three neuroscience studies. Modeling of the structure and dynamics of neuronal circuits creates understanding about how connectivity patterns are formed within a motor circuit and determining whether the connectivity map of neurons can be deduced by estimations of neuronal morphology. To address this problem, we study both boundary-based and centerline-based approaches for neuron reconstruction in static volumes. Neuronal mechanisms are related to the morphology dynamics; so the patterns of neuronal morphology changes are analyzed along with other aspects. In this case, the relationship between neuronal activity and morphology dynamics is explored to analyze locomotion procedures. Our tracking method models the morphology dynamics in the calcium image sequence designed for detecting neuronal activity. It follows the local-to-global design to handle calcium imaging issues and neuronal movement characteristics. Lastly, modeling the link between structural and functional development depicts the correlation between neuron growth and protein interactions. This requires the morphology analysis of different imaging modalities. It can be solved using the part-wise volume segmentation with artificial templates, the standardized representation of neurons. Our method follows the global-to-local approach to solve both part-wise segmentation and registration across modalities. Our methods address common issues in automated morphology analysis from extracting morphological features to tracking neurons, as well as mapping neurons across imaging modalities. The quantitative analysis delivered by our techniques enables a number of new applications and visualizations for advancing the investigation of phenomena in the nervous system

Lidar-based Obstacle Detection and Recognition for Autonomous Agricultural Vehicles

Today, agricultural vehicles are available that can drive autonomously and follow exact route plans more precisely than human operators. Combined with advancements in precision agriculture, autonomous agricultural robots can reduce manual labor, improve workflow, and optimize yield. However, as of today, human operators are still required for monitoring the environment and acting upon potential obstacles in front of the vehicle. To eliminate this need, safety must be ensured by accurate and reliable obstacle detection and avoidance systems.In this thesis, lidar-based obstacle detection and recognition in agricultural environments has been investigated. A rotating multi-beam lidar generating 3D point clouds was used for point-wise classification of agricultural scenes, while multi-modal fusion with cameras and radar was used to increase performance and robustness. Two research perception platforms were presented and used for data acquisition. The proposed methods were all evaluated on recorded datasets that represented a wide range of realistic agricultural environments and included both static and dynamic obstacles.For 3D point cloud classification, two methods were proposed for handling density variations during feature extraction. One method outperformed a frequently used generic 3D feature descriptor, whereas the other method showed promising preliminary results using deep learning on 2D range images. For multi-modal fusion, four methods were proposed for combining lidar with color camera, thermal camera, and radar. Gradual improvements in classification accuracy were seen, as spatial, temporal, and multi-modal relationships were introduced in the models. Finally, occupancy grid mapping was used to fuse and map detections globally, and runtime obstacle detection was applied on mapped detections along the vehicle path, thus simulating an actual traversal.The proposed methods serve as a first step towards full autonomy for agricultural vehicles. The study has thus shown that recent advancements in autonomous driving can be transferred to the agricultural domain, when accurate distinctions are made between obstacles and processable vegetation. Future research in the domain has further been facilitated with the release of the multi-modal obstacle dataset, FieldSAFE

Analysis and network simulations of honeybee interneurons responsive to waggle dance vibration signals

BACKGROUND: Honeybees have long fascinated neuroscientists with their highly evolved social structure and rich behavioral repertoire. They sense air vibrations with their antennae, which is vital for several activities during foraging, like waggle dance communication and flight. GOALS: This thesis presents the investigation of the function of an identified vibration-sensitive interneuron, DL-Int-1. Primary goals were the investigation of (i) adaptations during maturation and (ii) the role of DL-Int-1 in networks encoding distance information of waggle dance vibration signals. RESULTS: Visual inspection indicated that DL-Int-1 morphologies had similar gross structure, but were translated, rotated and scaled relative to each other. To enable detailed spatial comparison, an algorithm for the spatial co-registration of neuron morphologies, Reg-MaxS-N was developed and validated. Experimental data from DL-Int-1 was provided by our Japanese collaborators. Comparison of morphologies from newly emerged adult and forager DL-Int-1 revealed minor changes in gross dendritic features and consistent, region-dependent and spatially localized changes in dendritic density. Comparison of electrophysiological response properties showed an increase in firing rate differences between stimulus and non-stimulus periods during maturation. A putative disinhibitory network in the honeybee primary auditory center was proposed based on experimental evidence. Simulations showed that the network was consistent with experimental observations and clarified the central inhibitory role of DL-Int-1 in shaping the network output. RELEVANCE: Reg-MaxS-N presents a novel approach for the spatial co-registration of morphologies. Adaptations in DL-Int-1 morphology during maturation indicate improved connectivity and signal propagation. The central role of DL-Int-1 in a disinhibitory network in the honeybee primary auditory center combined with adaptions in its response properties during maturation could indicate better encoding of distance information from waggle dance vibration sig- nals

Spatial registration of neuron morphologies based on maximization of volume overlap

Background: Morphological features are widely used in the study of neuronal function and pathology. Invertebrate neurons are often structurally stereotypical, showing little variance in gross spatial features but larger variance in their fine features. Such variability can be quantified using detailed spatial analysis, which however requires the morphologies to be registered to a common frame of reference. Results: We outline here new algorithms - Reg-MaxS and Reg-MaxS-N - for co-registering pairs and groups of morphologies, respectively. Reg-MaxS applies a sequence of translation, rotation and scaling transformations, estimating at each step the transformation parameters that maximize spatial overlap between the volumes occupied by the morphologies. We test this algorithm with synthetic morphologies, showing that it can account for a wide range of transformation differences and is robust to noise. Reg-MaxS-N co-registers groups of more than two morphologies by iteratively calculating an average volume and registering all morphologies to this average using Reg-MaxS. We test Reg-MaxS-N using five groups of morphologies from the Droshophila melanogaster brain and identify the cases for which it outperforms existing algorithms and produce morphologies very similar to those obtained from registration to a standard brain atlas. Conclusions: We have described and tested algorithms for co-registering pairs and groups of neuron morphologies. We have demonstrated their application to spatial comparison of stereotypic morphologies and calculation of dendritic density profiles, showing how our algorithms for registering neuron morphologies can enable new approaches in comparative morphological analyses and visualization