Watershed Segmentation of Medical Volumes with Paint Drop Marking

We present an improvement of the classical marker-controlled watershed approach in the direction of a better exploitation of user-defined markers. The combined action of a partial flooding and paint drops falling downwards on the gray value relief from marker locations, leads to a robust and meaningful identification of the candidate basins, which is a prerequisite for an accurate segmentation. This is useful for user-controlled segmentation of biomedical volumes in that it facilitates robust identification of complex 3D structures with inhomogeneous borders. To this end, a visual interactive segmentation system has been implemented where different user-data interaction tools can be selected by physicians to generate machine-understandable knowledge in a quick and compact way. Experimental results on selected use-cases demonstrate the strengths of the proposed solutions

Correlative In Vivo 2-Photon Beam Scanning Electron Microscopy: 3D Analysis of Neuronal Ultrastructure

This protocol describes how dendrites and axons, imaged in vivo, can subsequently be analyzed in 3D using focused ion beam scanning electron microscopy (FIBSEM). The fluorescent structures are identified after chemical fixation and their position highlighted using the 2-photon laser to burn fiducial marks around the region. Once the section has been stained and resin embedded, a small block is trimmed close to these marks. Serially aligned EM images are acquired through this region, using FIBSEM, and the neurites of interest then reconstructed semi-automatically using the Ilastik software (ilastik.org). This fast and reliable imaging and reconstruction technique avoids the use of specific labels to identify the features of interest in the electron microscope and optimizes their preservation for high-quality imaging and 3D analysis

Serial block face scanning electron microscopy and the reconstruction of plant cell membrane systems

Serial block face imaging with the scanning electron microscope has been developed as an alternative to serial sectioning and transmission electron microscopy for the ultrastructural analysis of the three dimensional organisation of cells and tissues. An ultramicrotome within the microscope specimen chamber, permits sectioning and imaging to a depth of many microns within resin embedded specimens. The technology has only recently been adopted by plant microscopists and here we describe some specimen preparation procedures suitable for plant tissue, suggested microscope imaging parameters and discuss the software required for image reconstruction and analysis

Modeling Surfaces from Volume Data Using Nonparallel Contours

Magnetic resonance imaging: MRI) and computed tomography: CT) scanners have long been used to produce three-dimensional samplings of anatomy elements for use in medical visualization and analysis. From such datasets, physicians often need to construct surfaces representing anatomical shapes in order to conduct treatment, such as irradiating a tumor. Traditionally, this is done through a time-consuming and error-prone process in which an experienced scientist or physician marks a series of parallel contours that outline the structures of interest. Recent advances in surface reconstruction algorithms have led to methods for reconstructing surfaces from nonparallel contours that could greatly reduce the manual component of this process. Despite these technological advances, the segmentation process has remained unchanged. This dissertation takes the first steps toward bridging the gap between the new surface reconstruction technologies and bringing those methods to use in clinical practice. We develop VolumeViewer, a novel interface for modeling surfaces from volume data by allowing the user to sketch contours on arbitrarily oriented cross-sections of the volume. We design the algorithms necessary to support nonparallel contouring, and we evaluate the system with medical professionals using actual patient data. In this way, we begin to understand how nonparallel contouring can aid the segmentation process and expose the challenges associated with a nonparallel contouring system in practice

Doctor of Philosophy

dissertationElectron microscopy can visualize synapses at nanometer resolution, and can thereby capture the fine structure of these contacts. However, this imaging method lacks three key elements: temporal information, protein visualization, and large volume reconstruction. For my dissertation, I developed three methods in electron microscopy that overcame these limitations. First, I developed a method to freeze neurons at any desired time point after a stimulus to study synaptic vesicle cycle. Second, I developed a method to couple super-resolution fluorescence microscopy and electron microscopy to pinpoint the location of proteins in electron micrographs at nanometer resolution. Third, I collaborated with computer scientists to develop methods for semi-automated reconstruction of nervous system. I applied these techniques to answer two fundamental questions in synaptic biology. Which vesicles fuse in response to a stimulus? How are synaptic vesicles recovered at synapses after fusion? Only vesicles that are in direct contact with plasma membrane fuse upon stimulation. The active zone in C. elegans is broad, but primed vesicles are concentrated around the dense projection. Following exocytosis of synaptic vesicles, synaptic vesicle membrane was recovered rapidly at two distinct locations at a synapse: the dense projection and adherens junctions. These studies suggest that there may be a novel form of ultrafast endocytosis

Developing a 2D Cross-section Training Strategy for 3D Volume Segmentation by Analyzing Human Perception and Cognitive Tasks

3D volume segmentation is a fundamental process in many scientific and medical applications. Producing accurate segmentations, in an efficient way, is challenging, in part due to low imaging data quality (e.g., noise and low image resolution), and ambiguity in the data that can only be resolved with higher-level knowledge of the structure. Automatic algorithms do exist, but there are many use cases where they fail. The gold standard is still manual segmentation or review. Unfortunately, even for an expert, manual segmentation is laborious, time consuming, and prone to errors. Existing 3D segmentation tools are often designed based on the underlying algorithm, and do not take into account human mental models, their lower-level perception abilities, and higher-level cognitive tasks. In this research, we analyzed manual segmentation as a human-computer interaction paradigm to gain a better understanding of both low-level (perceptual) actions, and higher-level tasks and decision-making processes. We initially employed formative field studies using our novel hybrid protocol that blends observation, surveys, and eye-tracking. We then developed, and validated, data coding schemes to discern segmenters' low-level actions, higher-level tasks, and overall task structures. Using these methods, we successfully identified different segmentation strategies utilized by the segmenters. In addition, formative study results showed that the ability to understand 2D cross-sections of 3D structures is a necessary skill in 3D volume segmentation that can be improved through practice and training. We used the results of our formative studies to introduce a domain-agnostic 2D cross-section training strategy for 3D volume segmentation and developed an interactive training tool to help novices correctly identify 2D cross-sections of 3D structures. To evaluate the effectiveness of our training tool, we designed a novel 2D cross-section test instrument based on various spatial ability factors. We then conducted user studies and used the test instrument to measure participants' performance before and after the training. Study results show that the training tool is effective in improving participants' 2D cross-section understanding skills, which then can be used to perform a more accurate 3D volume segmentation

Connectivity of the Outer Plexiform Layer of the Mouse Retina

The retina has two synaptic layers: In the outer plexiform layer (OPL), signals from the photoreceptors (PRs) are relayed to the bipolar cells (BCs) with one type of horizontal cell (HC) as interneuron. In the inner plexiform layer (IPL), the retinal ganglion cells (RGCs) receive input from the bipolar cells, modulated by multiple types of amacrine cells. The axons of the retinal ganglion cells form the optic nerve which transmit the visual signal to the higher regions of the brain (Masland 2012). Studies of signal processing in the retina usually focus on the inner plexiform layer. Here, the main computations take place such as direction selectivity, orientation selectivity and object motion detection (Gollisch and Meister 2010). However, to fully understand how these computations arise, it is also important to understand how the input to the ganglion cells is computed and thus to understand the functional differences between BC signals. While these are shaped to some extent in the IPL through amacrine cell feedback (Franke et al. 2017), they are also influenced by computations in the OPL (Drinnenberg et al. 2018). Accordingly, it is essential to understand how the bipolar cell signals are formed and what the exact connectivity in the OPL is. This thesis project aims at a quantitative picture of the mouse outer retina connectome. It takes the approach of systematically analyzing connectivity between the cell types in the OPL based on available high-resolution 3D electron microscopy imaging data (Helmstaedter et al. 2013). We reconstructed photoreceptor axon terminals, horizontal cells and bipolar cells, and quantified their contact statistics. We identified a new structure on HC dendrites which likely defines a second synaptic layer in the OPL below the PRs. Based on the reconstructed morphology, we created a biophysical model of a HC dendrite to gain insights into potential functional mechanisms. Our results reveal several new connectivity patterns in the mouse OPL and suggest that HCs perform two functional roles at two distinct output sites at the same time. The project emphasizes how large-scale EM data can boost research on anatomical connectivity and beyond and highlights the value of the resulting data for detailed biophysical modeling. Moreover, it shows how the known amount of complexity increases with the level of detail with which we can study a subject. Beyond that, this thesis project demonstrates the benefits of data sharing and open science which only enabled our studies

A model-based method for 3D reconstruction of cerebellar parallel fibres from high-resolution electron microscope images

In order to understand how the brain works, we need to understand how its neural circuits process information. Electron microscopy remains the only imaging technique capable of providing sufficient resolution to reconstruct the dense connectivity between all neurons in a circuit. Automated electron microscopy techniques are approaching the point where usefully large circuits might be successfully imaged, but the development of automated reconstruction techniques lags far behind. No fully-automated reconstruction technique currently produces acceptably accurate reconstructions, and semi-automated approaches currently require an extreme amount of manual effort. This reconstruction bottleneck places severe limits on the size of neural circuits that can be reconstructed. Improved automated reconstruction techniques are therefore highly desired and under active development. The human brain contains ~86 billion neurons and ~80% of these are located in the cerebellum. Of these cerebellar neurons, the vast majority are granule cells. The axons of these granule cells are called parallel fibres and tend to be oriented in approximately the same direction, making 2+1D reconstruction approaches feasible. In this work we focus on the problem of reconstructing these parallel fibres and make four main contributions: (1) a model-based algorithm for reconstructing 2D parallel fibre cross-sections that achieves state of the art 2D reconstruction performance; (2) a fully-automated algorithm for reconstructing 3D parallel fibres that achieves state of the art 3D reconstruction performance; (3) a semi-automated approach for reconstructing 3D parallel fibres that significantly improves reconstruction accuracy compared to our fully-automated approach while requiring ~40 times less labelling effort than a purely manual reconstruction; (4) a "gold standard" ground truth data set for the molecular layer of the mouse cerebellum that will provide a valuable reference for the development and benchmarking of reconstruction algorithms