Machine learning of hierarchical clustering to segment 2D and 3D images

We aim to improve segmentation through the use of machine learning tools during region agglomeration. We propose an active learning approach for performing hierarchical agglomerative segmentation from superpixels. Our method combines multiple features at all scales of the agglomerative process, works for data with an arbitrary number of dimensions, and scales to very large datasets. We advocate the use of variation of information to measure segmentation accuracy, particularly in 3D electron microscopy (EM) images of neural tissue, and using this metric demonstrate an improvement over competing algorithms in EM and natural images.Comment: 15 pages, 8 figure

VolRoverN: Enhancing Surface and Volumetric Reconstruction for Realistic Dynamical Simulation of Cellular and Subcellular Function

Establishing meaningful relationships between cellular structure and function requires accurate morphological reconstructions. In particular, there is an unmet need for high quality surface reconstructions to model subcellular and synaptic interactions among neurons and glia at nanometer resolution. We address this need with VolRoverN, a software package that produces accurate, efficient, and automated 3D surface reconstructions from stacked 2D contour tracings. While many techniques and tools have been developed in the past for 3D visualization of cellular structure, the reconstructions from VolRoverN meet specific quality criteria that are important for dynamical simulations. These criteria include manifoldness, water-tightness, lack of self- and object-object-intersections, and geometric accuracy. These enhanced surface reconstructions are readily extensible to any cell type and are used here on spiny dendrites with complex morphology and axons from mature rat hippocampal area CA1. Both spatially realistic surface reconstructions and reduced skeletonizations are produced and formatted by VolRoverN for easy input into analysis software packages for neurophysiological simulations at multiple spatial and temporal scales ranging from ion electro-diffusion to electrical cable models

Doctor of Philosophy

dissertationElectron microscopy can visualize synapses at nanometer resolution, and can thereby capture the fine structure of these contacts. However, this imaging method lacks three key elements: temporal information, protein visualization, and large volume reconstruction. For my dissertation, I developed three methods in electron microscopy that overcame these limitations. First, I developed a method to freeze neurons at any desired time point after a stimulus to study synaptic vesicle cycle. Second, I developed a method to couple super-resolution fluorescence microscopy and electron microscopy to pinpoint the location of proteins in electron micrographs at nanometer resolution. Third, I collaborated with computer scientists to develop methods for semi-automated reconstruction of nervous system. I applied these techniques to answer two fundamental questions in synaptic biology. Which vesicles fuse in response to a stimulus? How are synaptic vesicles recovered at synapses after fusion? Only vesicles that are in direct contact with plasma membrane fuse upon stimulation. The active zone in C. elegans is broad, but primed vesicles are concentrated around the dense projection. Following exocytosis of synaptic vesicles, synaptic vesicle membrane was recovered rapidly at two distinct locations at a synapse: the dense projection and adherens junctions. These studies suggest that there may be a novel form of ultrafast endocytosis

Learning-based Segmentation for Connectomics

Recent advances in electron microscopy techniques make it possible to acquire highresolution, isotropic volume images of neural circuitry. In connectomics, neuroscientists seek to obtain the circuit diagram involving all neurons and synapses in such a volume image. Mapping neuron connectivity requires tracing each and every neural process through terabytes of image data. Due to the size and complexity of these volume images, fully automated analysis methods are desperately needed. In this thesis, I consider automated, machine learning-based neurite segmentation approaches based on a simultaneous merge decision of adjacent supervoxels. - Given a learned likelihood of merging adjacent supervoxels, Chapter 4 adapts a probabilistic graphical model which ensures that merge decisions are consistent and the surfaces of final segments are closed. This model can be posed as a multicut optimization problem and is solved with the cutting-plane method. In order to scale to large datasets, a fast search for (and good choice of) violated cycle constraints is crucial. Quantitative experiments show that the proposed closed-surface regularization significantly improves segmentation performance. - In Chapter 5, I investigate whether the edge weights of the previous model can be chosen to minimize the loss with respect to non-local segmentation quality measures (e.g. Rand Index). Suitable w are obtained from a structured learning approach. In the Structured Support Vector Machine formulation, a novel fast enumeration scheme is used to find the most violated constraint. Quantitative experiments show that structured learning can improve upon unstructured methods. Furthermore, I introduce a new approximate, hierarchical and blockwise optimization approach for large-scale multicut segmentation. Using this method, high-quality approximate solutions for large problem instances are found quickly. - Chapter 6 introduces another novel approximate scheme for multicut segmentation -- Cut, Glue&Cut -- which is based on the move-making paradigm. First, the graph is recursively partitioned into small regions (cut phase). Then, for any two adjacent regions, alternative cuts of these two regions define possible moves (glue&cut phase). The proposed algorithm finds segmentations that are { as measured by a loss function { as close to the ground-truth as the global optimum found by exact solvers, while being significantly faster than existing methods. - In order to jointly label resulting segments as well as to label the boundaries between segments, Chapter 7 proposes the Asymmetric Multi-way Cut model, a variant of Multi-way Cut. In this new model, within-class cuts are allowed for some labels, while being forbidden for other labels. Qualitative experiments show when such a formulation can be beneficial. In particular, an application to joint neurite and cell organelle labeling in EM volume images is discussed. - Custom software tools that can cope with the large data volumes common in the field of connectomics are a prerequisite for the implementation and evaluation of novel segmentation techniques. Chapter 3 presents version 1.0 of ilastik, a joint effort of multiple researchers. I have co-written its volume viewing component, volumina. ilastik provides an interactive pixel classification work ow on largerthan-RAM datasets as well as a semi-automated segmentation module useful for acquiring gold standard segmentations. Furthermore, I describe new software for dealing with hierarchies of cell complexes as well as for blockwise image processing operations on large datasets. The different segmentation methods presented in this thesis provide a promising direction towards reaching the required reliability as well as the required data throughput necessary for connectomics applications

Syväoppivat neuroverkot karkaistun lasin laadun arviointiin

In this work, automated methods for counting the number of shards in a picture of broken glass are reviewed. The minimum number of shards in a specific observation field is a standardized requirement for tempered glass. The operator of the tempering machine typically counts the shards manually. Manual counting is laboursome, time consuming and prone to human errors. Several image processing and machine learning methods for automating the counting task are experimented with in this work. The best performing method proved to be a deep learning convolutional neural network combined with a simple postprocessing scheme. The neural network segments each shard in an image and with postprocessing the shard count can be obtained robustly. Architecture of the framework is presented in detail in this work and its performance is evaluated extensively. The framework reached below 5 % mean absolute counting error on the used validation data.Tässä työssä tarkastellaan automaattisia menetelmiä murtuneen lasin sirulukumäärän laskemiseen kameralla otetusta kuvasta. Sirujen minimilukumäärä tietyllä tarkastelualueella on standardoitu vaatimus karkaistulle lasille. Laskennan suorittaa tyypillisesti karkaisukoneen operaattori manuaalisesti. Manuaalinen laskenta on työlästä, aikaa vievää ja altis inhimillisille virheille. Työssä tutkitaan erilaisia kuvankäsittelymenetelmiä ja koneoppimiseen perustuvia menetelmiä sirulaskennan automatisoimiseksi. Parhaaksi menetelmäksi valikoitui syväoppiva konvolutiivinen neuroverkko yhdistettynä yksinkertaiseen jälkikäsittelyyn. Neuroverkko erittelee sirut kuvasta ja jälkikäsittelyllä saadaan laskettua luotettavasti lukumäärä siruille. Systeemin rakenne esitetään työssä seikkaperäisesti ja sen suorituskykyä arvioidaan kattavasti. Menetelmällä saavutettiin alle 5 % keskimääräinen laskentavirhe käytetyllä validointidatalla

Structure-function properties of the gastrodigestive and hepatic systems of zebrafish (Danio rerio)

While they lack mammal-specific organs, zebrafish provide a high degree of resemblance in their genetic profile, molecular mechanisms and organ physiology to humans and have been established as an excellent complementary platform to rodents. However, their use in gastroenterology and hepatology is under-utilised, conceivably due to a lack of digestive system ultrastructural details as most anatomical studies were performed by light and fluorescence imaging. This thesis provides detailed insights into the structure and function of the zebrafish digestive system, particularly the liver. Multimodal bio-imaging approaches were developed in order to investigate the hepatic ultrastructure and function. Using a protocol that renders samples compatible with multiple imaging platforms, we produce a detailed map of the zebrafish gastrodigestive system from organ to subcellular levels. Findings were compared with the rodent/human counterparts and while some differences exist between the zebrafish and the rodent/human hepatic parenchymal cells and biliary system organisations, many similarities, at the sub/cellular levels, were also demonstrated. Using advances in genetics and a protocol that retains endogenous fluorescence within zebrafish at the same time as ultrastructure for electron microscopy, we further investigated key hepatic functional properties (e.g. macromolecular transport routes) by performing albumin injections and studying the liver macrophages. While we demonstrated similarities in the albumin uptake pathway and in the morphology of liver macrophages in zebrafish, we reveal that zebrafish liver macrophages lack of phagocytic function (a key aspect in rodents and human), which may limit their use in hepatic-immune diseases studies. Altogether, our studies provide new insights and novel protocols for the analysis of the zebrafish liver and lay a foundation to further evaluate uptake routes for gastro-digestive research and drug delivery in various diseases

Investigating the build-up of precedence effect using reflection masking

The auditory processing level involved in the build‐up of precedence [Freyman et al., J. Acoust. Soc. Am. 90, 874–884 (1991)] has been investigated here by employing reflection masked threshold (RMT) techniques. Given that RMT techniques are generally assumed to address lower levels of the auditory signal processing, such an approach represents a bottom‐up approach to the buildup of precedence. Three conditioner configurations measuring a possible buildup of reflection suppression were compared to the baseline RMT for four reflection delays ranging from 2.5–15 ms. No buildup of reflection suppression was observed for any of the conditioner configurations. Buildup of template (decrease in RMT for two of the conditioners), on the other hand, was found to be delay dependent. For five of six listeners, with reflection delay=2.5 and 15 ms, RMT decreased relative to the baseline. For 5‐ and 10‐ms delay, no change in threshold was observed. It is concluded that the low‐level auditory processing involved in RMT is not sufficient to realize a buildup of reflection suppression. This confirms suggestions that higher level processing is involved in PE buildup. The observed enhancement of reflection detection (RMT) may contribute to active suppression at higher processing levels