A Deep Learning Framework for Automated Vesicle Fusion Detection

Quantitative analysis of vesicle-plasma membrane fusion events in the fluorescence microscopy, has been proven to be important in the vesicle exocytosis study. In this paper, we present a framework to automatically detect fusion events. First, an iterative searching algorithm is developed to extract image patch sequences containing potential events. Then, we propose an event image to integrate the critical image patches of a candidate event into a single-image joint representation as the input to Convolutional Neural Networks (CNNs). According to the duration of candidate events, we design three CNN architectures to automatically learn features for the fusion event classification. Compared on 9 challenging datasets, our proposed method showed very competitive performance and outperformed two state-of-the-arts

Attention mechanism in deep neural networks for computer vision tasks

“Attention mechanism, which is one of the most important algorithms in the deep Learning community, was initially designed in the natural language processing for enhancing the feature representation of key sentence fragments over the context. In recent years, the attention mechanism has been widely adopted in solving computer vision tasks by guiding deep neural networks (DNNs) to focus on specific image features for better understanding the semantic information of the image. However, the attention mechanism is not only capable of helping DNNs understand semantics, but also useful for the feature fusion, visual cue discovering, and temporal information selection, which are seldom researched. In this study, we take the classic attention mechanism a step further by proposing the Semantic Attention Guidance Unit (SAGU) for multi-level feature fusion to tackle the challenging Biomedical Image Segmentation task. Furthermore, we propose a novel framework that consists of (1) Semantic Attention Unit (SAU), which is an advanced version of SAGU for adaptively bringing high-level semantics to mid-level features, (2) Two-level Spatial Attention Module (TSPAM) for discovering multiple visual cues within the image, and (3) Temporal Attention Module (TAM) for temporal information selection to solve the Videobased Person Re-identification task. To validate our newly proposed attention mechanisms, extensive experiments are conducted on challenging datasets. Our methods obtain competitive performance and outperform state-of-the-art methods. Selective publications are also presented in the Appendix”--Abstract, page iii

Doctor of Philosophy

dissertationElectron microscopy can visualize synapses at nanometer resolution, and can thereby capture the fine structure of these contacts. However, this imaging method lacks three key elements: temporal information, protein visualization, and large volume reconstruction. For my dissertation, I developed three methods in electron microscopy that overcame these limitations. First, I developed a method to freeze neurons at any desired time point after a stimulus to study synaptic vesicle cycle. Second, I developed a method to couple super-resolution fluorescence microscopy and electron microscopy to pinpoint the location of proteins in electron micrographs at nanometer resolution. Third, I collaborated with computer scientists to develop methods for semi-automated reconstruction of nervous system. I applied these techniques to answer two fundamental questions in synaptic biology. Which vesicles fuse in response to a stimulus? How are synaptic vesicles recovered at synapses after fusion? Only vesicles that are in direct contact with plasma membrane fuse upon stimulation. The active zone in C. elegans is broad, but primed vesicles are concentrated around the dense projection. Following exocytosis of synaptic vesicles, synaptic vesicle membrane was recovered rapidly at two distinct locations at a synapse: the dense projection and adherens junctions. These studies suggest that there may be a novel form of ultrafast endocytosis

Automated detection and localization of synaptic vesicles in electron microscopy images

Information transfer and integration in the brain occurs at chemical synapses and is mediated by the fusion of synaptic vesicles filled with neurotransmitter. Synaptic vesicle dynamic spatial organization regulates synaptic transmission as well as synaptic plasticity. Because of their small size, synaptic vesicles require electron microscopy for their imaging, and their analysis is conducted manually. The manual annotation and segmentation of the hundreds to thousands of synaptic vesicles, is highly time consuming and limits the throughput of data collection. To overcome this limitation, we built an algorithm, mainly relying on convolutional neural networks, capable of automatically detecting and localizing synaptic vesicles in electron micrographs. The algorithm was trained on murine synapses but we show that it works well on synapses from different species, ranging from zebrafish to human, and from different preparations. As output, we provide the vesicles count and coordinates, the nearest neighbor distance and the estimate of the vesicles area. We also provide a graphical user interface (GUI) to guide users through image analysis, result visualization and manual proof-reading. The application of our algorithm is especially recommended for images produced by transmission electron microscopy. Since this type of imaging is used routinely to investigate presynaptic terminals, our solution will likely be of interest for numerous research groups. SIGNIFICANCE STATEMENT: The analysis of synaptic vesicles provides important insights towards the understanding of synaptic transmission and plasticity mechanisms. However, up to date, this analysis is still a very time-consuming manual process. In the present study we present a user-friendly algorithm, mainly based on convolutional neural networks, for automating the detection of synaptic vesicles in electron micrographs. This approach allows faster and more standardized analyses