Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy

Neural circuits can be reconstructed from brain images acquired by serial section electron microscopy. Image analysis has been performed by manual labor for half a century, and efforts at automation date back almost as far. Convolutional nets were first applied to neuronal boundary detection a dozen years ago, and have now achieved impressive accuracy on clean images. Robust handling of image defects is a major outstanding challenge. Convolutional nets are also being employed for other tasks in neural circuit reconstruction: finding synapses and identifying synaptic partners, extending or pruning neuronal reconstructions, and aligning serial section images to create a 3D image stack. Computational systems are being engineered to handle petavoxel images of cubic millimeter brain volumes

Learned versus Hand-Designed Feature Representations for 3d Agglomeration

For image recognition and labeling tasks, recent results suggest that machine learning methods that rely on manually specified feature representations may be outperformed by methods that automatically derive feature representations based on the data. Yet for problems that involve analysis of 3d objects, such as mesh segmentation, shape retrieval, or neuron fragment agglomeration, there remains a strong reliance on hand-designed feature descriptors. In this paper, we evaluate a large set of hand-designed 3d feature descriptors alongside features learned from the raw data using both end-to-end and unsupervised learning techniques, in the context of agglomeration of 3d neuron fragments. By combining unsupervised learning techniques with a novel dynamic pooling scheme, we show how pure learning-based methods are for the first time competitive with hand-designed 3d shape descriptors. We investigate data augmentation strategies for dramatically increasing the size of the training set, and show how combining both learned and hand-designed features leads to the highest accuracy

Annotating Synapses in Large EM Datasets

Reconstructing neuronal circuits at the level of synapses is a central problem in neuroscience and becoming a focus of the emerging field of connectomics. To date, electron microscopy (EM) is the most proven technique for identifying and quantifying synaptic connections. As advances in EM make acquiring larger datasets possible, subsequent manual synapse identification ({\em i.e.}, proofreading) for deciphering a connectome becomes a major time bottleneck. Here we introduce a large-scale, high-throughput, and semi-automated methodology to efficiently identify synapses. We successfully applied our methodology to the Drosophila medulla optic lobe, annotating many more synapses than previous connectome efforts. Our approaches are extensible and will make the often complicated process of synapse identification accessible to a wider-community of potential proofreaders

Domain Adaptive Synapse Detection with Weak Point Annotations

The development of learning-based methods has greatly improved the detection of synapses from electron microscopy (EM) images. However, training a model for each dataset is time-consuming and requires extensive annotations. Additionally, it is difficult to apply a learned model to data from different brain regions due to variations in data distributions. In this paper, we present AdaSyn, a two-stage segmentation-based framework for domain adaptive synapse detection with weak point annotations. In the first stage, we address the detection problem by utilizing a segmentation-based pipeline to obtain synaptic instance masks. In the second stage, we improve model generalizability on target data by regenerating square masks to get high-quality pseudo labels. Benefiting from our high-accuracy detection results, we introduce the distance nearest principle to match paired pre-synapses and post-synapses. In the WASPSYN challenge at ISBI 2023, our method ranks the 1st place

Focused Proofreading: Efficiently Extracting Connectomes from Segmented EM Images

Identifying complex neural circuitry from electron microscopic (EM) images may help unlock the mysteries of the brain. However, identifying this circuitry requires time-consuming, manual tracing (proofreading) due to the size and intricacy of these image datasets, thus limiting state-of-the-art analysis to very small brain regions. Potential avenues to improve scalability include automatic image segmentation and crowd sourcing, but current efforts have had limited success. In this paper, we propose a new strategy, focused proofreading, that works with automatic segmentation and aims to limit proofreading to the regions of a dataset that are most impactful to the resulting circuit. We then introduce a novel workflow, which exploits biological information such as synapses, and apply it to a large dataset in the fly optic lobe. With our techniques, we achieve significant tracing speedups of 3-5x without sacrificing the quality of the resulting circuit. Furthermore, our methodology makes the task of proofreading much more accessible and hence potentially enhances the effectiveness of crowd sourcing

Progress towards mammalian whole-brain cellular connectomics

Neurons are the fundamental structural units of the nervous system i.e., the Neuron Doctrine as the pioneering work of Santiago Ramon y Cajal in the 1880's clearly demonstrated through careful observation of Golgi-stained neuronal morphologies. However, at that time sample preparation, imaging methods and computational tools were either nonexistent or insufficiently developed to permit the precise mapping of an entire brain with all of its neurons and their connections. Some measure of the "mesoscopic" connectional organization of the mammalian brain has been obtained over the past decade by alignment of sparse subsets of labeled neurons onto a reference atlas or via MRI-based diffusion tensor imaging. Neither method, however, provides data on the complete connectivity of all neurons comprising an individual brain. Fortunately, whole-brain cellular connectomics now appears within reach due to recent advances in whole-brain sample preparation and high-throughput electron microscopy (EM), though substantial obstacles remain with respect to large volume electron microscopic acquisitions and automated neurite reconstructions. This perspective examines the current status and problems associated with generating a mammalian whole-brain cellular connectome and argues that the time is right to launch a concerted connectomic attack on a small mammalian whole-brain