Gotta trace ‘em all: A mini-review on tools and procedures for segmenting single neurons toward deciphering the structural connectome

Decoding the morphology and physical connections of all the neurons populating a brain is necessary for predicting and studying the relationships between its form and function, as well as for documenting structural abnormalities in neuropathies. Digitizing a complete and high-fidelity map of the mammalian brain at the micro-scale will allow neuroscientists to understand disease, consciousness, and ultimately what it is that makes us humans. The critical obstacle for reaching this goal is the lack of robust and accurate tools able to deal with 3D datasets representing dense-packed cells in their native arrangement within the brain. This obliges neuroscientist to manually identify the neurons populating an acquired digital image stack, a notably time-consuming procedure prone to human bias. Here we review the automatic and semi-automatic algorithms and software for neuron segmentation available in the literature, as well as the metrics purposely designed for their validation, highlighting their strengths and limitations. In this direction, we also briefly introduce the recent advances in tissue clarification that enable significant improvements in both optical access of neural tissue and image stack quality, and which could enable more efficient segmentation approaches. Finally, we discuss new methods and tools for processing tissues and acquiring images at sub-cellular scales, which will require new robust algorithms for identifying neurons and their sub-structures (e.g., spines, thin neurites). This will lead to a more detailed structural map of the brain, taking twenty-first century cellular neuroscience to the next level, i.e., the Structural Connectome

Rivulet: 3D Neuron Morphology Tracing with Iterative Back-Tracking

The digital reconstruction of single neurons from 3D confocal microscopic images is an important tool for understanding the neuron morphology and function. However the accurate automatic neuron reconstruction remains a challenging task due to the varying image quality and the complexity in the neuronal arborisation. Targeting the common challenges of neuron tracing, we propose a novel automatic 3D neuron reconstruction algorithm, named Rivulet, which is based on the multi-stencils fast-marching and iterative backtracking. The proposed Rivulet algorithm is capable of tracing discontinuous areas without being interrupted by densely distributed noises. By evaluating the proposed pipeline with the data provided by the Diadem challenge and the recent BigNeuron project, Rivulet is shown to be robust to challenging microscopic imagestacks. We discussed the algorithm design in technical details regarding the relationships between the proposed algorithm and the other state-of-the-art neuron tracing algorithms

Learning Approach to Delineation of Curvilinear Structures in 2D and 3D Images

Detection of curvilinear structures has long been of interest due to its wide range of applications. Large amounts of imaging data could be readily used in many fields, but it is practically not possible to analyze them manually. Hence, the need for automated delineation approaches. In the recent years Computer Vision witnessed a paradigm shift from mathematical modelling to data-driven methods based on Machine Learning. This led to improvements in performance and robustness of the detection algorithms. Nonetheless, most Machine Learning methods are general-purpose and they do not exploit the specificity of the delineation problem. In this thesis, we present learning methods suited for this task and we apply them to various kinds of microscopic and natural images, proving the general applicability of the presented solutions. First, we introduce a topology loss - a new training loss term, which captures higher-level features of curvilinear networks such as smoothness, connectivity and continuity. This is in contrast to most Deep Learning segmentation methods that do not take into account the geometry of the resulting prediction. In order to compute the new loss term, we extract topology features of prediction and ground-truth using a pre-trained network, whose filters are activated by structures at different scales and orientations. We show that this approach yields better results in terms of conventional segmentation metrics and overall topology of the resulting delineation. Although segmentation of curvilinear structures provides useful information, it is not always sufficient. In many cases, such as neuroscience and cartography, it is crucial to estimate the network connectivity. In order to find the graph representation of the structure depicted in the image, we propose an approach for joint segmentation and connection classification. Apart from pixel probabilities, this approach also returns the likelihood of a proposed path being a part of the reconstructed network. We show that segmentation and path classification are closely related tasks and can benefit from the synergy. The aforementioned methods rely on Machine Learning, which requires significant amounts of annotated ground-truth data to train models. The labelling process often requires expertise, it is costly and tiresome. To alleviate this problem, we introduce an Active Learning method that significantly decreases the time spent on annotating images. It queries the annotator only about the most informative examples, in this case the hypothetical paths belonging to the structure of interest. Contrary to conventional Active Learning methods, our approach exploits local consistency of linear paths to pick the ones that stand out from their neighborhood. Our final contribution is a method suited for both Active Learning and proofreading the result, which often requires more time than the automated delineation itself. It investigates edges of the delineation graph and determines the ones that are especially significant for the global reconstruction by perturbing their weights. Our Active Learning and proofreading strategies are combined with a new efficient formulation of an optimal subgraph computation and reduce the annotation effort by up to 80%

Tracing biofilaments from images : analysis of existing methods to quantify the three-dimensional growth of filamentous fungi on solid substrates

Orientador: Prof. Dr. David Alexander MitchellCoorientadora: Prof. Dr. Maura Harumi Sugai-GuériosDissertação (mestrado) - Universidade Federal do Paraná, Setor de Tecnologia, Programa de Pós-Graduação em Engenharia Química. Defesa : Curitiba, 29/05/2018Inclui referências: p.100-113Área de concentração: Engenharia QuímicaResumo: Análise de imagens de biofilamentos tem se tornado uma parte importante na pesquisa em biologia e biotecnologia, pois ela não só elucida a morfologia destas estruturas, mas também fornece ideias sobre o desenvolvimento destas estruturas. Além disso, a morfologia pode ser correlacionada com outras variáveis. Por exemplo, análise de imagem de fungos filamentosos permite correlacionar produtividade de enzimas com diferentes morfologias. Há um interesse em compreender como o micélio de um fungo filamentoso se desevolve ao crescer em substratos sólidos. Para estudar isso, foram obtidas imagens 3D do fungo em crescimento em diversos tempos, objetivando computar dados da morfologia e dinâmica de crescimento: velocidades de extensão da colônia e de pontas, número e posição de ramificações e pontas, comprimentos de segmentos, entre outros dados. Porém, antes de computar estes dados, foi feita uma análise da literatura de métodos de traçado de biofilamentos. A análise foi realizada para facilitar a compreensão do vasto número de métodos disponíveis, desde componentes individuais (e.g. técnicas de realçe de filamentos) a workflows completas de traçado de biofilamentos. Também há muitas opções de implementações de software. Na análise, foram incluídas 87 publicações envolvendo workflows de traçado de filamentos ou componentes. Para a análise, criou-se uma classificação (10 classes, que incluem interação com o usuário, abordagem teórica, técnica de imageamento, entre outras classes e 120 sub-classes) para apoiar a análise com o uso de conceitos de teoria de grafos. A metodologia proposta poderá ser utilizada no futuro com ferramentas de semântica web e uma base de dados e permitirá analisar um número maior de dados. Desta análise, identificaram-se os métodos mais comuns de melhoramento de imagem (Realçe de filamentos, 44.9%, suavização, 16.3% e Subtração de background 14.3%) e as tendências em abordagens teóricas (e.g. abordagens baseadas em grafos juntas à algoritmos de aprendizado de máquina, realçe de filamentos como o gradient vector flow seguidos de abordagem Levei-set fast-marching). Após a análise da literatura, foram selecionados os métodos de melhoramento mais comuns e avaliados segundo seu impacto na qualidade da imagem. Os testes foram realizados em duas amostras de imagem (experimentos do crescimento de Aspergillus niger de microscopia confocal de varredura a laser) através de um planejamento fatorial completo e análise do índice de similaridade estrutural, SSIM, e razão sinal-ruído, SNR. Resultados mostraram que o algoritmo rolling bali de subtração de background com raio 20 pixels teve o maior efeito positivo em SSIM e SNR no geral. Então, ao utilizar as imagens melhoradas como entrada, foram testados 5 métodos de traçado de filamentos (APP, APP2, NeuTube, NeuronStudio e NeuroGPS-Tree). Os resultados do traçado foram avaliados qualitativamente: O método NeuTube mostrou os resultados visualmente mais acurados. Definiu-se então o método e foram traçadas as imagens completas 3D e no tempo e obtivemos parâmetros morfométricos e da dinâmica do crescimento do fungo (perfis de biomassa e comprimentos totais, por exemplo). Embora se observe que o uso de traçado de filamentos é promisor para obter mais dados do crescimento de fungos filamentosos, discutiu-se a necessidade de aprimorar as técnicas de preparo de amostra e das configurações na aquisição das imagens, de maneira a aumentar a qualidade final das imagens e fornecer resultados mais confiáveis e concretos após o traçado para então tirar conclusões dos dados. Palavras-chave: fungos filamentosos, filamentos biológicos, análise de imagem, traçado de filamentos, melhoramento de imagem.Abstract: Image analysis of biofilaments is becoming an important part of research on biology and biotechnology because it does not only elucidates the morphology of such structures but also gives insights into their development. Additionally, the morphology can be correlated with other variables. For example, image analysis of filamentous fungi allows the correlation of enzyme productivity with different morphologies. We are interested in understanding how the mycelium of a filamentous fungus develops during growth on solid substrates. In order to study that, time-lapsed 3D images of the fungus during growth were obtained, with the intention of computing growth dynamics and morphometric data: colony and tip extension rates, number and positions of branches and tips, segment lengths, among others. However, prior to computing this data, we analysed the literature of biofilament tracing methods. The analysis was done to facilitate the understanding of the vast number of methods available, from single components (e.g. filament enhancement techniques, and specialized model-based approaches) to complete biofilament tracing workflows. There were also many software implementations options. The analysis comprised 87 publications proposing complete biofilament tracing workflows or workflow components. For the analysis, we created a classification methodology (10 main classes, including user interaction, theoretical approach, imaging technique, among other classes and 120 sub-classes) and analysed the publications using graph theory concepts. The proposed methodology could be used in the future with semantic web tools and crowd-sourced web-based databases, allowing the analysis of greater number of data. Out of this analysis, we identified the most common image enhancement methods (Filament enhancement 44.9%, smoothing 16.3%, background subtraction 14.3%) and the theoretical approach trends for biofilament tracing (e.g. graph-based approaches coupled with machine learning algorithms, image enhancement such as gradient vector flows followed by model-based fast marching approach). Following the literature analysis, we selected the most common image enhancement methods to be used prior to biofilament tracing and evaluated their impact on image quality. The tests were done on two sample images (experiments of the growth of Aspergillus niger on two different carbon sources obtained by confocal laser scanning microscopy) through a full factorial design of experiments and analysis of the structural similarity index, SSIM and signal-to-noise ratio, SNR. Results show that background subtraction (Rolling-ball algorithm, 20 pixels radius) had the most positive effect on SSIM and SNR. Then, using the enhanced images as input, we tested 5 different biofilament tracing methods (APP1, APP2, NeuTube, NeuronStudio and NeuroGPS-Tree). We evaluated the tracing results visually and qualitatively: NeuTube was the method with the most visually accurate results. After choosing NeuTube as the best method, we applied it to our complete 3D time-lapsed images and computed some growth dynamics and morphmetric parameters (e.g. biomass profiles, segment and total lengths). Although we indicate that biofilament tracing methods are a promising approach to obtain more data on the growth of the filamentous fungi, we discuss the need to improve the sample preparation techniques and image acquisition set-up in order to increase the quality of the images so the tracing results provide more reliable and concrete results to draw conclusions. Keywords: filamentous fungi, biological filaments, image analysis, filament tracing, image enhancement

Anatomical and functional characterization of neocortical circuits involved in transforming whisker sensory processing into goal-directed licking

The choice of an action upon perception of an external stimulus, arriving at a sensory organ of an animal, depends on previous experiences and outcomes throughout its life. In the rodent brain, the underlying mechanisms involved in simple sensorimotor transformations, such as the detection of a whisker stimulus through goal-directed licking, still remain largely unknown. In this thesis, using as a model the mouse somatosensory system, I explored the anatomical and functional properties of neuronal circuits at different stages of this cortical processing. To start with, using state-of-the-art viral tracing techniques, I investigated the thalamocortical circuits relaying sensory signals to the primary and secondary whisker somatosensory cortices (wS1, wS2). Challenging the "classical" views, the results indicated two streams of information carrying whisker-selective tactile signals. The principal trigeminal nucleus (Pr5) innervates the ventral posterior medial nucleus of the thalamus (VPM) and finally reaching layer 4 of wS1 while the spinal trigeminal nucleus (Sp5) through the rostral part of the posterior medial (POm) thalamus drives the layer 4 of wS2. Finally, a caudal part of the POm, which does not receive brainstem input, innervates layer 1 and layer 5A. Apart from their anatomical differences, those pathways conveyed distinct whisker sensory signals during goal-directed behaviors. Afterwards, I studied the cortical control of jaw and tongue movements during licking for rewards, using multisensory and multimotor whisker detection tasks. The data revealed a frontal tongue-jaw primary motor area (tjM1) which is necessary and encodes for directional licking, independently of the sensory stimulus type, shedding light on how the neocortex orchestrates the main motor output of the animal. Subsequently, I focused on changes in the L2/3 neuronal networks of wS1 after learning of a whisker stimulus. Using as a benchmark a novel "fast" learning and reward-dependent whisker detection task, I carried out inactivations of wS1 during different stages of learning and chronic two-photon (2P) calcium imaging in the L2/3 of the C2 barrel column. The inactivation results indicated that wS1 is indispensable for the acquisition of the novel stimulus and the execution of the task at expert levels. Moreover, the neural data suggested a learning-induced and "long-lasting" enhancement in the whisker sensory responses even when animals were unmotivated to lick. At a network level, a re-organization of the neuronal circuits was observed at different timescales with some of the alterations accompanying the rapid changes in the animal behavior. Additionally, the changes in the whisker sensory responses of neurons in wS1, after learning, were projection-pathway specific with wS2-projecting neurons showing higher whisker responses than whisker primary motor cortex (wM1)-projecting ones. In the final part, acknowledging the importance of a better characterization of the cortical-cortical communication of wS1, I described recent technical advancements in neuronal reconstructions. In vivo single-cell electroporation combined with 2P tomography and registration to a digital atlas, demonstrated the diversity of the projection targets of neurons in the L2/3 of wS1. Overall, I presented different results which contribute to a pre-existing body of research and help to decipher fundamentals and yet highly complex neural computations of the mammalian brain