AxonDeepSeg: automatic axon and myelin segmentation from microscopy data using convolutional neural networks

Segmentation of axon and myelin from microscopy images of the nervous system provides useful quantitative information about the tissue microstructure, such as axon density and myelin thickness. This could be used for instance to document cell morphometry across species, or to validate novel non-invasive quantitative magnetic resonance imaging techniques. Most currently-available segmentation algorithms are based on standard image processing and usually require multiple processing steps and/or parameter tuning by the user to adapt to different modalities. Moreover, only few methods are publicly available. We introduce AxonDeepSeg, an open-source software that performs axon and myelin segmentation of microscopic images using deep learning. AxonDeepSeg features: (i) a convolutional neural network architecture; (ii) an easy training procedure to generate new models based on manually-labelled data and (iii) two ready-to-use models trained from scanning electron microscopy (SEM) and transmission electron microscopy (TEM). Results show high pixel-wise accuracy across various species: 85% on rat SEM, 81% on human SEM, 95% on mice TEM and 84% on macaque TEM. Segmentation of a full rat spinal cord slice is computed and morphological metrics are extracted and compared against the literature. AxonDeepSeg is freely available at https://github.com/neuropoly/axondeepsegComment: 14 pages, 7 figure

Automated tracing of myelinated axons and detection of the nodes of Ranvier in serial images of peripheral nerves

The development of realistic neuroanatomical models of peripheral nerves for simulation purposes requires the reconstruction of the morphology of the myelinated fibres in the nerve, including their nodes of Ranvier. Currently, this information has to be extracted by semimanual procedures, which severely limit the scalability of the experiments. In this contribution, we propose a supervised machine learning approach for the detailed reconstruction of the geometry of fibres inside a peripheral nerve based on its high-resolution serial section images. Learning from sparse expert annotations, the algorithm traces myelinated axons, even across the nodes of Ranvier. The latter are detected automatically. The approach is based on classifying the myelinated membranes in a supervised fashion, closing the membrane gaps by solving an assignment problem, and classifying the closed gaps for the nodes of Ranvier detection. The algorithm has been validated on two very different datasets: (i) rat vagus nerve subvolume, SBFSEM microscope, 200 × 200 × 200 nm resolution, (ii) rat sensory branch subvolume, confocal microscope, 384 × 384 × 800 nm resolution. For the first dataset, the algorithm correctly reconstructed 88% of the axons (241 out of 273) and achieved 92% accuracy on the task of Ranvier node detection. For the second dataset, the gap closing algorithm correctly closed 96.2% of the gaps, and 55% of axons were reconstructed correctly through the whole volume. On both datasets, training the algorithm on a small data subset and applying it to the full dataset takes a fraction of the time required by the currently used semiautomated protocols. Our software, raw data and ground truth annotations are available at http://hci.iwr.uni-heidelberg.de/Benchmarks/. The development version of the code can be found at https://github.com/RWalecki/ATMA

Automatic Axon and Myelin Segmentation of Microscopy Images and Morphometrics Extraction

Dans le système nerveux, la transmission des signaux électriques se fait par l’intermédiaire des axones de la matière blanche. La plupart de ces axones, aussi connus sous le nom de fibres nerveuses, sont entourés par la gaine de myéline. Le rôle principal de la gaine de myéline est d’accroître la vitesse de transmission du signal nerveux le long de l’axone, un élément crucial pour la communication sur de longues distances. Lors de pathologies démyélinisantes comme la sclérose en plaques, la gaine de myéline des axones du système nerveux central est attaquée par des cellules du système immunitaire. Ceci peut conduire à la dégénérescence de la myéline, qui peut se manifester de diverses façons : une perte du contenu en myéline, une diminution du nombre d’axones myélinisés ou même des dommages axonaux. La microscopie à haute résolution des tissus myélinisés offre l’avantage de pouvoir imager la microstructure du tissu au niveau cellulaire. L’extraction d’information quantitative sur la morphologie passe par la segmentation des axones et gaines de myélines composant le tissu sur les images microscopiques acquises. L’extraction de métriques morphologiques des fibres nerveuses à partir d’image microscopiques pourrait contribuer à plusieurs applications intéressantes : documentation de la morphométrie sur différentes espèces et tissus, étude des origines et effets des maladies démyélinisantes, et validation de nouveaux biomarqueurs d’Imagerie par Résonance Magnétique sensibles au contenu en myéline dans le tissu. L’objectif principal de ce projet de recherche est de concevoir, implémenter et valider un framework de segmentation automatique d’axones et de gaines de myéline sur des images microscopiques et d’en extraire des morphométriques pertinentes. Plusieurs approches de segmentation ont été explorées dans la littérature, mais la plupart ne sont pas totalement automatiques, sont conçues pour une modalité de microscopie spécifique, ou bien leur implémentation n’est pas publiquement disponible pour la communauté scientifique. Deux frameworks de segmentation ont été développés dans le cadre de ce projet : AxonSeg et AxonDeepSeg. Le framework AxonSeg (https://github.com/neuropoly/axonseg) se base sur une approche de traitement d’image classique pour la segmentation. Le pipeline de segmentation inclut une transformée de type extended-minima, un modèle d’analyse discriminante combinant des features de forme et d’intensité, un algorithme de détection de contours et un double algorithme de contours actifs. Le résultat de la segmentation est utilisé pour l’extraction de morphométriques. La validation du framework a été réalisée sur des échantillons de microscopie optique, microscopie électronique et microscopie Raman stimulée (CARS). Le framework AxonDeepSeg (https://github.com/neuropoly/axondeepseg) utilise plutôt une approche basée sur des réseaux neuronaux convolutifs. Un réseau convolutif a été conçu pour la segmentation sémantique des axones myélinisés. Un modèle de microscopie électronique à balayage (MEB) a été entraîné sur des échantillons de moelle épinière de rat et un modèle de microscopie électronique à transmission (MET) a été entraîné sur des échantillons de corps calleux de souris. Les deux modèles ont démontré une haute précision pixel par pixel sur les échantillons test (85% sur le MEB de rat, 81% sur le MEB d’humain, 95% sur le MET de souris, 84% sur le MET de macaque). On démontre également que les modèles entrainés sont robustes aux ajouts de bruit, au flou et aux changements d’intensité. Le modèle MEB de AxonDeepSeg a été utilisé pour segmenter une coupe transversale complète de moelle épinière de rat et les morphométriques extraites à partir des tracts de la matière blanche correspondaient bien aux tendances rapportées dans la littérature. AxonDeepSeg a démontré une plus grande précision au niveau de la segmentation lorsque comparé à AxonSeg. Les deux outils logiciels développés sont open source (licence MIT) et donc à disposition de la communauté scientifique. Des futures itérations sont prévues afin d’améliorer et d’étendre ce travail. Les objectifs à court terme sont l’entraînement de nouveaux modèles pour d’autres modalités de microscopie, l’entraînement sur des datasets plus larges afin d’améliorer la généralisation et la robustesse des modèles, et l’exploration de nouvelles architectures de réseaux neuronaux. De plus, les modèles de segmentations développés jusqu’à maintenant ont seulement été testés sur des images de tissus sains. Un développement futur important serait de tester la performance de ces modèles sur des échantillons démyélinisés.----------ABSTRACT In the nervous system, the transmission of electrical signals is ensured by the axons of the white matter. A large portion of these axons, also known as nerve fibers, is surrounded by a myelin sheath. The main role of the myelin sheath is to increase the transmission speed along the axons, which is crucial for long distance communication. In demyelinating diseases such as multiple sclerosis, the myelin sheath of the central nervous system is attacked by cells of the immune system. Myelin degeneration caused by such disorders can manifest itself in different ways at the microstructural level: loss of myelin content, decrease in the number of myelinated axons, or even axonal damage. High resolution microscopy of myelinated tissues can provide in-depth microstructural information about the tissue under study. Segmentation of the axon and myelin content of a microscopy image is a necessary step in order to extract quantitative morphological information from the tissue. Being able to extract morphometrics from the tissue would benefit several applications: document nerve morphometry across species or tissues, get a better understanding of the origins of demyelinating diseases, and validate novel magnetic resonance imaging biomarkers sensitive to myelin content. The main objective of this research project is to design, implement and validate an automatic axon and myelin segmentation framework for microscopy images and use it to extract relevant morphological metrics. Several segmentation approaches exist in the literature for similar applications, but most of them are not fully automatic, are designed to work on a specific microscopy modality and/or are not made available to the research community. Two segmentation frameworks were developed as part of this project: AxonSeg and AxonDeepSeg. The AxonSeg package (https://github.com/neuropoly/axonseg) uses a segmentation approach based on standard image processing. The segmentation pipeline includes an extendedminima transform, a discriminant analysis model based on shape and intensity features, an edge detection algorithm, and a double active contours step. The segmentation output is used to compute morphological metrics. Validation of the framework was performed on optical, electron and CARS microscopy. The AxonDeepSeg package (https://github.com/neuropoly/axondeepseg) uses a segmentation approach based on convolutional neural networks. A fully convolutional network architecture was designed for the semantic 3-class segmentation of myelinated axons. A scanning electron microscopy (SEM) model trained on rat spinal cord samples and a transmission electron microscopy (TEM) model trained on mice corpus callosum samples are presented. Both models presented high pixel-wise accuracy on test datasets (85% on rat SEM, 81% on human SEM, 95% on mice TEM and 84% on macaque TEM). We show that AxonDeepSeg models are robust to noise, blurring and intensity changes. AxonDeepSeg was used to segment a full rat spinal cord slice, and morphological metrics extracted from white matter tracks correlated well with the literature. The AxonDeepSeg framework presented a higher segmentation accuracy when compared to AxonSeg. Both AxonSeg and AxonDeepSeg are open source (MIT license) and thus freely available for use by the research community. Future iterations are planned to improve and extend this work. Training of new models for other microscopy modalities, training on larger datasets to improve generalization and robustness, and exploration of novel deep learning architectures are some of the short-term objectives. Moreover, the current segmentation models have only been tested on healthy tissues. Another important short-term objective would be to assess the performance of these models on demyelinated samples

AxonSeg: open source software for axon and myelin segmentation and morphometric analysis

Segmenting axon and myelin from microscopic images is relevant for studying the peripheral and central nervous system and for validating new MRI techniques that aim at quantifying tissue microstructure. While several software packages have been proposed, their interface is sometimes limited and/or they are designed to work with a specific modality (e.g., scanning electron microscopy (SEM) only). Here we introduce AxonSeg, which allows to perform automatic axon and myelin segmentation on histology images, and to extract relevant morphometric information, such as axon diameter distribution, axon density and the myelin g-ratio. AxonSeg includes a simple and intuitive MATLABbased graphical user interface (GUI) and can easily be adapted to a variety of imaging modalities. The main steps of AxonSeg consist of: (i) image pre-processing; (ii) pre-segmentation of axons over a cropped image and discriminant analysis (DA) to select the best parameters based on axon shape and intensity information; (iii) automatic axon and myelin segmentation over the full image; and (iv) atlas-based statistics to extract morphometric information. Segmentation results from standard optical microscopy (OM), SEM and coherent anti-Stokes Raman scattering (CARS) microscopy are presented, along with validation against manual segmentations. Being fully-automatic after a quick manual intervention on a cropped image, we believe AxonSeg will be useful to researchers interested in large throughput histology

Automatic Morphometry of Nerve Histological Sections

A method for the automatic segmentation, recognition and measurement of neuronal myelinated fibers in nerve histological sections is presented. In this method, the fiber parameters i.e. perimeter, area, position of the fiber and myelin sheath thickness are automatically computed. Obliquity of the sections may be taken into account. First, the image is thresholded to provide a coarse classification between myelin and non-myelin pixels. Next, the resulting binary image is further simplified using connected morphological operators. By applying semantic rules to the zonal graph axon candidates are identified. Those are either isolated or still connected. Then, separation of connected fibers is performed by evaluating myelin sheath thickness around each candidate area with an Euclidean distance transformation. Finally, properties of each detected fiber are computed and false positives are removed. The accuracy of the method is assessed by evaluating missed detection, false positive ratio and comparing the results to the manual procedure with sampling. In the evaluated nerve surface, a 0.9% of false positives was found, along with 6.36% of missed detections. The resulting histograms show strong correlation with those obtained by manual measure. The noise introduced by this method is significantly lower than the intrinsic sampling variability. This automatic method constitutes an original tool for morphometrical analysis

Calibration of the stereological estimation of the number of myelinated axons in the rat sciatic nerve: a multicenter study.

Several sources of variability can affect stereological estimates. Here we measured the impact of potential sources of variability on numerical stereological estimates of myelinated axons in the adult rat sciatic nerve. Besides biological variation, parameters tested included two variations of stereological methods (unbiased counting frame versus 2D-disector), two sampling schemes (few large versus frequent small sampling boxes), and workstations with varying degrees of sophistication. All estimates were validated against exhaustive counts of the same nerve cross sections to obtain calibrated true numbers of myelinated axons (gold standard). In addition, we quantified errors in particle identification by comparing light microscopic and electron microscopic images of selected consecutive sections. Biological variation was 15.6%. There was no significant difference between the two stereological approaches or workstations used, but sampling schemes with few large samples yielded larger differences (20.7%±3.7% SEM) of estimates from true values, while frequent small samples showed significantly smaller differences (12.7%±1.9% SEM). Particle identification was accurate in 94% of cases (range: 89–98%). The most common identification error was due to profiles of Schwann cell nuclei mimicking profiles of small myelinated nerve fibers. We recommend sampling frequent small rather than few large areas, and conclude that workstations with basic stereological equipment are sufficient to obtain accurate estimates. Electron microscopic verification showed that particle misidentification had a surprisingly variable and large impact of up to 11%, corresponding to 2/3 of the biological variation (15.6%). Thus, errors in particle identification require further attention, and we provide a simple nerve fiber recognition test to assist investigators with self-testing and training

Beyond imaging with coherent anti-Stokes Raman scattering microscopy

La microscopie optique permet de visualiser des échantillons biologiques avec une bonne sensibilité et une résolution spatiale élevée tout en interférant peu avec les échantillons. La microscopie par diffusion Raman cohérente (CARS) est une technique de microscopie non linéaire basée sur l’effet Raman qui a comme avantage de fournir un mécanisme de contraste endogène sensible aux vibrations moléculaires. La microscopie CARS est maintenant une modalité d’imagerie reconnue, en particulier pour les expériences in vivo, car elle élimine la nécessité d’utiliser des agents de contraste exogènes, et donc les problèmes liés à leur distribution, spécificité et caractère invasif. Cependant, il existe encore plusieurs obstacles à l’adoption à grande échelle de la microscopie CARS en biologie et en médecine : le coût et la complexité des systèmes actuels, les difficultés d’utilisation et d’entretient, la rigidité du mécanisme de contraste, la vitesse de syntonisation limitée et le faible nombre de méthodes d’analyse d’image adaptées. Cette thèse de doctorat vise à aller au-delà de certaines des limites actuelles de l’imagerie CARS dans l’espoir que cela encourage son adoption par un public plus large. Tout d’abord, nous avons introduit un nouveau système d’imagerie spectrale CARS ayant une vitesse de syntonisation de longueur d’onde beaucoup plus rapide que les autres techniques similaires. Ce système est basé sur un laser à fibre picoseconde synchronisé qui est à la fois robuste et portable. Il peut accéder à des lignes de vibration Raman sur une plage importante (2700–2950 cm-1) à des taux allant jusqu’à 10 000 points spectrales par seconde. Il est parfaitement adapté pour l’acquisition d’images spectrales dans les tissus épais. En second lieu, nous avons proposé une nouvelle méthode d’analyse d’images pour l’évaluation de la structure de la myéline dans des images de sections longitudinales de moelle épinière. Nous avons introduit un indicateur quantitatif sensible à l’organisation de la myéline et démontré comment il pourrait être utilisé pour étudier certaines pathologies. Enfin, nous avons développé une méthode automatisé pour la segmentation d’axones myélinisés dans des images CARS de coupes transversales de tissu nerveux. Cette méthode a été utilisée pour extraire des informations morphologique des fibres nerveuses dans des images CARS de grande échelle.Optical-based microscopy techniques can sample biological specimens using many contrast mechanisms providing good sensitivity and high spatial resolution while minimally interfering with the samples. Coherent anti-Stokes Raman scattering (CARS) microscopy is a nonlinear microscopy technique based on the Raman effect. It shares common characteristics of other optical microscopy modalities with the added benefit of providing an endogenous contrast mechanism sensitive to molecular vibrations. CARS is now recognized as a great imaging modality, especially for in vivo experiments since it eliminates the need for exogenous contrast agents, and hence problems related to the delivery, specificity, and invasiveness of those markers. However, there are still several obstacles preventing the wide-scale adoption of CARS in biology and medicine: cost and complexity of current systems as well as difficulty to operate and maintain them, lack of flexibility of the contrast mechanism, low tuning speed and finally, poor accessibility to adapted image analysis methods. This doctoral thesis strives to move beyond some of the current limitations of CARS imaging in the hope that it might encourage a wider adoption of CARS as a microscopy technique. First, we introduced a new CARS spectral imaging system with vibrational tuning speed many orders of magnitude faster than other narrowband techniques. The system presented in this original contribution is based on a synchronized picosecond fibre laser that is both robust and portable. It can access Raman lines over a significant portion of the highwavenumber region (2700–2950 cm-1) at rates of up to 10,000 spectral points per second and is perfectly suitable for the acquisition of CARS spectral images in thick tissue. Secondly, we proposed a new image analysis method for the assessment of myelin health in images of longitudinal sections of spinal cord. We introduced a metric sensitive to the organization/disorganization of the myelin structure and showed how it could be used to study pathologies such as multiple sclerosis. Finally, we have developped a fully automated segmentation method specifically designed for CARS images of transverse cross sections of nerve tissue.We used our method to extract nerve fibre morphology information from large scale CARS images