Enhancing Hierarchical Transformers for Whole Brain Segmentation with Intracranial Measurements Integration

Whole brain segmentation with magnetic resonance imaging (MRI) enables the non-invasive measurement of brain regions, including total intracranial volume (TICV) and posterior fossa volume (PFV). Enhancing the existing whole brain segmentation methodology to incorporate intracranial measurements offers a heightened level of comprehensiveness in the analysis of brain structures. Despite its potential, the task of generalizing deep learning techniques for intracranial measurements faces data availability constraints due to limited manually annotated atlases encompassing whole brain and TICV/PFV labels. In this paper, we enhancing the hierarchical transformer UNesT for whole brain segmentation to achieve segmenting whole brain with 133 classes and TICV/PFV simultaneously. To address the problem of data scarcity, the model is first pretrained on 4859 T1-weighted (T1w) 3D volumes sourced from 8 different sites. These volumes are processed through a multi-atlas segmentation pipeline for label generation, while TICV/PFV labels are unavailable. Subsequently, the model is finetuned with 45 T1w 3D volumes from Open Access Series Imaging Studies (OASIS) where both 133 whole brain classes and TICV/PFV labels are available. We evaluate our method with Dice similarity coefficients(DSC). We show that our model is able to conduct precise TICV/PFV estimation while maintaining the 132 brain regions performance at a comparable level. Code and trained model are available at: https://github.com/MASILab/UNesT/wholebrainSeg

Visualization and Analysis of 3D Microscopic Images

In a wide range of biological studies, it is highly desirable to visualize and analyze three-dimensional (3D) microscopic images. In this primer, we first introduce several major methods for visualizing typical 3D images and related multi-scale, multi-time-point, multi-color data sets. Then, we discuss three key categories of image analysis tasks, namely segmentation, registration, and annotation. We demonstrate how to pipeline these visualization and analysis modules using examples of profiling the single-cell gene-expression of C. elegans and constructing a map of stereotyped neurite tracts in a fruit fly brain

Illuminating tissue organization by imaging the spatial transcriptome

Our bodies consist of a large collection of cells that each have their own function in the organ that they reside in. The cells are grouped by functionality in cell types that arise during development as the result of the gene regulatory network encoded in the genome. With the development of novel single cell technologies, we are starting to understand just how diverse our cells are. In the brain for instance there are at least 3,000 distinguishable types. However, we have little understanding of how all these cell types are spatially organized in the tissue, because conventional labeling and microscopy techniques are incapable of resolving such high complexity in a single experiment. In this thesis I present the development of two methods that can resolve the cellular complexity and spatial organization of mouse and (developmental) human brain samples. These methods are built upon the concept of cyclic RNA labeling with single molecule Fluorescent in situ Hybridization (smFISH) to detect hundreds of gene targets in tissue samples. The resulting RNA localizations can then be used to study spatial gene expression and to identify the cell type of each cell in the sample. The cellular identity and position can then be used to study spatial relationships between cells to understand the tissue architecture. To place the development of these two methods into context, I will first review the field of spatially resolved transcriptomics. I will discuss the methods that are based on microscopy and spatially tagged RNA sequencing, where I will compare their strengths and weaknesses. Then I will present the two projects: Paper I presents the development of a cyclic smFISH protocol called osmFISH that leverages the high detection efficiency of smFISH to measure the gene expression of 33 cell type marker genes in the mouse somatosensory cortex at single cell resolution. We developed the labeling technology, instrumentation and analysis software to enable the study of cellular organization at multiple length scales. Even though osmFISH and related microscopy-based methods generate high quality data they are limited by the spatial throughput so that only small tissue areas can be processed. In paper II I present another method called EEL FISH that uses electrophoresis to transfer the RNA from a 3D tissue section onto a flat surface. The collapsing of one dimension substantially reduces the time needed to image, while retaining the information, so that the complex spatial gene expression profiles of entire mouse brain sections, sub-structures of the human brain and human developmental tissues can be studied. Lastly, I will discuss these results and look at the future of the field of spatially resolved transcriptomics

Serial sectioning block-face imaging of post-mortem human brain

No current imaging technology can directly and without significant distortion visualize the defining microscopic features of the human brain. Ex vivo histological techniques yield exquisite planar images, but the cutting, mounting and staining they require induce slice-specific distortions, introducing cross-slice differences that prohibit true 3D analysis. Clearing techniques have proven difficult to apply to large blocks of human tissue and cause dramatic distortions as well. Thus, we have only a poor understanding of human brain structures that occur at a scale of 1–100 μm, in which neurons are organized into functional cohorts. To date, mesoscopic features which are critical components of this spatial context, have only been quantified in studies of 2D histologic images acquired in a small number of subjects and/or over a small region of the brain, typically in the coronal orientation, implying that features that are oblique or orthogonal to the coronal plane are difficult to properly analyze. A serial sectioning optical coherence tomography (OCT) imaging infrastructure will be developed and utilized to obtain images of cyto- and myelo-architectural features and microvasculature network of post-mortem human brain tissue. Our imaging infrastructure integrates vibratome with imaging head along with pre and post processing algorithms to construct volumetric OCT images of cubic centimeters of brain tissue blocks. Imaging is performed on tissue block-face prior to sectioning, which preserves the 3D information. Serial sections cut from the block can be subsequently treated with multiplexed histological staining of multiple molecular markers that will facilitate cellular classification or imaged with high-resolution transmission birefringence microscope. The successful completion of this imaging infrastructure enables the automated reconstruction of undistorted volume of human tissue brain blocks and permits studying the pathological alternations arising from diseases. Specifically, the mesoscopic and microscopic pathological alternations, as well as the optical properties and cortical morphological alternations of the dorsolateral prefrontal cortical region of two difference neurodegeneration diseases, Chronic Traumatic Encephalopathy (CTE) and Alzheimer’s Disease (AD), were evaluated using this imaging infrastructure