Patch-based nonlinear image registration for gigapixel whole slide images

Producción CientíficaImage registration of whole slide histology images allows the fusion of fine-grained information-like different immunohistochemical stains-from neighboring tissue slides. Traditionally, pathologists fuse this information by looking subsequently at one slide at a time. If the slides are digitized and accurately aligned at cell level, automatic analysis can be used to ease the pathologist's work. However, the size of those images exceeds the memory capacity of regular computers. Methods: We address the challenge to combine a global motion model that takes the physical cutting process of the tissue into account with image data that is not simultaneously globally available. Typical approaches either reduce the amount of data to be processed or partition the data into smaller chunks to be processed separately. Our novel method first registers the complete images on a low resolution with a nonlinear deformation model and later refines this result on patches by using a second nonlinear registration on each patch. Finally, the deformations computed on all patches are combined by interpolation to form one globally smooth nonlinear deformation. The NGF distance measure is used to handle multistain images. Results: The method is applied to ten whole slide image pairs of human lung cancer data. The alignment of 85 corresponding structures is measured by comparing manual segmentations from neighboring slides. Their offset improves significantly, by at least 15%, compared to the low-resolution nonlinear registration. Conclusion/Significance: The proposed method significantly improves the accuracy of multistain registration which allows us to compare different antibodies at cell level

Multi-modal matching of 2D images with 3D medical data

Image registration is the process of aligning images of the same object taken at different time points or with different imaging modalities with the aim to compare them in one coordinate system. Image registration is particularly important in biomedical imaging, where a multitude of imaging modalities exist. For example, images can be obtained with X-ray computed tomography (CT) which is based on the object’s X-ray beam attenuation whereas magnetic resonance imaging (MRI) underlines its local proton density. The gold standard in pathology for tissue analysis is histology. Histology, however, provides only 2D information in the selected sections of the 3D tissue. To evaluate the tissue’s 3D structure, volume imaging techniques, such as CT or MRI, are preferable. The combination of functional information from histology with 3D morphological data from CT is essential for tissue analysis. Furthermore, histology can validate anatomical features identified in CT data. Therefore, the registration of these two modalities is indispensable to provide a more complete overview of the tissue. Previously proposed algorithms for the registration of histological slides into 3D volumes usually rely on manual interactions, which is time-consuming and prone to bias. The high complexity of this type of registration originates from the large number of degrees of freedom. The goal of my thesis was to develop an automatic method for histology to 3D volume registration to master these challenges. The first stage of the developed algorithm uses a scale-invariant feature detector to find common matches between the histology slide and each tomography slice in a 3D dataset. A plane of the most likely position is then fitted into the feature point cloud using a robust model fitting algorithm. The second stage builds upon the first one and introduces fine-tuning of the slice position using normalized Mutual Information (NMI). Additionally, using previously developed 2D-2D registration techniques we find the rotation and translation of the histological slide within the plane. Moreover, the framework takes into account any potential nonlinear deformations of the histological slides that might occur during tissue preparation. The application of the algorithm to MRI data is investigated in our third work. The developed extension of the multi-modal feature detector showed promising results, however, the registration of a histological slide to the direct MRI volume remains a challenging task

Registration of histology and magnetic resonance imaging of the brain

Combining histology and non-invasive imaging has been attracting the attention of the medical imaging community for a long time, due to its potential to correlate macroscopic information with the underlying microscopic properties of tissues. Histology is an invasive procedure that disrupts the spatial arrangement of the tissue components but enables visualisation and characterisation at a cellular level. In contrast, macroscopic imaging allows non-invasive acquisition of volumetric information but does not provide any microscopic details. Through the establishment of spatial correspondences obtained via image registration, it is possible to compare micro- and macroscopic information and to recover the original histological arrangement in three dimensions. In this thesis, I present: (i) a survey of the literature relative to methods for histology reconstruction with and without the help of 3D medical imaging; (ii) a graph-theoretic method for histology volume reconstruction from sets of 2D sections, without external information; (iii) a method for multimodal 2D linear registration between histology and MRI based on partial matching of shape-informative boundaries

A multimodal computational pipeline for 3D histology of the human brain

ABSTRACT: Ex vivo imaging enables analysis of the human brain at a level of detail that is not possible in vivo with MRI. In particular, histology can be used to study brain tissue at the microscopic level, using a wide array of different stains that highlight different microanatomical features. Complementing MRI with histology has important applications in ex vivo atlas building and in modeling the link between microstructure and macroscopic MR signal. However, histology requires sectioning tissue, hence distorting its 3D structure, particularly in larger human samples. Here, we present an open-source computational pipeline to produce 3D consistent histology reconstructions of the human brain. The pipeline relies on a volumetric MRI scan that serves as undistorted reference, and on an intermediate imaging modality (blockface photography) that bridges the gap between MRI and histology. We present results on 3D histology reconstruction of whole human hemispheres from two donors

Imaging the subthalamic nucleus in Parkinson’s disease

This thesis is comprised of a set of work that aims to visualize and quantify the anatomy, structural variability, and connectivity of the subthalamic nucleus (STN) with optimized neuroimaging methods. The study populations include both healthy cohorts and individuals living with Parkinson's disease (PD). PD was chosen specifically due to the involvement of the STN in the pathophysiology of the disease. Optimized neuroimaging methods were primarily obtained using ultra-high field (UHF) magnetic resonance imaging (MRI). An additional component of this thesis was to determine to what extent UHF-MRI can be used in a clinical setting, specifically for pre-operative planning of deep brain stimulation (DBS) of the STN for patients with advanced PD. The thesis collectively demonstrates that i, MRI research, and clinical applications must account for the different anatomical and structural changes that occur in the STN with both age and PD. ii, Anatomical connections involved in preparatory motor control, response inhibition, and decision-making may be compromised in PD. iii. The accuracy of visualizing and quantifying the STN strongly depends on the type of MR contrast and voxel size. iv, MRI at a field strength of 3 Tesla (T) can under certain circumstances be optimized to produce results similar to that of 7 T at the expense of increased acquisition time

Cellular anatomy of the mouse primary motor cortex.

An essential step toward understanding brain function is to establish a structural framework with cellular resolution on which multi-scale datasets spanning molecules, cells, circuits and systems can be integrated and interpreted1. Here, as part of the collaborative Brain Initiative Cell Census Network (BICCN), we derive a comprehensive cell type-based anatomical description of one exemplar brain structure, the mouse primary motor cortex, upper limb area (MOp-ul). Using genetic and viral labelling, barcoded anatomy resolved by sequencing, single-neuron reconstruction, whole-brain imaging and cloud-based neuroinformatics tools, we delineated the MOp-ul in 3D and refined its sublaminar organization. We defined around two dozen projection neuron types in the MOp-ul and derived an input-output wiring diagram, which will facilitate future analyses of motor control circuitry across molecular, cellular and system levels. This work provides a roadmap towards a comprehensive cellular-resolution description of mammalian brain architecture