Multimodal retinal image registration using a fast principal component analysis hybrid-based similarity measure

Multimodal retinal images (RI) are extensively used for analysing various eye diseases and conditions such as myopia and diabetic retinopathy. The incorporation of either two or more RI modalities provides complementary structure information in the presence of non-uniform illumination and low-contrast homogeneous regions. It also presents significant challenges for retinal image registration (RIR). This paper investigates how the Expectation Maximization for Principal Component Analysis with Mutual Information (EMPCA-MI) algorithm can effectively achieve multimodal RIR. This iterative hybrid-based similarity measure combines spatial features with mutual information to provide enhanced registration without recourse to either segmentation or feature extraction. Experimental results for clinical multimodal RI datasets comprising colour fundus and scanning laser ophthalmoscope images confirm EMPCA-MI is able to consistently afford superior numerical and qualitative registration performance compared with existing RIR techniques, such as the bifurcation structures method

MEMO: Dataset and Methods for Robust Multimodal Retinal Image Registration with Large or Small Vessel Density Differences

The measurement of retinal blood flow (RBF) in capillaries can provide a powerful biomarker for the early diagnosis and treatment of ocular diseases. However, no single modality can determine capillary flowrates with high precision. Combining erythrocyte-mediated angiography (EMA) with optical coherence tomography angiography (OCTA) has the potential to achieve this goal, as EMA can measure the absolute 2D RBF of retinal microvasculature and OCTA can provide the 3D structural images of capillaries. However, multimodal retinal image registration between these two modalities remains largely unexplored. To fill this gap, we establish MEMO, the first public multimodal EMA and OCTA retinal image dataset. A unique challenge in multimodal retinal image registration between these modalities is the relatively large difference in vessel density (VD). To address this challenge, we propose a segmentation-based deep-learning framework (VDD-Reg) and a new evaluation metric (MSD), which provide robust results despite differences in vessel density. VDD-Reg consists of a vessel segmentation module and a registration module. To train the vessel segmentation module, we further designed a two-stage semi-supervised learning framework (LVD-Seg) combining supervised and unsupervised losses. We demonstrate that VDD-Reg outperforms baseline methods quantitatively and qualitatively for cases of both small VD differences (using the CF-FA dataset) and large VD differences (using our MEMO dataset). Moreover, VDD-Reg requires as few as three annotated vessel segmentation masks to maintain its accuracy, demonstrating its feasibility.Comment: Submitted to IEEE JBH

Microscopic Inner Retinal Hyper-reflective Phenotypes in Retinal and Neurologic Disease

Purpose. We surveyed inner retinal microscopic features in retinal and neurologic disease using a reflectance confocal adaptive optics scanning light ophthalmoscope (AOSLO). Methods. Inner retinal images from 101 subjects affected by one of 38 retinal or neurologic conditions and 11 subjects with no known eye disease were examined for the presence of hyper-reflective features other than vasculature, retinal nerve fiber layer, and foveal pit reflex. The hyper-reflective features in the AOSLO images were grouped based on size, location, and subjective texture. Clinical imaging, including optical coherence tomography (OCT), scanning laser ophthalmoscopy, and fundus photography was analyzed for comparison. Results. Seven categories of hyper-reflective inner retinal structures were identified, namely punctate reflectivity, nummular (disc-shaped) reflectivity, granular membrane, waxy membrane, vessel-associated membrane, microcysts, and striate reflectivity. Punctate and nummular reflectivity also was found commonly in normal volunteers, but the features in the remaining five categories were found only in subjects with retinal or neurologic disease. Some of the features were found to change substantially between follow up imaging months apart. Conclusions. Confocal reflectance AOSLO imaging revealed a diverse spectrum of normal and pathologic hyper-reflective inner and epiretinal features, some of which were previously unreported. Notably, these features were not disease-specific, suggesting that they might correspond to common mechanisms of degeneration or repair in pathologic states. Although prospective studies with larger and better characterized populations, along with imaging of more extensive retinal areas are needed, the hyper-reflective structures reported here could be used as disease biomarkers, provided their specificity is studied further

Multimodal Imaging of Photoreceptor Structure in Choroideremia

Purpose Choroideremia is a progressive X-linked recessive dystrophy, characterized by degeneration of the retinal pigment epithelium (RPE), choroid, choriocapillaris, and photoreceptors. We examined photoreceptor structure in a series of subjects with choroideremia with particular attention to areas bordering atrophic lesions. Methods Twelve males with clinically-diagnosed choroideremia and confirmed hemizygous mutations in the CHM gene were examined. High-resolution images of the retina were obtained using spectral domain optical coherence tomography (SD-OCT) and both confocal and non-confocal split-detector adaptive optics scanning light ophthalmoscope (AOSLO) techniques. Results Eleven CHM gene mutations (3 novel) were identified; three subjects had the same mutation and one subject had two mutations. SD-OCT findings included interdigitation zone (IZ) attenuation or loss in 10/12 subjects, often in areas with intact ellipsoid zones; RPE thinning in all subjects; interlaminar bridges in the imaged areas of 10/12 subjects; and outer retinal tubulations (ORTs) in 10/12 subjects. Only split-detector AOSLO could reliably resolve cones near lesion borders, and such cones were abnormally heterogeneous in morphology, diameter and density. On split-detector imaging, the cone mosaic terminated sharply at lesion borders in 5/5 cases examined. Split-detector imaging detected remnant cone inner segments within ORTs, which were generally contiguous with a central patch of preserved retina. Conclusions Early IZ dropout and RPE thinning on SD-OCT are consistent with previously published results. Evidence of remnant cone inner segments within ORTs and the continuity of the ORTs with preserved retina suggests that these may represent an intermediate state of retinal degeneration prior to complete atrophy. Taken together, these results supports a model of choroideremia in which the RPE degenerates before photoreceptors

In vivo imaging reveals transient microglia recruitment and functional recovery of photoreceptor signaling after injury.

Microglia respond to damage and microenvironmental changes within the central nervous system by morphologically transforming and migrating to the lesion, but the real-time behavior of populations of these resident immune cells and the neurons they support have seldom been observed simultaneously. Here, we have used in vivo high-resolution optical coherence tomography (OCT) and scanning laser ophthalmoscopy with and without adaptive optics to quantify the 3D distribution and dynamics of microglia in the living retina before and after local damage to photoreceptors. Following photoreceptor injury, microglia migrated both laterally and vertically through the retina over many hours, forming a tight cluster within the area of visible damage that resolved over 2 wk. In vivo OCT optophysiological assessment revealed that the photoreceptors occupying the damaged region lost all light-driven signaling during the period of microglia recruitment. Remarkably, photoreceptors recovered function to near-baseline levels after the microglia had departed the injury locus. These results demonstrate the spatiotemporal dynamics of microglia engagement and restoration of neuronal function during tissue remodeling and highlight the need for mechanistic studies that consider the temporal and structural dynamics of neuron-microglia interactions in vivo

Image database system for glaucoma diagnosis support

Tato práce popisuje přehled standardních a pokročilých metod používaných k diagnose glaukomu v ranném stádiu. Na základě teoretických poznatků je implementován internetově orientovaný informační systém pro oční lékaře, který má tři hlavní cíle. Prvním cílem je možnost sdílení osobních dat konkrétního pacienta bez nutnosti posílat tato data internetem. Druhým cílem je vytvořit účet pacienta založený na kompletním očním vyšetření. Posledním cílem je aplikovat algoritmus pro registraci intenzitního a barevného fundus obrazu a na jeho základě vytvořit internetově orientovanou tři-dimenzionální vizualizaci optického disku. Tato práce je součásti DAAD spolupráce mezi Ústavem Biomedicínského Inženýrství, Vysokého Učení Technického v Brně, Oční klinikou v Erlangenu a Ústavem Informačních Technologií, Friedrich-Alexander University, Erlangen-Nurnberg.This master thesis describes a conception of standard and advanced eye examination methods used for glaucoma diagnosis in its early stage. According to the theoretical knowledge, a web based information system for ophthalmologists with three main aims is implemented. The first aim is the possibility to share medical data of a concrete patient without sending his personal data through the Internet. The second aim is to create a patient account based on a complete eye examination procedure. The last aim is to improve the HRT diagnostic method with an image registration algorithm for the fundus and intensity images and create an optic nerve head web based 3D visualization. This master thesis is a part of project based on DAAD co-operation between Department of Biomedical Engineering, Brno University of Technology, Eye Clinic in Erlangen and Department of Computer Science, Friedrich-Alexander University, Erlangen-Nurnberg.

A Hybrid Similarity Measure Framework for Multimodal Medical Image Registration

Medical imaging is widely used today to facilitate both disease diagnosis and treatment planning practice, with a key prerequisite being the systematic process of medical image registration (MIR) to align either mono or multimodal images of different anatomical parts of the human body. MIR utilises a similarity measure (SM) to quantify the level of spatial alignment and is particularly demanding due to the presence of inherent modality characteristics like intensity non-uniformities (INU) in magnetic resonance images and large homogeneous non-vascular regions in retinal images. While various intensity and feature-based SMs exist for MIR, mutual information (MI) has become established because of its computational efficiency and ability to register multimodal images. It is however, very sensitive to interpolation artefacts in the presence of INU with noise and can be compromised when overlapping areas are small. Recently MI-based hybrid variants which combine regional features with intensity have emerged, though these incur high dimensionality and large computational overheads. To address these challenges and secure accurate, efficient and robust registration of images containing high INU, noise and large homogeneous regions, this thesis presents a new hybrid SM framework for 2D multimodal rigid MIR. The framework consistently provides superior quantitative and qualitative performance, while offering a uniquely flexible design trade-off between registration accuracy and computational time. It makes three significant technical contributions to the field: i) An expectation maximisation-based principal component analysis with mutual information (EMPCA-MI) framework incorporating neighbourhood feature information; ii) Two innovative enhancements to reduce information redundancy and improve MI computational efficiency; and iii) an adaptive algorithm to select the most significant principal components for feature selection. The thesis findings conclusively confirm the hybrid SM framework offers an accurate and robust 2D registration solution for challenging multimodal medical imaging datasets, while its inherent flexibility means it can also be extended to the 3D registration domain

Assessing Photoreceptor Structure Associated with Ellipsoid Zone Disruptions Visualized with Optical Coherence Tomography

Purpose: To compare images of photoreceptor layer disruptions obtained with optical coherence tomography (OCT) and adaptive optics scanning light ophthalmoscopy (AOSLO) in a variety of pathologic states.Methods: Five subjects with photoreceptor ellipsoid zone disruption as per OCT and clinical diagnoses of closed-globe blunt ocular trauma (n = 2), macular telangiectasia type 2 (n = 1), blue-cone monochromacy (n = 1), or cone-rod dystrophy (n = 1) were included. Images were acquired within and around photoreceptor lesions using spectral domain OCT, confocal AOSLO, and split-detector AOSLO.Results: There were substantial differences in the extent and appearance of the photoreceptor mosaic as revealed by confocal AOSLO, split-detector AOSLO, and spectral domain OCT en face view of the ellipsoid zone.Conclusion: Clinically available spectral domain OCT, viewed en face or as B-scan, may lead to misinterpretation of photoreceptor anatomy in a variety of diseases and injuries. This was demonstrated using split-detector AOSLO to reveal substantial populations of photoreceptors in areas of no, low, or ambiguous ellipsoid zone reflectivity with en face OCT and confocal AOSLO. Although it is unclear if these photoreceptors are functional, their presence offers hope for therapeutic strategies aimed at preserving or restoring photoreceptor function