Cross-Domain Depth Estimation Network for 3D Vessel Reconstruction in OCT Angiography

Optical Coherence Tomography Angiography (OCTA) has been widely used by ophthalmologists for decision-making due to its superiority in providing caplillary details. Many of the OCTA imaging devices used in clinic provide high-quality 2D en face representations, while their 3D data quality are largely limited by low signal-to-noise ratio and strong projection artifacts, which restrict the performance of depth-resolved 3D analysis. In this paper, we propose a novel 2D-to-3D vessel reconstruction framework based on the 2D en face OCTA images. This framework takes advantage of the detailed 2D OCTA depth map for prediction and thus does not rely on any 3D volumetric data. Based on the data with available vessel depth labels, we first introduce a network with structure constraint blocks to estimate the depth map of blood vessels in other cross-domain en face OCTA data with unavailable labels. Afterwards, a depth adversarial adaptation module is proposed for better unsupervised cross-domain training, since images captured using different devices may suffer from varying image contrast and noise levels. Finally, vessels are reconstructed in 3D space by utilizing the estimated depth map and 2D vascular information. Experimental results demonstrate the effectiveness of our method and its potential to guide subsequent vascular analysis in 3D domain

Toward Improving Safety in Neurosurgery with an Active Handheld Instrument

Microsurgical procedures, such as petroclival meningioma resection, require careful surgical actions in order to remove tumor tissue, while avoiding brain and vessel damaging. Such procedures are currently performed under microscope magnification. Robotic tools are emerging in order to filter surgeons’ unintended movements and prevent tools from entering forbidden regions such as vascular structures. The present work investigates the use of a handheld robotic tool (Micron) to automate vessel avoidance in microsurgery. In particular, we focused on vessel segmentation, implementing a deep-learning-based segmentation strategy in microscopy images, and its integration with a feature-based passive 3D reconstruction algorithm to obtain accurate and robust vessel position. We then implemented a virtual-fixture-based strategy to control the handheld robotic tool and perform vessel avoidance. Clay vascular phantoms, lying on a background obtained from microscopy images recorded during petroclival meningioma surgery, were used for testing the segmentation and control algorithms. When testing the segmentation algorithm on 100 different phantom images, a median Dice similarity coefficient equal to 0.96 was achieved. A set of 25 Micron trials of 80 s in duration, each involving the interaction of Micron with a different vascular phantom, were recorded, with a safety distance equal to 2 mm, which was comparable to the median vessel diameter. Micron’s tip entered the forbidden region 24% of the time when the control algorithm was active. However, the median penetration depth was 16.9 μm, which was two orders of magnitude lower than median vessel diameter. Results suggest the system can assist surgeons in performing safe vessel avoidance during neurosurgical procedures

Multidimensional en-face OCT imaging of the retina.

Fast T-scanning (transverse scanning, en-face) was used to build B-scan or C-scan optical coherence tomography (OCT) images of the retina. Several unique signature patterns of en-face (coronal) are reviewed in conjunction with associated confocal images of the fundus and B-scan OCT images. Benefits in combining T-scan OCT with confocal imaging to generate pairs of OCT and confocal images similar to those generated by scanning laser ophthalmoscopy (SLO) are discussed in comparison with the spectral OCT systems. The multichannel potential of the OCT/SLO system is demonstrated with the addition of a third hardware channel which acquires and generates indocyanine green (ICG) fluorescence images. The OCT, confocal SLO and ICG fluorescence images are simultaneously presented in a two or a three screen format. A fourth channel which displays a live mix of frames of the ICG sequence superimposed on the corresponding coronal OCT slices for immediate multidimensional comparison, is also included. OSA ISP software is employed to illustrate the synergy between the simultaneously provided perspectives. This synergy promotes interpretation of information by enhancing diagnostic comparisons and facilitates internal correction of movement artifacts within C-scan and B-scan OCT images using information provided by the SLO channel

Reconstruction of Three-Dimensional Blood Vessel Model Using Fractal Interpolation

Fractal method is used in the image processing and studying the irregular and the complex shapes in the image. It is also used in the reconstruction and smoothing of one-, two-, and three-dimensional data. In this chapter, we present an interpolating fractal algorithm to reconstruct 3D blood vessels. Firstly, the proposed method determines the blood vessel centerline from the 2D retina image, and then it uses the Douglas-Peucker algorithm to detect the control points. Secondly, we use the 3D fractal interpolation and iterated function systems for the visualization and reconstruction of these blood vessels. The results showed that the obtained reduction rate is between 71 and 94% depending on the tolerance value. The 3D blood vessels model can be reconstructed efficiently by using the 3D fractal interpolation method

Single-breath-hold photoacoustic computed tomography of the breast

We have developed a single-breath-hold photoacoustic computed tomography (SBH-PACT) system to reveal detailed angiographic structures in human breasts. SBH-PACT features a deep penetration depth (4 cm in vivo) with high spatial and temporal resolutions (255 µm in-plane resolution and a 10 Hz 2D frame rate). By scanning the entire breast within a single breath hold (~15 s), a volumetric image can be acquired and subsequently reconstructed utilizing 3D back-projection with negligible breathing-induced motion artifacts. SBH-PACT clearly reveals tumors by observing higher blood vessel densities associated with tumors at high spatial resolution, showing early promise for high sensitivity in radiographically dense breasts. In addition to blood vessel imaging, the high imaging speed enables dynamic studies, such as photoacoustic elastography, which identifies tumors by showing less compliance. We imaged breast cancer patients with breast sizes ranging from B cup to DD cup, and skin pigmentations ranging from light to dark. SBH-PACT identified all the tumors without resorting to ionizing radiation or exogenous contrast, posing no health risks

Generalizable automated pixel-level structural segmentation of medical and biological data

Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution. This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D structural segmentation in a more generalizable manner, yet has enough adaptability to address a number of specific image modalities, spanning retinal funduscopy, sequential fluorescein angiography and two-photon microscopy. The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D. To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-) pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations. Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this into consideration, we introduce a 5D orientation mapping to capture these orientation properties. This mapping is incorporated into the local feature map description prior to a learning machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods. For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces