Sliding to predict: vision-based beating heart motion estimation by modeling temporal interactions

Purpose: Technical advancements have been part of modern medical solutions as they promote better surgical alternatives that serve to the benefit of patients. Particularly with cardiovascular surgeries, robotic surgical systems enable surgeons to perform delicate procedures on a beating heart, avoiding the complications of cardiac arrest. This advantage comes with the price of having to deal with a dynamic target which presents technical challenges for the surgical system. In this work, we propose a solution for cardiac motion estimation. Methods: Our estimation approach uses a variational framework that guarantees preservation of the complex anatomy of the heart. An advantage of our approach is that it takes into account different disturbances, such as specular reflections and occlusion events. This is achieved by performing a preprocessing step that eliminates the specular highlights and a predicting step, based on a conditional restricted Boltzmann machine, that recovers missing information caused by partial occlusions. Results: We carried out exhaustive experimentations on two datasets, one from a phantom and the other from an in vivo procedure. The results show that our visual approach reaches an average minima in the order of magnitude of 10-7 while preserving the heart’s anatomical structure and providing stable values for the Jacobian determinant ranging from 0.917 to 1.015. We also show that our specular elimination approach reaches an accuracy of 99% compared to a ground truth. In terms of prediction, our approach compared favorably against two well-known predictors, NARX and EKF, giving the lowest average RMSE of 0.071. Conclusion: Our approach avoids the risks of using mechanical stabilizers and can also be effective for acquiring the motion of organs other than the heart, such as the lung or other deformable objects.Peer ReviewedPostprint (published version

Sliding to predict: vision-based beating heart motion estimation by modeling temporal interactions.

PURPOSE: Technical advancements have been part of modern medical solutions as they promote better surgical alternatives that serve to the benefit of patients. Particularly with cardiovascular surgeries, robotic surgical systems enable surgeons to perform delicate procedures on a beating heart, avoiding the complications of cardiac arrest. This advantage comes with the price of having to deal with a dynamic target which presents technical challenges for the surgical system. In this work, we propose a solution for cardiac motion estimation. METHODS: Our estimation approach uses a variational framework that guarantees preservation of the complex anatomy of the heart. An advantage of our approach is that it takes into account different disturbances, such as specular reflections and occlusion events. This is achieved by performing a preprocessing step that eliminates the specular highlights and a predicting step, based on a conditional restricted Boltzmann machine, that recovers missing information caused by partial occlusions. RESULTS: We carried out exhaustive experimentations on two datasets, one from a phantom and the other from an in vivo procedure. The results show that our visual approach reaches an average minima in the order of magnitude of [Formula: see text] while preserving the heart's anatomical structure and providing stable values for the Jacobian determinant ranging from 0.917 to 1.015. We also show that our specular elimination approach reaches an accuracy of 99% compared to a ground truth. In terms of prediction, our approach compared favorably against two well-known predictors, NARX and EKF, giving the lowest average RMSE of 0.071. CONCLUSION: Our approach avoids the risks of using mechanical stabilizers and can also be effective for acquiring the motion of organs other than the heart, such as the lung or other deformable objects

Generalizable automated pixel-level structural segmentation of medical and biological data

Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution. This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D structural segmentation in a more generalizable manner, yet has enough adaptability to address a number of specific image modalities, spanning retinal funduscopy, sequential fluorescein angiography and two-photon microscopy. The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D. To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-) pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations. Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this into consideration, we introduce a 5D orientation mapping to capture these orientation properties. This mapping is incorporated into the local feature map description prior to a learning machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods. For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces

Advances in Biomedical Applications and Assessment of Ultrasound Nonrigid Image Registration.

Image volume based registration (IVBaR) is the process of determining a one-to-one transformation between points in two images that relates the information in one image to that in the other image quantitatively. IVBaR is done primarily to spatially align the two images in the same coordinate system in order to allow better comparison and visualization of changes. The potential use of IVBaR has been explored in three different contexts. In a preliminary study on identification of biometric from internal finger structure, semi-automated IVBaR-based study provided a sensitivity and specificity of 0.93 and 1.00 respectively. Visual matching of all image pairs by four readers yielded 96% successful match. IVBaR could potentially be useful for routine breast cancer screening and diagnosis. Nearly whole breast ultrasound (US) scanning with mammographic-style compression and successful IVBaR were achieved. The image volume was registered off-line with a mutual information cost function and global interpolation based on the non-rigid thin-plate spline deformation. This Institutional Review Board approved study was conducted on 10 patients undergoing chemotherapy and 14 patients with a suspicious/unknown mass scheduled to undergo biopsy. IVBaR was successful with mean registration error (MRE) of 5.2±2 mm in 12 of 17 ABU image pairs collected before, during or after 115±14 days of chemotherapy. Semi-automated tumor volume estimation was performed on registered image volumes giving 86±8% mean accuracy compared with a radiologist hand-segmented tumor volume on 7 cases with correlation coefficient of 0.99 (p<0.001). In a reader study by 3 radiologists assigned to mark the tumor boundary, significant reduction in time taken (p<0.03) was seen due to IVBaR in 6 cases. Three new methods were developed for independent validation of IVBaR based on Doppler US signals. Non-rigid registration tools were also applied in the field of interventional guidance of medical tools used in minimally invasive surgery. The mean positional error in a CT scanner environment improved from 3.9±1.5 mm to 1.0±0.3 mm (p<0.0002). These results show that 3D image volumes and data can be spatially aligned using non-rigid registration for comparison as well as quantification of changes.Ph.D.Applied PhysicsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/64802/1/gnarayan_1.pd

Advanced imaging and data mining technologies for medical and food safety applications

As one of the most fast-developing research areas, biological imaging and image analysis receive more and more attentions, and have been already widely applied in many scientific fields including medical diagnosis and food safety inspection. To further investigate such a very interesting area, this research is mainly focused on advanced imaging and pattern recognition technologies in both medical and food safety applications, which include 1) noise reduction of ultra-low-dose multi-slice helical CT imaging for early lung cancer screening, and 2) automated discrimination between walnut shell and meat under hyperspectral florescence imaging. In the medical imaging and diagnosis area, because X-ray computed tomography (CT) has been applied to screen large populations for early lung cancer detection during the last decade, more and more attentions have been paid to studying low-dose, even ultra-low-dose X-ray CTs. However, reducing CT radiation exposure inevitably increases the noise level in the sinogram, thereby degrading the quality of reconstructed CT images. Thus, how to reduce the noise levels in the low-dose CT images becomes a meaningful topic. In this research, a nonparametric smoothing method with block based thin plate smoothing splines and the roughness penalty was introduced to restore the ultra-low-dose helical CT raw data, which was acquired under 120 kVp / 10 mAs protocol. The objective thorax image quality evaluation was first conducted to assess the image quality and noise level of proposed method. A web-based subjective evaluation system was also built for the total of 23 radiologists to compare proposed approach with traditional sinogram restoration method. Both objective and subjective evaluation studies showed the effectiveness of proposed thin-plate based nonparametric regression method in sinogram restoration of multi-slice helical ultra-low-dose CT. In food quality inspection area, automated discrimination between walnut shell and meat has become an imperative task in the walnut postharvest processing industry in the U.S. This research developed two hyperspectral fluorescence imaging based approaches, which were capable of differentiating walnut small shell fragments from meat. Firstly, a principal component analysis (PCA) and Gaussian mixture model (PCA-GMM)-based Bayesian classification method was introduced. PCA was used to extract features, and then the optimal number of components in PCA was selected by a cross-validation technique. The PCA-GMM-based Bayesian classifier was further applied to differentiate the walnut shell and meat according to the class-conditional probability and the prior estimated by the Gaussian mixture model. The experimental results showed the effectiveness of this PCA-GMM approach, and an overall 98.2% recognition rate was achieved. Secondly, Gaussian-kernel based Support Vector Machine (SVM) was presented for the walnut shell and meat discrimination in the hyperspectral florescence imagery. SVM was applied to seek an optimal low to high dimensional mapping such that the nonlinear separable input data in the original input data space became separable on the mapped high dimensional space, and hence fulfilled the classification between walnut shell and meat. An overall recognition rate of 98.7% was achieved by this method. Although the hyperspectral fluorescence imaging is capable of differentiating between walnut shell and meat, one persistent problem is how to deal with huge amount of data acquired by the hyperspectral imaging system, and hence improve the efficiency of application system. To solve this problem, an Independent Component Analysis with k-Nearest Neighbor Classifier (ICA-kNN) approach was presented in this research to reduce the data redundancy while not sacrifice the classification performance too much. An overall 90.6% detection rate was achieved given 10 optimal wavelengths, which constituted only 13% of the total acquired hyperspectral image data. In order to further evaluate the proposed method, the classification results of the ICA-kNN approach were also compared to the kNN classifier method alone. The experimental results showed that the ICA-kNN method with fewer wavelengths had the same performance as the kNN classifier alone using information from all 79 wavelengths. This demonstrated the effectiveness of the proposed ICA-kNN method for the hyperspectral band selection in the walnut shell and meat classification

A Non-Rigid Registration Method for Analyzing Myocardial Wall Motion for Cardiac CT Images

Cardiac resynchronization therapy (CRT) has a high percentage of non-responders. Successfully locating the optimal location for CRT lead placement on a priori images can increase efficiency in procedural preparation and execution and could potentially increase the rate of CRT responders. Registration has been used in the past to assess the motion of medical images. Specifically, one method of non-rigid registration has been utilized to assess the motion of left ventricular MR cardiac images. As CT imaging is often performed as part of resynchronization treatment planning and is a fast and accessible means of imaging, extending this registration method to assessing left ventricular motion of CT images could provide another means of reproducible contractility assessment. This thesis investigates the use of non-rigid registration to evaluate the myocardium motion in multi-phase multi-slice computed tomography (MSCT) cardiac imaging for the evaluation of mechanical contraction of the left ventricle

Stereo vision-based tracking of soft tissue motion with application to online ablation control in laser microsurgery

Recent research has revealed that image-based methods can enhance accuracy and safety in laser microsurgery. In this study, non-rigid tracking using surgical stereo imaging and its application to laser ablation is discussed. A recently developed motion estimation framework based on piecewise affine deformation modeling is extended by a mesh refinement step and considering texture information. This compensates for tracking inaccuracies potentially caused by inconsistent feature matches or drift. To facilitate online application of the method, computational load is reduced by concurrent processing and affine-invariant fusion of tracking and refinement results. The residual latency-dependent tracking error is further minimized by Kalman filter-based upsampling, considering a motion model in disparity space. Accuracy is assessed in laparoscopic, beating heart, and laryngeal sequences with challenging conditions, such as partial occlusions and significant deformation. Performance is compared with that of state-of-the-art methods. In addition, the online capability of the method is evaluated by tracking two motion patterns performed by a high-precision parallel-kinematic platform. Related experiments are discussed for tissue substitute and porcine soft tissue in order to compare performances in an ideal scenario and in a setup mimicking clinical conditions. Regarding the soft tissue trial, the tracking error can be significantly reduced from 0.72 mm to below 0.05 mm with mesh refinement. To demonstrate online laser path adaptation during ablation, the non-rigid tracking framework is integrated into a setup consisting of a surgical Er:YAG laser, a three-axis scanning unit, and a low-noise stereo camera. Regardless of the error source, such as laser-to-camera registration, camera calibration, image-based tracking, and scanning latency, the ablation root mean square error is kept below 0.21 mm when the sample moves according to the aforementioned patterns. Final experiments regarding motion-compensated laser ablation of structurally deforming tissue highlight the potential of the method for vision-guided laser surgery.EU/FP/-ICT/28866

Analysis of cardiac motion using MRI and nonrigid image registration

Imperial Users onl