Model-based registration for pneumothorax deformation analysis using intraoperative cone-beam CT images

[2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 20-24 July 2020, Montreal, QC, Canada]Because the lung deforms during surgery because of pneumothorax, it is important to be able to track the location of a tumor. Deformation of the whole lung can be estimated using intraoperative cone-beam CT (CBCT) images. In this study, we used deformable mesh registration methods for paired CBCT images in the inflated and deflated states, and analyzed their deformation. We proposed a deformable mesh registration framework for deformations of partial organ shapes involving large deformation and rotation. Experimental results showed that the proposed methods reduced errors in point-to-point correspondence. As a result of registration using surgical clips placed on the lung surface during imaging, it was confirmed that an average error of 3.9 mm occurred in eight cases. The result of analysis showed that both tissue rotation and contraction had large effects on displacement

Deformation analysis of surface and bronchial structures in intraoperative pneumothorax using deformable mesh registration

The positions of nodules can change because of intraoperative lung deflation, and the modeling of pneumothorax-associated deformation remains a challenging issue for intraoperative tumor localization. In this study, we introduce spatial and geometric analysis methods for inflated/deflated lungs and discuss heterogeneity in pneumothorax-associated lung deformation. Contrast-enhanced CT images simulating intraoperative conditions were acquired from live Beagle dogs. The images contain the overall shape of the lungs, including all lobes and internal bronchial structures, and were analyzed to provide a statistical deformation model that could be used as prior knowledge to predict pneumothorax. To address the difficulties of mapping pneumothorax CT images with topological changes and CT intensity shifts, we designed deformable mesh registration techniques for mixed data structures including the lobe surfaces and the bronchial centerlines. Three global-to-local registration steps were performed under the constraint that the deformation was spatially continuous and smooth, while matching visible bronchial tree structures as much as possible. The developed framework achieved stable registration with a Hausdorff distance of less than 1 mm and a target registration error of less than 5 mm, and visualized deformation fields that demonstrate per-lobe contractions and rotations with high variability between subjects. The deformation analysis results show that the strain of lung parenchyma was 35% higher than that of bronchi, and that deformation in the deflated lung is heterogeneous

Kernel-based modeling of pneumothorax deformation using intraoperative cone-beam CT images

Event: SPIE Medical Imaging, 2021, Online OnlyIn this study, we introduce statistical modeling methods for pneumothorax deformation using paired cone-beam computed tomography (CT) images. We designed a deformable mesh registration framework for shape changes involving non-linear deformation and rotation of the lungs. The registered meshes with local correspondences are available for both surgical guidance in thoracoscopic surgery and building statistical deformation models with inter-patient variations. In addition, a kernel-based deformation learning framework is proposed to reconstruct intraoperative dfl ated states of the lung from the preoperative CT models. This paper reports the findings of pneumothorax deformation and evaluation results of the kernel-based deformation framework

Deformable mesh registration of partial lung shapes based on learning of pneumothorax deformation

術中気胸は回転を含む大変形であり, 手術時に撮像可能なCone-beam CT (CBCT) は撮像領域が限られるため, 含気/虚脱肺間の正確な対応を得ることが難しい. 本研究では, 手術時に取得可能なCone-beam CT画像に含まれる肺の部分形状を対象に, 気胸変形の統計的性質の学習に基づいた可変形メッシュ位置合わせ方法を提案する. 10例の含気/虚脱肺のCBCT画像を対象に位置合わせ精度を確認する実験を行い, 従来手法と比較してより誤差が小さく安定な位置合わせが達成されることを確認したので報告する.Intraoperative pneumothorax is accompanied by large deformation including rotation. As intraoperative cone-beam CT (CBCT) images have a limited imaging area, so it is difficult to obtain an accurate correspondence between inflated and deflated lungs. In this study, we propose a deformable mesh registration method based on learning the statistical characteristics of pneumothorax deformation, targeting the partial shape of the lung included in paired CBCT images in the inflated and deflated states. We have evaluated registration accuracy for 10 CBCT images of inflated/deflated lungs, and confirmed that the proposed method achieves stable registration with smaller errors compared to existing methods

AUGMENTED REALITY AND INTRAOPERATIVE C-ARM CONE-BEAM COMPUTED TOMOGRAPHY FOR IMAGE-GUIDED ROBOTIC SURGERY

Minimally-invasive robotic-assisted surgery is a rapidly-growing alternative to traditionally open and laparoscopic procedures; nevertheless, challenges remain. Standard of care derives surgical strategies from preoperative volumetric data (i.e., computed tomography (CT) and magnetic resonance (MR) images) that benefit from the ability of multiple modalities to delineate different anatomical boundaries. However, preoperative images may not reflect a possibly highly deformed perioperative setup or intraoperative deformation. Additionally, in current clinical practice, the correspondence of preoperative plans to the surgical scene is conducted as a mental exercise; thus, the accuracy of this practice is highly dependent on the surgeon’s experience and therefore subject to inconsistencies. In order to address these fundamental limitations in minimally-invasive robotic surgery, this dissertation combines a high-end robotic C-arm imaging system and a modern robotic surgical platform as an integrated intraoperative image-guided system. We performed deformable registration of preoperative plans to a perioperative cone-beam computed tomography (CBCT), acquired after the patient is positioned for intervention. From the registered surgical plans, we overlaid critical information onto the primary intraoperative visual source, the robotic endoscope, by using augmented reality. Guidance afforded by this system not only uses augmented reality to fuse virtual medical information, but also provides tool localization and other dynamic intraoperative updated behavior in order to present enhanced depth feedback and information to the surgeon. These techniques in guided robotic surgery required a streamlined approach to creating intuitive and effective human-machine interferences, especially in visualization. Our software design principles create an inherently information-driven modular architecture incorporating robotics and intraoperative imaging through augmented reality. The system's performance is evaluated using phantoms and preclinical in-vivo experiments for multiple applications, including transoral robotic surgery, robot-assisted thoracic interventions, and cocheostomy for cochlear implantation. The resulting functionality, proposed architecture, and implemented methodologies can be further generalized to other C-arm-based image guidance for additional extensions in robotic surgery

DCE-MRI perfusion and permeability parameters as predictors of tumor response to CCRT in patients with locally advanced NSCLC

In this prospective study, 36 patients with stage III non-small cell lung cancers (NSCLC), who underwent dynamic contrast-enhanced MRI (DCE-MRI) before concurrent chemo-radiotherapy (CCRT) were enrolled. Pharmacokinetic analysis was carried out after non-rigid motion registration. The perfusion parameters including Blood Flow (BF), Blood Volume (BV), Mean Transit Time (MTT) and permeability parameters including endothelial transfer constant (Ktrans), reflux rate (Kep), fractional extravascular extracellular space volume (Ve), fractional plasma volume (Vp) were calculated, and their relationship with tumor regression was evaluated. The value of these parameters on predicting responders were calculated by receiver operating characteristic (ROC) curve. Multivariate logistic regression analysis was conducted to find the independent variables. Tumor regression rate is negatively correlated with V e and its standard variation V e-SD and positively correlated with K trans and Kep. Significant differences between responders and non-responders existed in Ktrans, Kep, Ve, Ve-SD, MTT, BV-SD and MTT-SD (P < 0.05). ROC indicated that Ve < 0.24 gave the largest area under curve of 0.865 to predict responders. Multivariate logistic regression analysis also showed Ve was a significant predictor. Baseline perfusion and permeability parameters calculated from DCE-MRI were seen to be a viable tool for predicting the early treatment response after CCRT of NSCLC. © 2016 The Author(s)

Surface guided radiotherapy

Modern radiotherapy aims to treat the decease while minimizing the radiation dose to the adjacent normal tissue, to minimize acute and late effects of the treatment. The foremost technological approaches have been intensity modulated radiotherapy (IMRT) and intensity modulated proton therapy (IMPT) in combination with image guided radiotherapy (IGRT). IMRT and IMPT is characterized by a more conform dose distribution, often accompanied by steep dose gradients. In turn, accurate patient localization and motion management becomes more important. Several image guidance systems are available for radiotherapy (RT), with 3-dimensional (3D) volumetric images with cone beam computed tomography (CBCT) as a gold standard. In recent years, surface imaging (SI) using an optical surface scanning system has been included in the IGRT toolbox. The SI system CatalystTM (C-rad Positioning AB, Uppsala Sweden) visualize 3D surface images of the patient topography, and direct correlate the patient localization to the initial planned position. SI offers the largest field-of-view in RT, does not contribute to radiation exposure, provides real-time feedback and sub-millimeter spatial resolution. These characteristics are suitable for both patient positioning and motion management during RT.Integration with the linac provides beam control and automatic couch shifts, which imposes rigorous attention to quality assurance (QA) of the SI systems. In order to integrate the beam control, beam latency times (beam-on and beam-off) should be characterized, which required the development PIN diode circuit as a QA tool. Of extra importance was the measurements of the beam-off latency time, since it represents the time the linac continues to irradiate after the beam hold signal was sent from the SI system. The automatic couch shift is calculated by a deformable image registration (DIR) algorithm, unique for the CatalystTM surface scanning system. Positioning accuracy is dependent on the image registration, and hence, a deformable thorax phantom was developed to investigate accuracy of the DIR with anatomical realistic deformations present as a QA tool.Compared to traditional 3-point localization for patient positioning, this thesis has shown that SI improve the positioning for both breast and prostate cancer patients. Also, the SI workflow has shown to be time efficient for positioning of prostate cancer patients. A respiratory motion management technique is deep inspiration breath hold (DIBH), where the patient is instructed to hold his/her breath during the treatment delivery. The aim using DIBH, is to create an anatomical distance between the treatment volume and surrounding organs-at-risk (OARs). Comparative treatment planning studies, within the work of this thesis, showed that DIBH can be an effective method for both left sided breast cancer and Hodgkin’s lymphoma (HL) in order to spare dose to the heart. For HL, the combination of IMPT and DIBH was found to spare dose to OARs, however, due to the spread in target localization individual deviations from this treatment technique were observed. The real-time feedback from the surface image system was used to investigate the reproducibility of the DIBH to ensure correct dose distribution during the treatment delivery. High reproducibility of the isocenter position during DIBH was observed, however, for a few breath holds larger deviations occurred which urges the need to use beam control tolerance for the isocenter. The overall conclusion is that optical imaging systems, developed within the work of this thesis, can be used as an imaging tool for accurate and faster patient setup, intrafractional motion monitoring and reduced dose to OARs during treatment in DIBH

Model-Based Iterative Reconstruction in Cone-Beam Computed Tomography: Advanced Models of Imaging Physics and Prior Information

Cone-beam computed tomography (CBCT) represents a rapidly developing imaging modality that provides three-dimensional (3D) volumetric images with sub-millimeter spatial resolution and soft-tissue visibility from a single gantry rotation. CBCT tends to have small footprint, mechanical simplicity, open geometry, and low cost compared to conventional diagnostic CT. Because of these features, CBCT has been used in a variety of specialty diagnostic applications, image-guided radiation therapy (on-board CBCT), and surgical guidance (e.g., C-arm based CBCT). However, the current generation of CBCT systems face major challenges in low-contrast, soft-tissue image quality – for example, in the detection of acute intracranial hemorrhage (ICH), which requires a fairly high level of image uniformity, spatial resolution, and contrast resolution. Moreover, conventional approaches in both diagnostic and image-guided interventions that involve a series of imaging studies fail to leverage the wealth of patient-specific anatomical information available from previous scans. Leveraging the knowledge gained from prior images holds the potential for major gains in image quality and dose reduction. Model-based iterative reconstruction (MBIR) attempts to make more efficient use of the measurement data by incorporating a forward model of physical detection processes. Moreover, MBIR allows incorporation of various forms of prior information into image reconstruction, ranging from image smoothness and sharpness to patient-specific anatomical information. By leveraging such advantages, MBIR has demonstrated improved tradeoffs between image quality and radiation dose in multi-detector CT in the past decade and has recently shown similar promise in CBCT. However, the full potential of MBIR in CBCT is yet to be realized. This dissertation explores the capabilities of MBIR in improving image quality (especially low-contrast, soft-tissue image quality) and reducing radiation dose in CBCT. The presented work encompasses new MBIR methods that: i) modify the noise model in MBIR to compensate for noise amplification from artifact correction; ii) design regularization by explicitly incorporating task-based imaging performance as the objective; iii) mitigate truncation effects in a computationally efficient manner; iv) leverage a wealth of patient-specific anatomical information from a previously acquired image; and v) prospectively estimate the optimal amount of prior image information for accurate admission of specific anatomical changes. Specific clinical challenges are investigated in the detection of acute ICH and surveillance of lung nodules. The results show that MBIR can substantially improve low-contrast, soft-tissue image quality in CBCT and enable dose reduction techniques in sequential imaging studies. The thesis demonstrates that novel MBIR methods hold strong potential to overcome conventional barriers to CBCT image quality and open new clinical applications that would benefit from high-quality 3D imaging

Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future