Synergistic Visualization And Quantitative Analysis Of Volumetric Medical Images

The medical diagnosis process starts with an interview with the patient, and continues with the physical exam. In practice, the medical professional may require additional screenings to precisely diagnose. Medical imaging is one of the most frequently used non-invasive screening methods to acquire insight of human body. Medical imaging is not only essential for accurate diagnosis, but also it can enable early prevention. Medical data visualization refers to projecting the medical data into a human understandable format at mediums such as 2D or head-mounted displays without causing any interpretation which may lead to clinical intervention. In contrast to the medical visualization, quantification refers to extracting the information in the medical scan to enable the clinicians to make fast and accurate decisions. Despite the extraordinary process both in medical visualization and quantitative radiology, efforts to improve these two complementary fields are often performed independently and synergistic combination is under-studied. Existing image-based software platforms mostly fail to be used in routine clinics due to lack of a unified strategy that guides clinicians both visually and quan- titatively. Hence, there is an urgent need for a bridge connecting the medical visualization and automatic quantification algorithms in the same software platform. In this thesis, we aim to fill this research gap by visualizing medical images interactively from anywhere, and performing a fast, accurate and fully-automatic quantification of the medical imaging data. To end this, we propose several innovative and novel methods. Specifically, we solve the following sub-problems of the ul- timate goal: (1) direct web-based out-of-core volume rendering, (2) robust, accurate, and efficient learning based algorithms to segment highly pathological medical data, (3) automatic landmark- ing for aiding diagnosis and surgical planning and (4) novel artificial intelligence algorithms to determine the sufficient and necessary data to derive large-scale problems

Advanced Endoscopic Navigation:Surgical Big Data,Methodology,and Applications

随着科学技术的飞速发展，健康与环境问题日益成为人类面临的最重大问题之一。信息科学、计算机技术、电子工程与生物医学工程等学科的综合应用交叉前沿课题，研究现代工程技术方法，探索肿瘤癌症等疾病早期诊断、治疗和康复手段。本论文综述了计算机辅助微创外科手术导航、多模态医疗大数据、方法论及其临床应用：从引入微创外科手术导航概念出发，介绍了医疗大数据的术前与术中多模态医学成像方法、阐述了先进微创外科手术导航的核心流程包括计算解剖模型、术中实时导航方案、三维可视化方法及交互式软件技术，归纳了各类微创外科手术方法的临床应用。同时，重点讨论了全球各种手术导航技术在临床应用中的优缺点，分析了目前手术导航领域内的最新技术方法。在此基础上，提出了微创外科手术方法正向数字化、个性化、精准化、诊疗一体化、机器人化以及高度智能化的发展趋势。【Abstract】Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.X.L. acknowledges funding from the Fundamental Research Funds for the Central Universities. T.M.P. acknowledges funding from the Canadian Foundation for Innovation, the Canadian Institutes for Health Research, the National Sciences and Engineering Research Council of Canada, and a grant from Intuitive Surgical Inc

InterNAV3D: A Navigation Tool for Robot-Assisted Needle-Based Intervention for the Lung

Lung cancer is one of the leading causes of cancer deaths in North America. There are recent advances in cancer treatment techniques that can treat cancerous tumors, but require a real-time imaging modality to provide intraoperative assistive feedback. Ultrasound (US) imaging is one such modality. However, while its application to the lungs has been limited because of the deterioration of US image quality (due to the presence of air in the lungs); recent work has shown that appropriate lung deflation can help to improve the quality sufficiently to enable intraoperative, US-guided robotics-assisted techniques to be used. The work described in this thesis focuses on this approach. The thesis describes a project undertaken at Canadian Surgical Technologies and Advanced Robotics (CSTAR) that utilizes the image processing techniques to further enhance US images and implements an advanced 3D virtual visualization software approach. The application considered is that for minimally invasive lung cancer treatment using procedures such as brachytherapy and microwave ablation while taking advantage of the accuracy and teleoperation capabilities of surgical robots, to gain higher dexterity and precise control over the therapy tools (needles and probes). A number of modules and widgets are developed and explained which improve the visibility of the physical features of interest in the treatment and help the clinician to have more reliable and accurate control of the treatment. Finally the developed tools are validated with extensive experimental evaluations and future developments are suggested to enhance the scope of the applications

Cohort-based T-SSIM Visual Computing for Radiation Therapy Prediction and Exploration

We describe a visual computing approach to radiation therapy (RT) planning, based on spatial similarity within a patient cohort. In radiotherapy for head and neck cancer treatment, dosage to organs at risk surrounding a tumor is a large cause of treatment toxicity. Along with the availability of patient repositories, this situation has lead to clinician interest in understanding and predicting RT outcomes based on previously treated similar patients. To enable this type of analysis, we introduce a novel topology-based spatial similarity measure, T-SSIM, and a predictive algorithm based on this similarity measure. We couple the algorithm with a visual steering interface that intertwines visual encodings for the spatial data and statistical results, including a novel parallel-marker encoding that is spatially aware. We report quantitative results on a cohort of 165 patients, as well as a qualitative evaluation with domain experts in radiation oncology, data management, biostatistics, and medical imaging, who are collaborating remotely.Comment: IEEE VIS (SciVis) 201

Rapid Segmentation Techniques for Cardiac and Neuroimage Analysis

Recent technological advances in medical imaging have allowed for the quick acquisition of highly resolved data to aid in diagnosis and characterization of diseases or to guide interventions. In order to to be integrated into a clinical work flow, accurate and robust methods of analysis must be developed which manage this increase in data. Recent improvements in in- expensive commercially available graphics hardware and General-Purpose Programming on Graphics Processing Units (GPGPU) have allowed for many large scale data analysis problems to be addressed in meaningful time and will continue to as parallel computing technology improves. In this thesis we propose methods to tackle two clinically relevant image segmentation problems: a user-guided segmentation of myocardial scar from Late-Enhancement Magnetic Resonance Images (LE-MRI) and a multi-atlas segmentation pipeline to automatically segment and partition brain tissue from multi-channel MRI. Both methods are based on recent advances in computer vision, in particular max-flow optimization that aims at solving the segmentation problem in continuous space. This allows for (approximately) globally optimal solvers to be employed in multi-region segmentation problems, without the particular drawbacks of their discrete counterparts, graph cuts, which typically present with metrication artefacts. Max-flow solvers are generally able to produce robust results, but are known for being computationally expensive, especially with large datasets, such as volume images. Additionally, we propose two new deformable registration methods based on Gauss-Newton optimization and smooth the resulting deformation fields via total-variation regularization to guarantee the problem is mathematically well-posed. We compare the performance of these two methods against four highly ranked and well-known deformable registration methods on four publicly available databases and are able to demonstrate a highly accurate performance with low run times. The best performing variant is subsequently used in a multi-atlas segmentation pipeline for the segmentation of brain tissue and facilitates fast run times for this computationally expensive approach. All proposed methods are implemented using GPGPU for a substantial increase in computational performance and so facilitate deployment into clinical work flows. We evaluate all proposed algorithms in terms of run times, accuracy, repeatability and errors arising from user interactions and we demonstrate that these methods are able to outperform established methods. The presented approaches demonstrate high performance in comparison with established methods in terms of accuracy and repeatability while largely reducing run times due to the employment of GPU hardware

Registration of pre-operative lung cancer PET/CT scans with post-operative histopathology images

Non-invasive imaging modalities used in the diagnosis of lung cancer, such as Positron Emission Tomography (PET) or Computed Tomography (CT), currently provide insuffcient information about the cellular make-up of the lesion microenvironment, unless they are compared against the gold standard of histopathology.The aim of this retrospective study was to build a robust imaging framework for registering in vivo and post-operative scans from lung cancer patients, in order to have a global, pathology-validated multimodality map of the tumour and its surroundings.;Initial experiments were performed on tissue-mimicking phantoms, to test different shape reconstruction methods. The choice of interpolator and slice thickness were found to affect the algorithm's output, in terms of overall volume and local feature recovery. In the second phase of the study, nine lung cancer patients referred for radical lobectomy were recruited. Resected specimens were inflated with agar, sliced at 5 mm intervals, and each cross-section was photographed. The tumour area was delineated on the block-face pathology images and on the preoperative PET/CT scans.;Airway segments were also added to the reconstructed models, to act as anatomical fiducials. Binary shapes were pre-registered by aligning their minimal bounding box axes, and subsequently transformed using rigid registration. In addition, histopathology slides were matched to the block-face photographs using moving least squares algorithm.;A two-step validation process was used to evaluate the performance of the proposed method against manual registration carried out by experienced consultants. In two out of three cases, experts rated the results generated by the algorithm as the best output, suggesting that the developed framework outperforms the current standard practice.Non-invasive imaging modalities used in the diagnosis of lung cancer, such as Positron Emission Tomography (PET) or Computed Tomography (CT), currently provide insuffcient information about the cellular make-up of the lesion microenvironment, unless they are compared against the gold standard of histopathology.The aim of this retrospective study was to build a robust imaging framework for registering in vivo and post-operative scans from lung cancer patients, in order to have a global, pathology-validated multimodality map of the tumour and its surroundings.;Initial experiments were performed on tissue-mimicking phantoms, to test different shape reconstruction methods. The choice of interpolator and slice thickness were found to affect the algorithm's output, in terms of overall volume and local feature recovery. In the second phase of the study, nine lung cancer patients referred for radical lobectomy were recruited. Resected specimens were inflated with agar, sliced at 5 mm intervals, and each cross-section was photographed. The tumour area was delineated on the block-face pathology images and on the preoperative PET/CT scans.;Airway segments were also added to the reconstructed models, to act as anatomical fiducials. Binary shapes were pre-registered by aligning their minimal bounding box axes, and subsequently transformed using rigid registration. In addition, histopathology slides were matched to the block-face photographs using moving least squares algorithm.;A two-step validation process was used to evaluate the performance of the proposed method against manual registration carried out by experienced consultants. In two out of three cases, experts rated the results generated by the algorithm as the best output, suggesting that the developed framework outperforms the current standard practice

Improving Dose-Response Correlations for Locally Advanced NSCLC Patients Treated with IMRT or PSPT

The standard of care for locally advanced non-small cell lung cancer (NSCLC) is concurrent chemo-radiotherapy. Despite recent advancements in radiation delivery methods, the median survival time of NSCLC patients remains below 28 months. Higher tumor dose has been found to increase survival but also a higher rate of radiation pneumonitis (RP) that affects breathing capability. In fear of such toxicity, less-aggressive treatment plans are often clinically preferred, leading to metastasis and recurrence. Therefore, accurate RP prediction is crucial to ensure tumor coverage to improve treatment outcome. Current models have associated RP with increased dose but with limited accuracy as they lack spatial correlation between accurate dose representation and quantitative RP representation. These models represent lung tissue damage with radiation dose distribution planned pre-treatment, which assumes a fixed patient geometry and inevitably renders imprecise dose delivery due to intra-fractional breathing motion and inter-fractional anatomy response. Additionally, current models employ whole-lung dose metrics as the contributing factor to RP as a qualitative, binary outcome but these global dose metrics discard microscopic, voxel-(3D pixel)-level information and prevent spatial correlations with quantitative RP representation. To tackle these limitations, we developed advanced deformable image registration (DIR) techniques that registered corresponding anatomical voxels between images for tracking and accumulating dose throughout treatment. DIR also enabled voxel-level dose-response correlation when CT image density change (IDC) was used to quantify RP. We hypothesized that more accurate estimates of biologically effective dose distributions actually delivered, achieved through (a) dose accumulation using deformable registration of weekly 4DCT images acquired over the course or radiotherapy and (b) the incorporation of variable relative biological effectiveness (RBE), would lead to statistically and clinically significant improvement in the correlation of RP with biologically effective dose distributions. Our work resulted in a robust intra-4DCT and inter-4DCT DIR workflow, with the accuracy meeting AAPM TG-132 recommendations for clinical implementation of DIR. The automated DIR workflow allowed us to develop a fully automated 4DCT-based dose accumulation pipeline in RayStation (RaySearch Laboratories, Stockholm, Sweden). With a sample of 67 IMRT patients, our results showed that the accumulated dose was statistically different than the planned dose across the entire cohort with an average MLD increase of ~1 Gy and clinically different for individual patients where 16% resulted in difference in the score of the normal tissue complication probability (NTCP) using an established, clinically used model, which could qualify the patients for treatment planning re-evaluation. Lastly, we associated dose difference with accuracy difference by establishing and comparing voxel-level dose-IDC correlations and concluded that the accumulated dose better described the localized damage, thereby a closer representation of the delivered dose. Using the same dose-response correlation strategy, we plotted the dose-IDC relationships for both photon patients (N = 51) and proton patients (N = 67), we measured the variable proton RBE values to be 3.07–1.27 from 9–52 Gy proton voxels. With the measured RBE values, we fitted an established variable proton RBE model with pseudo-R2 of 0.98. Therefore, our results led to statistically and clinically significant improvement in the correlation of RP with accumulated and biologically effective dose distributions and demonstrated the potential of incorporating the effect of anatomical change and biological damage in RP prediction models

Diseases of the Chest, Breast, Heart and Vessels 2019-2022

This open access book focuses on diagnostic and interventional imaging of the chest, breast, heart, and vessels. It consists of a remarkable collection of contributions authored by internationally respected experts, featuring the most recent diagnostic developments and technological advances with a highly didactical approach. The chapters are disease-oriented and cover all the relevant imaging modalities, including standard radiography, CT, nuclear medicine with PET, ultrasound and magnetic resonance imaging, as well as imaging-guided interventions. As such, it presents a comprehensive review of current knowledge on imaging of the heart and chest, as well as thoracic interventions and a selection of "hot topics". The book is intended for radiologists, however, it is also of interest to clinicians in oncology, cardiology, and pulmonology

A multi-technique hierarchical X-ray phase-based approach for the characterization and quantification of the effects of novel radiotherapies

Cancer is the first or second leading cause of premature deaths worldwide with an overall rapidly growing burden. Standard cancer therapies include surgery, chemotherapy and radiotherapy (RT) and often a combination of the three is applied to improve the probability of tumour control. Standard therapy protocols have been established for many types of cancers and new approaches are under study especially for treating radio-resistant tumours associated to an overall poor prognosis, as for brain and lung cancers. Follow up techniques able to monitor and investigate the effects of therapies are important for surveying the efficacy of conventionally applied treatments and are key for accessing the curing capabilities and the onset of acute and late adverse effects of new therapies. In this framework, this doctoral Thesis proposes the X-ray Phase Contrast Im-aging - Computed Tomography (XPCI-CT) technique as an imaging-based tool to study and quantify the effects of novel RTs, namely Microbeam and Minibeam Radiation therapy (MRT and MB), and to compare them to the standard Broad Beam (BB) induced effects on brain and lungs. MRT and MB are novel radiotherapies that deliver an array of spatially fractionated X-ray beamlets issued from a synchrotron radiation source, with widths of tens or hundreds of micrometres, respectively. MRT and MB exploit the so-called dose-volume effect: hundreds of Grays are well tolerated by healthy tissues and show a preferential effect on tumour cells and vasculature when delivered in a micrometric sized micro-plane, while induce lethal effects if applied over larger uniform irradiation fields. Such highly collimated X-ray beams need a high-resolution and a full-organ approach that can visualize, with high sensitivity, the effects of the treatment along and outside the beamlets path. XPCI-CT is here suggested and proven as a powerful imaging technique able to determine and quantify the effects of the radiation on normal and tumour-bearing tissues. Moreover, it is shown as an effective technique to complement, with 3D information, the histology findings in the follow-up of the RT treatments. Using a multi-scale and multi-technique X-ray-based approach, I have visualized and analysed the effects of RT delivery on healthy and glioblastoma multiforme (GBM)-bearing rat brains as well as on healthy rat lungs. Ex-vivo XPCI-CT datasets acquired with isotropic voxel sizes in the range 3.253 – 0.653 μm3 could distinguish, with high sensitivity, the idiopathic effects of MRT, MB and BB therapies. Histology, immunohistochemistry, Small- and Wide-Angle X-ray Scattering and X-ray Fluorescence experiments were also carried out to accurately interpret and complement the XPCI-CT findings as well as to obtain a detailed structural and chemical characterization of the detected pathological features. Overall, this multi-technique approach could detect: i) a different radio-sensitivity for the MRT-treated brain areas; ii) Ca and Fe deposits, hydroxyapatite crystals formation; iii) extended and isolated fibrotic contents. Full-organ XPCI-CT datasets allowed for the quantification of tumour and mi-crocalcifications’ volumes in treated brains and the amount of scarring tissue in irradiated lungs. Herein, the role of XPCI-CT as a 3D virtual histology technique for the follow-up of ex-vivo RT effects has been assessed as a complementary method for an accurate volumetric investigation of normal and pathological states in brains and lungs, in a small animal model. Moreover, the technique is proposed as a guidance and auxiliary tool for conventional histology, which is the gold standard for pathological evaluations, owing to its 3D capabilities and the possibility of virtually navigating within samples. This puts a landmark for XPCI-CT inclusion in the pre-clinical studies pipeline and for advancing towards in-vivo XPCI-CT imaging of treated organs.Weltweit gilt Krebs als häufigste bzw. zweithäufigste Ursache eines zu früh erfolgenden Todes, wobei die Zahlen rasch ansteigen. Standardmäßige Krebstherapien umfassen chirurgische Eingriffe, Chemotherapie und Strahlentherapie (radiotherapy, RT); oft kommt eine Kombination daraus zur Anwendung, um die Wahrscheinlichkeit der Tumorkontrolle zu erhöhen. Es wurden Standardtherapieprotokolle für zahlreiche Krebsarten eingerichtet und es wird vor allem in der Behandlung von strahlenresistenten Tumoren mit allgemein schlechter Prognose wie bei Hirn- und Lungentumoren an neuen Ansätzen geforscht. Nachverfolgungstechniken, welche die Auswirkungen von Therapien überwachen und ermitteln, sind zur Überwachung der Wirksamkeit herkömmlich angewandter Behandlungen wichtig und auch maßgeblich am Zugang zu den Fähigkeiten zur Heilung sowie zum Auftreten akuter und verzögerter Nebenwirkungen neuer Therapien beteiligt. In diesem Rahmenwerk unterbreitet diese Doktorarbeit die Technik der Röntgen-Phasenkontrast-Bildgebung über Computertomographie (X-ray Phase Contrast Imaging - Computed Tomography, XPCI‑CT) als bildverarbeitungs-basiertes Tool zur Untersuchung und Quantifizierung der Auswirkungen neuartiger Strahlentherapien, nämlich der Mikrobeam- und Minibeam-Strahlentherapie (MRT und MB), sowie zum Vergleich derselben mit den herkömmlichen durch Breitstrahlen (Broad Beam, BB) erzielten Auswirkungen auf Gehirn und Lunge. MRT und MB sind neuartige Strahlentherapien, die ein Array räumlich aufgeteilter Röntgenstrahlenbeamlets aus einer synchrotronen Strahlenquelle mit einer Breite von Zehnteln bzw. Hundersteln Mikrometern abgeben. MRT und MB nutzen den sogenannten Dosis-Volumen-Effekt: Hunderte Gray werden von gesundem Gewebe gut vertragen und wirken bei der Abgabe in einer Mikroebene im Mikrometerbereich vorrangig auf Tumorzellen und Blutgefäße, während sie bei einer Anwendung über größere gleichförmige Strahlungsfelder letale Auswirkungen aufweisen. Solche hoch kollimierten Röntgenstrahlen erfordern eine hohe Auflösung und einen Zugang zum gesamten Organ, bei dem die Auswirkungen der Behandlung entlang und außerhalb der Beamletpfade mit hoher Empfindlichkeit visualisiert werden können. Hier empfiehlt und bewährt sich die XPCI‑CT als leistungsstarke Bildverarbeitungstechnik, welche die Auswirkungen der Strahlung auf normale und tumortragende Gewebe feststellen und quantifizieren kann. Außerdem hat sich gezeigt, dass sie durch 3‑D-Informationen eine effektive Technik zur Ergänzung der histologischen Erkenntnisse in der Nachverfolgung der Strahlenbehandlung ist. Anhand eines mehrstufigen und multitechnischen röntgenbasierten Ansatzes habe ich die Auswirkungen der Strahlentherapie auf gesunde und von Glioblastomen (GBM) befallene Rattenhirne sowie auf gesunde Rattenlungen visualisiert und analysiert. Mit isotropen Voxelgrößen im Bereich von 3,53 bis 0,653 μm3 erfasste Ex-vivo-XPCI-CT-Datensätze konnten die idiopathischen Auswirkungen der MRT-, MB- und BB‑Behandlung mit hoher Empfindlichkeit unterscheiden. Es wurden auch Experimente zu Histologie, Immunhistochemie, Röntgenklein- und ‑weitwinkelstreuung und Röntgenfluoreszenz durchgeführt, um die XPCI‑CT-Erkenntnisse präzise zu interpretieren und zu ergänzen sowie eine detaillierte strukturelle und chemische Charakterisierung der nachgewiesenen pathologischen Merkmale zu erhalten. Im Allgemeinen wurde durch diesen multitechnischen Ansatz Folgendes ermittelt: i) eine un-terschiedliche Strahlenempfindlichkeit der mit MRT behandelten Gehirnbereiche; ii) Ca- und Fe-Ablagerungen und die Bildung von Hydroxylapatitkristallen; iii) ein ausgedehnter und isolierter Fibrosegehalt. XPCI‑CT-Datensätze des gesamten Organs ermöglichten die Quantifizierung der Volume von Tumoren und Mikroverkalkungen in den behandelten Gehirnen und der Menge des Narbengewebes in bestrahlten Lungen. Dabei wurde die Rolle der XPCI‑CT als virtuelle 3‑D-Histologietechnik für die Nachverfolgung von Ex-vivo-RT‑Auswirkungen als ergänzende Methode für eine präzise volumetrische Untersuchung des normalen und pathologischen Zustands von Gehirnen und Lungen im Kleintiermodell untersucht. Darüber hinaus wird die Technik aufgrund ihrer 3‑D-Fähigkeiten und der Möglichkeit zur virtuellen Navigation in den Proben als Leitfaden und Hilfstool für die herkömmliche Histologie vorgeschlagen, die der Goldstandard für die pathologische Evaluierung ist. Dies markiert einen Meilenstein für die Übernahme der XPCI‑CT in die Pipeline präklinischer Studien und für den Übergang zur In-vivo-XPCI‑CT von behandelten Organen