Segmentation, tracking, and kinematics of lung parenchyma and lung tumors from 4D CT with application to radiation treatment planning.

This thesis is concerned with development of techniques for efficient computerized analysis of 4-D CT data. The goal is to have a highly automated approach to segmentation of the lung boundary and lung nodules inside the lung. The determination of exact lung tumor location over space and time by image segmentation is an essential step to track thoracic malignancies. Accurate image segmentation helps clinical experts examine the anatomy and structure and determine the disease progress. Since 4-D CT provides structural and anatomical information during tidal breathing, we use the same data to also measure mechanical properties related to deformation of the lung tissue including Jacobian and strain at high resolutions and as a function of time. Radiation Treatment of patients with lung cancer can benefit from knowledge of these measures of regional ventilation. Graph-cuts techniques have been popular for image segmentation since they are able to treat highly textured data via robust global optimization, avoiding local minima in graph based optimization. The graph-cuts methods have been used to extract globally optimal boundaries from images by s/t cut, with energy function based on model-specific visual cues, and useful topological constraints. The method makes N-dimensional globally optimal segmentation possible with good computational efficiency. Even though the graph-cuts method can extract objects where there is a clear intensity difference, segmentation of organs or tumors pose a challenge. For organ segmentation, many segmentation methods using a shape prior have been proposed. However, in the case of lung tumors, the shape varies from patient to patient, and with location. In this thesis, we use a shape prior for tumors through a training step and PCA analysis based on the Active Shape Model (ASM). The method has been tested on real patient data from the Brown Cancer Center at the University of Louisville. We performed temporal B-spline deformable registration of the 4-D CT data - this yielded 3-D deformation fields between successive respiratory phases from which measures of regional lung function were determined. During the respiratory cycle, the lung volume changes and five different lobes of the lung (two in the left and three in the right lung) show different deformation yielding different strain and Jacobian maps. In this thesis, we determine the regional lung mechanics in the Lagrangian frame of reference through different respiratory phases, for example, Phase10 to 20, Phase10 to 30, Phase10 to 40, and Phase10 to 50. Single photon emission computed tomography (SPECT) lung imaging using radioactive tracers with SPECT ventilation and SPECT perfusion imaging also provides functional information. As part of an IRB-approved study therefore, we registered the max-inhale CT volume to both VSPECT and QSPECT data sets using the Demon\u27s non-rigid registration algorithm in patient subjects. Subsequently, statistical correlation between CT ventilation images (Jacobian and strain values), with both VSPECT and QSPECT was undertaken. Through statistical analysis with the Spearman\u27s rank correlation coefficient, we found that Jacobian values have the highest correlation with both VSPECT and QSPECT

Proceedings Virtual Imaging Trials in Medicine 2024

This submission comprises the proceedings of the 1st Virtual Imaging Trials in Medicine conference, organized by Duke University on April 22-24, 2024. The listed authors serve as the program directors for this conference. The VITM conference is a pioneering summit uniting experts from academia, industry and government in the fields of medical imaging and therapy to explore the transformative potential of in silico virtual trials and digital twins in revolutionizing healthcare. The proceedings are categorized by the respective days of the conference: Monday presentations, Tuesday presentations, Wednesday presentations, followed by the abstracts for the posters presented on Monday and Tuesday

Deep Learning in Cardiac Magnetic Resonance Image Analysis and Cardiovascular Disease Diagnosis

Cardiovascular diseases (CVDs) are the leading cause of death in the world, accounting for 17.9 million deaths each year, 31\% of all global deaths. According to the World Health Organisation (WHO), this number is expected to rise to 23 million by 2030. As a noninvasive technique, medical imaging with corresponding computer vision techniques is becoming more and more popular for detecting, understanding, and analysing CVDs. With the advent of deep learning, there are significant improvements in medical image analysis tasks (e.g. image registration, image segmentation, mesh reconstruction from image), achieving much faster and more accurate registration, segmentation, reconstruction, and disease diagnosis. This thesis focuses on cardiac magnetic resonance images, systematically studying critical tasks in CVD analysis, including image registration, image segmentation, cardiac mesh reconstruction, and CVD prediction/diagnosis. We first present a thorough review of deep learning-based image registration approaches, and subsequently, propose a novel solution to the problem of discontinuity-preserving intra-subject cardiac image registration, which is generally ignored in previous deep learning-based registration methods. On the basis of this, a joint segmentation and registration framework is further proposed to learn the joint relationship between these two tasks, leading to better registration and segmentation performance. In order to characterise the shape and motion of the heart in 3D, we present a deep learning-based 3D mesh reconstruction network that is able to recover accurate 3D cardiac shapes from 2D slice-wise segmentation masks/contours in a fast and robust manner. Finally, for CVD prediction/diagnosis, we design a multichannel variational autoencoder to learn the joint latent representation of the original cardiac image and mesh, resulting in a shape-aware image representation (SAIR) that serves as an explainable biomarker. SAIR has been shown to outperform traditional biomarkers in the prediction of acute myocardial infarction and the diagnosis of several other CVDs, and can supplement existing biomarkers to improve overall predictive performance

General Dynamic Surface Reconstruction: Application to the 3D Segmentation of the Left Ventricle

Aquesta tesi descriu la nostra contribució a la reconstrucció tridimensional de les superfícies interna i externa del ventricle esquerre humà. La reconstrucció és un primer procés dins d'una aplicació global de Realitat Virtual dissenyada com una important eina de diagnòstic per a hospitals. L'aplicació parteix de la reconstrucció de les superfícies i proveeix a l'expert de manipulació interactiva del model en temps real, a més de càlculs de volums i de altres paràmetres d'interès. El procés de recuperació de les superfícies es caracteritza per la seva velocitat de convergència, la suavitat a les malles finals i la precisió respecte de les dades recuperades. Donat que el diagnòstic de patologies cardíaques requereix d'experiència, temps i molt coneixement professional, la simulació és un procés clau que millora la eficiència.Els nostres algorismes i implementacions han estat aplicats a dades sintètiques i reals amb diferències relatives a la quantitat de dades inexistents, casuístiques presents a casos patològics i anormals. Els conjunts de dades inclouen adquisicions d'instants concrets i de cicles cardíacs complets. La bondat del sistema de reconstrucció ha estat avaluada mitjançant paràmetres mèdics per a poder comparar els nostres resultats finals amb aquells derivats a partir de programari típic utilitzat pels professionals de la medicina.A més de l'aplicació directa al diagnòstic mèdic, la nostra metodologia permet reconstruccions de tipus genèric en el camp dels Gràfics 3D per ordinador. Les nostres reconstruccions permeten generar models tridimensionals amb un baix cost en quant a la interacció manual necessària i a la càrrega computacional associada. Altrament, el nostre mètode pot entendre's com un robust algorisme de triangularització que construeix superfícies partint de núvols de punts que poden obtenir-se d'escàners làser o sensors magnètics, per exemple.Esta tesis describe nuestra contribución a la reconstrucción tridimensional de las superficies interna y externa del ventrículo izquierdo humano. La reconstrucción es un primer proceso que forma parte de una aplicación global de Realidad Virtual diseñada como una importante herramienta de diagnóstico para hospitales. La aplicación parte de la reconstrucción de las superficies y provee al experto de manipulación interactiva del modelo en tiempo real, además de cálculos de volúmenes y de otros parámetros de interés. El proceso de recuperación de las superficies se caracteriza por su velocidad de convergencia, la suavidad en las mallas finales y la precisión respecto de los datos recuperados. Dado que el diagnóstico de patologías cardíacas requiere experiencia, tiempo y mucho conocimiento profesional, la simulación es un proceso clave que mejora la eficiencia.Nuestros algoritmos e implementaciones han sido aplicados a datos sintéticos y reales con diferencias en cuanto a la cantidad de datos inexistentes, casuística presente en casos patológicos y anormales. Los conjuntos de datos incluyen adquisiciones de instantes concretos y de ciclos cardíacos completos. La bondad del sistema de reconstrucción ha sido evaluada mediante parámetros médicos para poder comparar nuestros resultados finales con aquellos derivados a partir de programario típico utilizado por los profesionales de la medicina.Además de la aplicación directa al diagnóstico médico, nuestra metodología permite reconstrucciones de tipo genérico en el campo de los Gráficos 3D por ordenador. Nuestras reconstrucciones permiten generar modelos tridimensionales con un bajo coste en cuanto a la interacción manual necesaria y a la carga computacional asociada. Por otra parte, nuestro método puede entenderse como un robusto algoritmo de triangularización que construye superficies a partir de nubes de puntos que pueden obtenerse a partir de escáneres láser o sensores magnéticos, por ejemplo.This thesis describes a contribution to the three-dimensional reconstruction of the internal and external surfaces of the human's left ventricle. The reconstruction is a first process fitting in a complete VR application that will serve as an important diagnosis tool for hospitals. Beginning with the surfaces reconstruction, the application will provide volume and interactive real-time manipulation to the model. We focus on speed, precision and smoothness for the final surfaces. As long as heart diseases diagnosis requires experience, time and professional knowledge, simulation is a key-process that enlarges efficiency.The algorithms and implementations have been applied to both synthetic and real datasets with differences regarding missing data, present in cases where pathologies and abnormalities arise. The datasets include single acquisitions and complete cardiac cycles. The goodness of the reconstructions has been evaluated with medical parameters in order to compare our results with those retrieved by typical software used by physicians.Besides the direct application to medicine diagnosis, our methodology is suitable for generic reconstructions in the field of computer graphics. Our reconstructions can serve for getting 3D models at low cost, in terms of manual interaction and CPU computation overhead. Furthermore, our method is a robust tessellation algorithm that builds surfaces from clouds of points that can be retrieved from laser scanners or magnetic sensors, among other available hardware

Coupled Shape Models for the Diagnosis of Organ Motion Restriction

Annähernd 30% der weltweiten Todesfälle sind auf Erkrankungen des Herzens und der Lunge zurückzuführen, wobei die meisten dieser Erkrankungen während ihres Verlaufs die Mobilität des betroffenen Organs verändern. Viele dieser To-desfälle könnten durch eine frühzeitige Erkennung und Behandlung der Erkran-kung vermieden werden. Deshalb wurden im Zuge dieser Arbeit Methoden ent-wickelt, um aus Segmentierungen von dynamischen Magnetresonanztomogra-phie-Daten quantitative Kennzahlen für die funktionale Analyse der Herz- und Lungenbewegung zu generieren. Ein automatisiertes Segmentierungsverfahren basierend auf gekoppelten Formmodellen wurde entwickelt, welches wechsel-seitige Informationen der Form und Geometrie mehrerer korrelierter Objekte mit einbezieht, und somit 40% bessere Ergebnisse im Vergleich zur Verwendung einzelner Modelle erzielte. Im Fall des Herzens wurde ein Volumenberechnungs-fehler von unter 13% erreicht, was in der Größenordnung der Interobserver-Variabilität liegt. Für die Lunge konnte ein Volumenfehler von unter 70ml gezeigt werden. Aus den Segmentierungsergebnissen wurden funktionale Parameter der lokalen Organdynamik abgeleitet und visualisiert, die gegen konventionelle Diag-nosemethoden evaluiert wurden und dabei gute Übereinstimmung zeigen, dar-über hinaus jedoch eine lokal und regionale Mobilitätscharakterisierung erlau-ben

Statistical anatomical modelling for efficient and personalised spine biomechanical models

Personalised medicine is redefining the present and future of healthcare by increasing treatment efficacy and predicting diseases before they actually manifest. This innovative approach takes into consideration patient’s unique genes, environment, and lifestyle. An essential component is physics-based simulations, which allows the outcome of a treatment or a disease to be replicated and visualised using a computer. The main requirement to perform this type of simulation is to build patient-specific models. These models require the extraction of realistic object geometries from images, as well as the detection of diseases or deformities to improve the estimation of the material properties of the studied object. The aim of this thesis was the design of a general framework for creating patient- specific models for biomechanical simulations using a framework based on statistical shape models. The proposed methodology was tested on the construction of spine models, including vertebrae and intervertebral discs (IVD). The proposed framework is divided into three well-defined components: The paramount and first step is the extraction of the organ or anatomical structure from medical images. In the case of the spine, IVDs and vertebrae were extracted from Magnetic Resonance images (MRI) and Computed Tomography (CT), respectively. The second step is the classification of objects according to different factors, for instance, bones by its type and grade of fracture or IVDs by its degree of degeneration. This process is essential to properly model material properties, which depends on the possible pathologies of the tissue. The last component of the framework is the creation of the patient-specific model itself by combining the information from previous steps. The behaviour of the developed algorithms was tested using different datasets of spine images from both computed tomography (CT) and Magnetic resonance (MR) images from different institutions, type of population and image resolution

Machine learning approaches for lung cancer diagnosis.

The enormity of changes and development in the field of medical imaging technology is hard to fathom, as it does not just represent the technique and process of constructing visual representations of the body from inside for medical analysis and to reveal the internal structure of different organs under the skin, but also it provides a noninvasive way for diagnosis of various disease and suggest an efficient ways to treat them. While data surrounding all of our lives are stored and collected to be ready for analysis by data scientists, medical images are considered a rich source that could provide us with a huge amount of data, that could not be read easily by physicians and radiologists, with valuable information that could be used in smart ways to discover new knowledge from these vast quantities of data. Therefore, the design of computer-aided diagnostic (CAD) system, that can be approved for use in clinical practice that aid radiologists in diagnosis and detecting potential abnormalities, is of a great importance. This dissertation deals with the development of a CAD system for lung cancer diagnosis, which is the second most common cancer in men after prostate cancer and in women after breast cancer. Moreover, lung cancer is considered the leading cause of cancer death among both genders in USA. Recently, the number of lung cancer patients has increased dramatically worldwide and its early detection doubles a patient’s chance of survival. Histological examination through biopsies is considered the gold standard for final diagnosis of pulmonary nodules. Even though resection of pulmonary nodules is the ideal and most reliable way for diagnosis, there is still a lot of different methods often used just to eliminate the risks associated with the surgical procedure. Lung nodules are approximately spherical regions of primarily high density tissue that are visible in computed tomography (CT) images of the lung. A pulmonary nodule is the first indication to start diagnosing lung cancer. Lung nodules can be benign (normal subjects) or malignant (cancerous subjects). Large (generally defined as greater than 2 cm in diameter) malignant nodules can be easily detected with traditional CT scanning techniques. However, the diagnostic options for small indeterminate nodules are limited due to problems associated with accessing small tumors. Therefore, additional diagnostic and imaging techniques which depends on the nodules’ shape and appearance are needed. The ultimate goal of this dissertation is to develop a fast noninvasive diagnostic system that can enhance the accuracy measures of early lung cancer diagnosis based on the well-known hypotheses that malignant nodules have different shape and appearance than benign nodules, because of the high growth rate of the malignant nodules. The proposed methodologies introduces new shape and appearance features which can distinguish between benign and malignant nodules. To achieve this goal a CAD system is implemented and validated using different datasets. This CAD system uses two different types of features integrated together to be able to give a full description to the pulmonary nodule. These two types are appearance features and shape features. For the appearance features different texture appearance descriptors are developed, namely the 3D histogram of oriented gradient, 3D spherical sector isosurface histogram of oriented gradient, 3D adjusted local binary pattern, 3D resolved ambiguity local binary pattern, multi-view analytical local binary pattern, and Markov Gibbs random field. Each one of these descriptors gives a good description for the nodule texture and the level of its signal homogeneity which is a distinguishable feature between benign and malignant nodules. For the shape features multi-view peripheral sum curvature scale space, spherical harmonics expansions, and different group of fundamental geometric features are utilized to describe the nodule shape complexity. Finally, the fusion of different combinations of these features, which is based on two stages is introduced. The first stage generates a primary estimation for every descriptor. Followed by the second stage that consists of an autoencoder with a single layer augmented with a softmax classifier to provide us with the ultimate classification of the nodule. These different combinations of descriptors are combined into different frameworks that are evaluated using different datasets. The first dataset is the Lung Image Database Consortium which is a benchmark publicly available dataset for lung nodule detection and diagnosis. The second dataset is our local acquired computed tomography imaging data that has been collected from the University of Louisville hospital and the research protocol was approved by the Institutional Review Board at the University of Louisville (IRB number 10.0642). These frameworks accuracy was about 94%, which make the proposed frameworks demonstrate promise to be valuable tool for the detection of lung cancer

Dosimétrie clinique en radiothérapie moléculaire

La radiothérapie moléculaire (RTM) est une radiothérapie systémique, où le produit radiopharmaceutique se lie spécifiquement sur les tumeurs pour détruire sélectivement les cibles cancéreuses tout en préservant les organes sains. Lutathera® (177Lu-DOTATATE) est un radiopharmaceutique récemment approuvé par la FDA/EMA pour le traitement des tumeurs neuroendocrines gastro-entéro-pancréatiques (GEP-NETs). Dans la pratique clinique, les patients reçoivent une activité fixe de Lutathera®, 4 cycles de 7,4 GBq, en supposant que la pharmacocinétique du radiopharmaceutique est même entre les patients. La dosimétrie spécifique au patient permet un changement de paradigme majeur dans l'administration de la RTM, passant d'une approche "taille unique" à une véritable médecine personnalisée où l'activité administrée est évaluée spécifiquement sur la base de l'irradiation délivrée à chaque patient. Pour ce faire, il faut généralement déterminer la distribution spatiale du radiopharmaceutique dans les organes par imagerie à différents moments (imagerie quantitative), estimer le nombre total de désintégrations radioactives en intégrant l'activité dans le temps (évaluation pharmacocinétique) et calculer la dose absorbée à partir des caractéristiques physiques du radionucléide et du transport de l'énergie dans les tissus du patient. Actuellement, il n'existe pas de procédures normalisées pour effectuer la dosimétrie clinique. En outre, l'évaluation des incertitudes associées à la procédure de dosimétrie n'est pas triviale. Le projet DosiTest a été lancé pour évaluer les incertitudes associées à chacune des étapes du flux de travail de la dosimétrie clinique, via une inter-comparaison multicentrique basée sur la modélisation de Monte Carlo (MC). La première phase de la thèse a consisté à comparer les analyses dosimétriques effectuées par différents centres utilisant le même logiciel et le même protocole sur le même ensemble de données de patients dans le cadre du projet IAEA-CRP E23005 afin d'évaluer la précision de la dosimétrie clinique. À notre connaissance, c'est la première fois qu'une comparaison dosimétrique multicentrique d'un seul ensemble de données cliniques sur un patient a été entreprise en utilisant le même protocole et le même logiciel par de nombreux centres dans le monde entier. Elle a mis en évidence le besoin crucial d'établir des points de contrôle et d'effectuer des vérifications de bon sens pour éliminer les disparités significatives entre les résultats et distinguer les pratiques erronées de la variabilité inter-opérateurs acceptable. Un résultat important de ce travail a été le manque d'assurance qualité en dosimétrie de médecine nucléaire clinique et la nécessité de développer des procédures de contrôle qualité. Alors que la dosimétrie gagne en popularité en médecine nucléaire, les meilleures pratiques doivent être adoptées pour garantir la fiabilité, la traçabilité et la reproductibilité des résultats. Cela met également en avant la nécessité de dispenser une formation suffisante après l'acquisition des progiciels relativement nouveaux, au-delà de quelques jours. Ceci est clairement insuffisant dans le contexte d'un domaine émergent où l'expérience professionnelle fait souvent défaut. Ensuite, l'étude de l'exactitude de la dosimétrie clinique nécessite de générer des ensembles de données de test, afin de définir la vérité de base par rapport à laquelle les procédures de dosimétrie clinique peuvent être comparées. La deuxième section de la thèse traite de la simulation de l'imagerie TEMP scintigraphique tridimensionnelle en implémentant le mouvement du détecteur d'auto-contournement dans la boîte à outils Monte Carlo GATE. Après la validation des projections TEMP/TDM sur des modèles anthropomorphes, une série d'images réalistes de patients cliniques a été générée. La dernière partie de la thèse a établi la preuve de concept du projet DosiTest, en utilisant un ensemble de données TEMP/TDM virtuelles (simulées) à différents moments, avec différentes gamma-caméras, permettant de comparer différentes techniques dosimétriques et d'évaluer la faisabilité clinique du projet dans certains départements de médecine nucléaire.Molecular radiotherapy (MRT) is a systemic radiotherapy where the radiopharmaceutical binds specifically to tumours to selectively destroy cancer targets while sparing healthy organs. Lutathera® (177Lu-DOTATATE) is a radiopharmaceutical that was recently FDA/EMA approved for the treatment of the GastroEnteroPancreatic NeuroEndocrine Tumours (GEP-NETs). In clinical practice, patients are administered with a fixed activity of Lutathera®, assuming that radiopharmaceutical distribution is the same for all patients. Patient-specific dosimetry allows for a major paradigm shift in the administration of MRT from "one-size-fits-all" approach, to real personalised medicine where administered activity is assessed specifically on the base of the irradiation delivered to each patient. This usually requires determining the spatial distribution of the radiopharmaceutical in various organs via imaging at different times (quantitative imaging), estimating the total number of radioactive decays by integrating activity over time (pharmacokinetic assessment) and calculating the absorbed dose using the physical characteristics of the radionuclide and implementing radiation transport in patient's tissues. Currently, there are no standardised procedures to perform clinical dosimetry. In addition, the assessment of the uncertainties associated with the dosimetry procedure is not trivial. The DosiTest project (http://www.dositest.org/) was initiated to evaluate uncertainties associated with each of the steps of the clinical dosimetry workflow, via a multicentric inter-comparison based on Monte Carlo (MC) modelling. The first phase of the thesis compared dosimetry analysis performed by various centres using the same software and protocol on the same patient dataset as a part of IAEA-CRP E23005 project in order to appraise the precision of clinical dosimetry. To our knowledge, this is the first time that a multi-centric dosimetry comparison of a single clinical patient dataset has been undertaken using the same protocol and software by many centres worldwide. It highlighted the critical need to establish checkpoints and conduct sanity checks to eliminate significant disparities among results, and distinguish erroneous practice with acceptable inter-operator variability. A significant outcome of this work was the lack of quality assurance in clinical nuclear medicine dosimetry and the need for the development of quality control procedures. While dosimetry is gaining popularity in nuclear medicine, best practices should be adopted to ensure that results are reliable, traceable, and reproducible. It also brings forward the need to deliver sufficient training after the acquisition of the relatively new software packages beyond a couple of days. This is clearly insufficient in a context of an emerging field where the professional experience is quite often lacking. Next, the study of clinical dosimetry accuracy requires generating test datasets, to define the ground truth against which clinical dosimetry procedures can be benchmarked. The second section of the thesis addressed the simulation of three-dimensional scintigraphic SPECT imaging by implementing auto-contouring detector motion in the GATE Monte Carlo toolkit. Following the validation of SPECT/CT projections on anthropomorphic models, a series of realistic clinical patient images were generated. The last part of the thesis established the proof of concept of the DosiTest project, using a virtual (simulated) SPECT/CT dataset at various time points, with various gamma cameras, enabling comparison of various dosimetric techniques and to assess the clinical feasibility of the project in selected nuclear medicine departments