Realistic tool-tissue interaction models for surgical simulation and planning

Surgical simulators present a safe and potentially effective method for surgical training, and can also be used in pre- and intra-operative surgical planning. Realistic modeling of medical interventions involving tool-tissue interactions has been considered to be a key requirement in the development of high-fidelity simulators and planners. The soft-tissue constitutive laws, organ geometry and boundary conditions imposed by the connective tissues surrounding the organ, and the shape of the surgical tool interacting with the organ are some of the factors that govern the accuracy of medical intervention planning.\ud \ud This thesis is divided into three parts. First, we compare the accuracy of linear and nonlinear constitutive laws for tissue. An important consequence of nonlinear models is the Poynting effect, in which shearing of tissue results in normal force; this effect is not seen in a linear elastic model. The magnitude of the normal force for myocardial tissue is shown to be larger than the human contact force discrimination threshold. Further, in order to investigate and quantify the role of the Poynting effect on material discrimination, we perform a multidimensional scaling study. Second, we consider the effects of organ geometry and boundary constraints in needle path planning. Using medical images and tissue mechanical properties, we develop a model of the prostate and surrounding organs. We show that, for needle procedures such as biopsy or brachytherapy, organ geometry and boundary constraints have more impact on target motion than tissue material parameters. Finally, we investigate the effects surgical tool shape on the accuracy of medical intervention planning. We consider the specific case of robotic needle steering, in which asymmetry of a bevel-tip needle results in the needle naturally bending when it is inserted into soft tissue. We present an analytical and finite element (FE) model for the loads developed at the bevel tip during needle-tissue interaction. The analytical model explains trends observed in the experiments. We incorporated physical parameters (rupture toughness and nonlinear material elasticity) into the FE model that included both contact and cohesive zone models to simulate tissue cleavage. The model shows that the tip forces are sensitive to the rupture toughness. In order to model the mechanics of deflection of the needle, we use an energy-based formulation that incorporates tissue-specific parameters such as rupture toughness, nonlinear material elasticity, and interaction stiffness, and needle geometric and material properties. Simulation results follow similar trends (deflection and radius of curvature) to those observed in macroscopic experimental studies of a robot-driven needle interacting with gels

Tissue mimicking materials for imaging and therapy phantoms: a review

Tissue mimicking materials (TMMs), typically contained within phantoms, have been used for many decades in both imaging and therapeutic applications. This review investigates the specifications that are typically being used in development of the latest TMMs. The imaging modalities that have been investigated focus around CT, mammography, SPECT, PET, MRI and ultrasound. Therapeutic applications discussed within the review include radiotherapy, thermal therapy and surgical applications. A number of modalities were not reviewed including optical spectroscopy, optical imaging and planar x-rays. The emergence of image guided interventions and multimodality imaging have placed an increasing demand on the number of specifications on the latest TMMs. Material specification standards are available in some imaging areas such as ultrasound. It is recommended that this should be replicated for other imaging and therapeutic modalities. Materials used within phantoms have been reviewed for a series of imaging and therapeutic applications with the potential to become a testbed for cross-fertilization of materials across modalities. Deformation, texture, multimodality imaging and perfusion are common themes that are currently under development

Artificial General Intelligence for Radiation Oncology

The emergence of artificial general intelligence (AGI) is transforming radiation oncology. As prominent vanguards of AGI, large language models (LLMs) such as GPT-4 and PaLM 2 can process extensive texts and large vision models (LVMs) such as the Segment Anything Model (SAM) can process extensive imaging data to enhance the efficiency and precision of radiation therapy. This paper explores full-spectrum applications of AGI across radiation oncology including initial consultation, simulation, treatment planning, treatment delivery, treatment verification, and patient follow-up. The fusion of vision data with LLMs also creates powerful multimodal models that elucidate nuanced clinical patterns. Together, AGI promises to catalyze a shift towards data-driven, personalized radiation therapy. However, these models should complement human expertise and care. This paper provides an overview of how AGI can transform radiation oncology to elevate the standard of patient care in radiation oncology, with the key insight being AGI's ability to exploit multimodal clinical data at scale

Software and Hardware-based Tools for Improving Ultrasound Guided Prostate Brachytherapy

Minimally invasive procedures for prostate cancer diagnosis and treatment, including biopsy and brachytherapy, rely on medical imaging such as two-dimensional (2D) and three-dimensional (3D) transrectal ultrasound (TRUS) and magnetic resonance imaging (MRI) for critical tasks such as target definition and diagnosis, treatment guidance, and treatment planning. Use of these imaging modalities introduces challenges including time-consuming manual prostate segmentation, poor needle tip visualization, and variable MR-US cognitive fusion. The objective of this thesis was to develop, validate, and implement software- and hardware-based tools specifically designed for minimally invasive prostate cancer procedures to overcome these challenges. First, a deep learning-based automatic 3D TRUS prostate segmentation algorithm was developed and evaluated using a diverse dataset of clinical images acquired during prostate biopsy and brachytherapy procedures. The algorithm significantly outperformed state-of-the-art fully 3D CNNs trained using the same dataset while a segmentation time of 0.62 s demonstrated a significant reduction compared to manual segmentation. Next, the impact of dataset size, image quality, and image type on segmentation performance using this algorithm was examined. Using smaller training datasets, segmentation accuracy was shown to plateau with as little as 1000 training images, supporting the use of deep learning approaches even when data is scarce. The development of an image quality grading scale specific to 3D TRUS images will allow for easier comparison between algorithms trained using different datasets. Third, a power Doppler (PD) US-based needle tip localization method was developed and validated in both phantom and clinical cases, demonstrating reduced tip error and variation for obstructed needles compared to conventional US. Finally, a surface-based MRI-3D TRUS deformable image registration algorithm was developed and implemented clinically, demonstrating improved registration accuracy compared to manual rigid registration and reduced variation compared to the current clinical standard of physician cognitive fusion. These generalizable and easy-to-implement tools have the potential to improve workflow efficiency and accuracy for minimally invasive prostate procedures

A 3D US Guidance System for Permanent Breast Seed Implantation: Development and Validation

Permanent breast seed implantation (PBSI) is a promising breast radiotherapy technique that suffers from operator dependence. We propose and have developed an intraoperative 3D ultrasound (US) guidance system for PBSI. A tracking arm mounted to a 3D US scanner registers a needle template to the image. Images were validated for linear and volumetric accuracy, and image quality in a volunteer. The tracking arm was calibrated, and the 3D image registered to the scanner. Tracked and imaged needle positions were compared to assess accuracy and a patient-specific phantom procedure guided with the system. Median/mean linear and volumetric error was ±1.1% and ±4.1%, respectively, with clinically suitable volunteer scans. Mean tracking arm error was 0.43mm and 3D US target registration error ≤0.87mm. Mean needle tip/trajectory error was 2.46mm/1.55°. Modelled mean phantom procedure seed displacement was 2.50mm. To our knowledge, this is the first reported PBSI phantom procedure with intraoperative 3D image guidance

Quantitative PET-CT Perfusion Imaging of Prostate Cancer

Functional imaging of 18F-Fluorocholine PET holds promise in the detection of dominant prostatic lesions. Quantitative parameters from PET-CT Perfusion may be capable of measuring choline kinase activity, which could assist in identification of the dominant prostatic lesion for more accurate targeting of biopsies and radiation dose escalation. The objectives of this thesis are: 1) investigate the feasibility of using venous TACs in quantitative graphical analysis, and 2) develop and test a quantitative PET-CT Perfusion imaging technique that shows promise for identifying dominant prostatic lesions. Chapter 2 describes the effect of venous dispersion on distribution volume measurements with the Logan Plot. The dispersion of venous PET curves was simulated based on the arterio-venous transit time spectrum measured in a perfusion CT study of the human forearm. The analysis showed good agreement between distribution volume measurements produced by the arterial and venous TACs. Chapter 3 details the mathematical implementation of a linearized solution of the 3-Compartment kinetic model for hybrid PET-CT Perfusion imaging. A noise simulation determined the effect of incorporating CT perfusion parameters into the PET model on the accuracy and variability of measurements of the choline kinase activity. Results indicated that inclusion of CT perfusion parameters known a priori can significantly improve the accuracy and variability of imaging parameters measured with PET. Chapter 4 presents the implementation of PET-CT Perfusion imaging in a xenograft mouse model of human prostate cancer. Image-derived arterial TACs from the left ventricle were corrected for partial volume and spillover effects and validated by comparing to blood sampled curves. The PET-CT Perfusion imaging technique produced parametric maps of the choline kinase activity, k3. The results showed that the partial volume and spillover corrected arterial TACs agreed well with the blood sampled curves, and that k3max was significantly correlated with tumor volume, while SUV was not. In summary, this thesis establishes a solid foundation for future clinical research into 18F-fluorocholine PET imaging for the identification of dominant prostatic lesions. Quantitative PET-CT Perfusion imaging shows promise for assisting targeting of biopsy and radiation dose escalation of prostate cancer

Segmentierung medizinischer Bilddaten und bildgestützte intraoperative Navigation

Die Entwicklung von Algorithmen zur automatischen oder semi-automatischen Verarbeitung von medizinischen Bilddaten hat in den letzten Jahren mehr und mehr an Bedeutung gewonnen. Das liegt zum einen an den immer besser werdenden medizinischen Aufnahmemodalitäten, die den menschlichen Körper immer feiner virtuell abbilden können. Zum anderen liegt dies an der verbesserten Computerhardware, die eine algorithmische Verarbeitung der teilweise im Gigabyte-Bereich liegenden Datenmengen in einer vernünftigen Zeit erlaubt. Das Ziel dieser Habilitationsschrift ist die Entwicklung und Evaluation von Algorithmen für die medizinische Bildverarbeitung. Insgesamt besteht die Habilitationsschrift aus einer Reihe von Publikationen, die in drei übergreifende Themenbereiche gegliedert sind: -Segmentierung medizinischer Bilddaten anhand von vorlagenbasierten Algorithmen -Experimentelle Evaluation quelloffener Segmentierungsmethoden unter medizinischen Einsatzbedingungen -Navigation zur Unterstützung intraoperativer Therapien Im Bereich Segmentierung medizinischer Bilddaten anhand von vorlagenbasierten Algorithmen wurden verschiedene graphbasierte Algorithmen in 2D und 3D entwickelt, die einen gerichteten Graphen mittels einer Vorlage aufbauen. Dazu gehört die Bildung eines Algorithmus zur Segmentierung von Wirbeln in 2D und 3D. In 2D wird eine rechteckige und in 3D eine würfelförmige Vorlage genutzt, um den Graphen aufzubauen und das Segmentierungsergebnis zu berechnen. Außerdem wird eine graphbasierte Segmentierung von Prostatadrüsen durch eine Kugelvorlage zur automatischen Bestimmung der Grenzen zwischen Prostatadrüsen und umliegenden Organen vorgestellt. Auf den vorlagenbasierten Algorithmen aufbauend, wurde ein interaktiver Segmentierungsalgorithmus, der einem Benutzer in Echtzeit das Segmentierungsergebnis anzeigt, konzipiert und implementiert. Der Algorithmus nutzt zur Segmentierung die verschiedenen Vorlagen, benötigt allerdings nur einen Saatpunkt des Benutzers. In einem weiteren Ansatz kann der Benutzer die Segmentierung interaktiv durch zusätzliche Saatpunkte verfeinern. Dadurch wird es möglich, eine semi-automatische Segmentierung auch in schwierigen Fällen zu einem zufriedenstellenden Ergebnis zu führen. Im Bereich Evaluation quelloffener Segmentierungsmethoden unter medizinischen Einsatzbedingungen wurden verschiedene frei verfügbare Segmentierungsalgorithmen anhand von Patientendaten aus der klinischen Routine getestet. Dazu gehörte die Evaluierung der semi-automatischen Segmentierung von Hirntumoren, zum Beispiel Hypophysenadenomen und Glioblastomen, mit der frei verfügbaren Open Source-Plattform 3D Slicer. Dadurch konnte gezeigt werden, wie eine rein manuelle Schicht-für-Schicht-Vermessung des Tumorvolumens in der Praxis unterstützt und beschleunigt werden kann. Weiterhin wurde die Segmentierung von Sprachbahnen in medizinischen Aufnahmen von Hirntumorpatienten auf verschiedenen Plattformen evaluiert. Im Bereich Navigation zur Unterstützung intraoperativer Therapien wurden Softwaremodule zum Begleiten von intra-operativen Eingriffen in verschiedenen Phasen einer Behandlung (Therapieplanung, Durchführung, Kontrolle) entwickelt. Dazu gehört die erstmalige Integration des OpenIGTLink-Netzwerkprotokolls in die medizinische Prototyping-Plattform MeVisLab, die anhand eines NDI-Navigationssystems evaluiert wurde. Außerdem wurde hier ebenfalls zum ersten Mal die Konzeption und Implementierung eines medizinischen Software-Prototypen zur Unterstützung der intraoperativen gynäkologischen Brachytherapie vorgestellt. Der Software-Prototyp enthielt auch ein Modul zur erweiterten Visualisierung bei der MR-gestützten interstitiellen gynäkologischen Brachytherapie, welches unter anderem die Registrierung eines gynäkologischen Brachytherapie-Instruments in einen intraoperativen Datensatz einer Patientin ermöglichte. Die einzelnen Module führten zur Vorstellung eines umfassenden bildgestützten Systems für die gynäkologische Brachytherapie in einem multimodalen Operationssaal. Dieses System deckt die prä-, intra- und postoperative Behandlungsphase bei einer interstitiellen gynäkologischen Brachytherapie ab