Machine learning-based automated segmentation with a feedback loop for 3D synchrotron micro-CT

Die Entwicklung von Synchrotronlichtquellen der dritten Generation hat die Grundlage für die Untersuchung der 3D-Struktur opaker Proben mit einer Auflösung im Mikrometerbereich und höher geschaffen. Dies führte zur Entwicklung der Röntgen-Synchrotron-Mikro-Computertomographie, welche die Schaffung von Bildgebungseinrichtungen zur Untersuchung von Proben verschiedenster Art förderte, z.B. von Modellorganismen, um die Physiologie komplexer lebender Systeme besser zu verstehen. Die Entwicklung moderner Steuerungssysteme und Robotik ermöglichte die vollständige Automatisierung der Röntgenbildgebungsexperimente und die Kalibrierung der Parameter des Versuchsaufbaus während des Betriebs. Die Weiterentwicklung der digitalen Detektorsysteme führte zu Verbesserungen der Auflösung, des Dynamikbereichs, der Empfindlichkeit und anderer wesentlicher Eigenschaften. Diese Verbesserungen führten zu einer beträchtlichen Steigerung des Durchsatzes des Bildgebungsprozesses, aber auf der anderen Seite begannen die Experimente eine wesentlich größere Datenmenge von bis zu Dutzenden von Terabyte zu generieren, welche anschließend manuell verarbeitet wurden. Somit ebneten diese technischen Fortschritte den Weg für die Durchführung effizienterer Hochdurchsatzexperimente zur Untersuchung einer großen Anzahl von Proben, welche Datensätze von besserer Qualität produzierten. In der wissenschaftlichen Gemeinschaft besteht daher ein hoher Bedarf an einem effizienten, automatisierten Workflow für die Röntgendatenanalyse, welcher eine solche Datenlast bewältigen und wertvolle Erkenntnisse für die Fachexperten liefern kann. Die bestehenden Lösungen für einen solchen Workflow sind nicht direkt auf Hochdurchsatzexperimente anwendbar, da sie für Ad-hoc-Szenarien im Bereich der medizinischen Bildgebung entwickelt wurden. Daher sind sie nicht für Hochdurchsatzdatenströme optimiert und auch nicht in der Lage, die hierarchische Beschaffenheit von Proben zu nutzen. Die wichtigsten Beiträge der vorliegenden Arbeit sind ein neuer automatisierter Analyse-Workflow, der für die effiziente Verarbeitung heterogener Röntgendatensätze hierarchischer Natur geeignet ist. Der entwickelte Workflow basiert auf verbesserten Methoden zur Datenvorverarbeitung, Registrierung, Lokalisierung und Segmentierung. Jede Phase eines Arbeitsablaufs, die eine Trainingsphase beinhaltet, kann automatisch feinabgestimmt werden, um die besten Hyperparameter für den spezifischen Datensatz zu finden. Für die Analyse von Faserstrukturen in Proben wurde eine neue, hochgradig parallelisierbare 3D-Orientierungsanalysemethode entwickelt, die auf einem neuartigen Konzept der emittierenden Strahlen basiert und eine präzisere morphologische Analyse ermöglicht. Alle entwickelten Methoden wurden gründlich an synthetischen Datensätzen validiert, um ihre Anwendbarkeit unter verschiedenen Abbildungsbedingungen quantitativ zu bewerten. Es wurde gezeigt, dass der Workflow in der Lage ist, eine Reihe von Datensätzen ähnlicher Art zu verarbeiten. Darüber hinaus werden die effizienten CPU/GPU-Implementierungen des entwickelten Workflows und der Methoden vorgestellt und der Gemeinschaft als Module für die Sprache Python zur Verfügung gestellt. Der entwickelte automatisierte Analyse-Workflow wurde erfolgreich für Mikro-CT-Datensätze angewandt, die in Hochdurchsatzröntgenexperimenten im Bereich der Entwicklungsbiologie und Materialwissenschaft gewonnen wurden. Insbesondere wurde dieser Arbeitsablauf für die Analyse der Medaka-Fisch-Datensätze angewandt, was eine automatisierte Segmentierung und anschließende morphologische Analyse von Gehirn, Leber, Kopfnephronen und Herz ermöglichte. Darüber hinaus wurde die entwickelte Methode der 3D-Orientierungsanalyse bei der morphologischen Analyse von Polymergerüst-Datensätzen eingesetzt, um einen Herstellungsprozess in Richtung wünschenswerter Eigenschaften zu lenken

Incorporating Cardiac Substructures Into Radiation Therapy For Improved Cardiac Sparing

Growing evidence suggests that radiation therapy (RT) doses to the heart and cardiac substructures (CS) are strongly linked to cardiac toxicities, though only the heart is considered clinically. This work aimed to utilize the superior soft-tissue contrast of magnetic resonance (MR) to segment CS, quantify uncertainties in their position, assess their effect on treatment planning and an MR-guided environment. Automatic substructure segmentation of 12 CS was completed using a novel hybrid MR/computed tomography (CT) atlas method and was improved upon using a 3-dimensional neural network (U-Net) from deep learning. Intra-fraction motion due to respiration was then quantified. The inter-fraction setup uncertainties utilizing a novel MR-linear accelerator were also quantified. Treatment planning comparisons were performed with and without substructure inclusions and methods to reduce radiation dose to sensitive CS were evaluated. Lastly, these described technologies (deep learning U-Net) were translated to an MR-linear accelerator and a segmentation pipeline was created. Automatic segmentations from the hybrid MR/CT atlas was able to generate accurate segmentations for the chambers and great vessels (Dice similarity coefficient (DSC) \u3e 0.75) but coronary artery segmentations were unsuccessful (DSC\u3c0.3). After implementing deep learning, DSC for the chambers and great vessels was ≥0.85 along with an improvement in the coronary arteries (DSC\u3e0.5). Similar accuracy was achieved when implementing deep learning for MR-guided RT. On average, automatic segmentations required ~10 minutes to generate per patient and deep learning only required 14 seconds. The inclusion of CS in the treatment planning process did not yield statistically significant changes in plan complexity, PTV, or OAR dose. Automatic segmentation results from deep learning pose major efficiency and accuracy gains for CS segmentation offering high potential for rapid implementation into radiation therapy planning for improved cardiac sparing. Introducing CS into RT planning for MR-guided RT presented an opportunity for more effective sparing with limited increase in plan complexity

Incorporating Cardiac Substructures Into Radiation Therapy For Improved Cardiac Sparing

Growing evidence suggests that radiation therapy (RT) doses to the heart and cardiac substructures (CS) are strongly linked to cardiac toxicities, though only the heart is considered clinically. This work aimed to utilize the superior soft-tissue contrast of magnetic resonance (MR) to segment CS, quantify uncertainties in their position, assess their effect on treatment planning and an MR-guided environment. Automatic substructure segmentation of 12 CS was completed using a novel hybrid MR/computed tomography (CT) atlas method and was improved upon using a 3-dimensional neural network (U-Net) from deep learning. Intra-fraction motion due to respiration was then quantified. The inter-fraction setup uncertainties utilizing a novel MR-linear accelerator were also quantified. Treatment planning comparisons were performed with and without substructure inclusions and methods to reduce radiation dose to sensitive CS were evaluated. Lastly, these described technologies (deep learning U-Net) were translated to an MR-linear accelerator and a segmentation pipeline was created. Automatic segmentations from the hybrid MR/CT atlas was able to generate accurate segmentations for the chambers and great vessels (Dice similarity coefficient (DSC) \u3e 0.75) but coronary artery segmentations were unsuccessful (DSC\u3c0.3). After implementing deep learning, DSC for the chambers and great vessels was ≥0.85 along with an improvement in the coronary arteries (DSC\u3e0.5). Similar accuracy was achieved when implementing deep learning for MR-guided RT. On average, automatic segmentations required ~10 minutes to generate per patient and deep learning only required 14 seconds. The inclusion of CS in the treatment planning process did not yield statistically significant changes in plan complexity, PTV, or OAR dose. Automatic segmentation results from deep learning pose major efficiency and accuracy gains for CS segmentation offering high potential for rapid implementation into radiation therapy planning for improved cardiac sparing. Introducing CS into RT planning for MR-guided RT presented an opportunity for more effective sparing with limited increase in plan complexity

A review of machine learning applications for the proton MR spectroscopy workflow

This literature review presents a comprehensive overview of machine learning (ML) applications in proton MR spectroscopy (MRS). As the use of ML techniques in MRS continues to grow, this review aims to provide the MRS community with a structured overview of the state-of-the-art methods. Specifically, we examine and summarize studies published between 2017 and 2023 from major journals in the MR field. We categorize these studies based on a typical MRS workflow, including data acquisition, processing, analysis, and artificial data generation. Our review reveals that ML in MRS is still in its early stages, with a primary focus on processing and analysis techniques, and less attention given to data acquisition. We also found that many studies use similar model architectures, with little comparison to alternative architectures. Additionally, the generation of artificial data is a crucial topic, with no consistent method for its generation. Furthermore, many studies demonstrate that artificial data suffers from generalization issues when tested on in vivo data. We also conclude that risks related to ML models should be addressed, particularly for clinical applications. Therefore, output uncertainty measures and model biases are critical to investigate. Nonetheless, the rapid development of ML in MRS and the promising results from the reviewed studies justify further research in this field.</p

Optimizing MRI-guided prostate ultrasound ablation therapy using retrospective analyses and artificial intelligence

Magnetic resonance imaging (MRI)-guided transurethral ultrasound ablation (TULSA) is an emerging therapy that has been used to treat prostate cancer (PCa). TULSA destroys prostate tissue with heat using therapeutic ultrasound. The heating is monitored in real-time using MRI thermometry. Despite TULSA’s promise, there are several challenges that have slowed its widespread adoption. Fortunately, MRI images and heating parameters from all TULSA treatments are stor ed. By conducting detailed retrospective analyses and applying deep learning on existing treatments, we can extract valuable information and then leverage this knowledge to optimize future TULSA treatments. One major challenge occurs for those patients who had PCa radiation therapy failure and are seeking salvage treatment with TULSA. Many of these patients have leftover metal markers in the prostate. These markers can hamper subsequent TULSA therapy because they introduce susceptibility artifacts in the MRI image and may also block the ultrasound, which may compromise treatment safety and efficacy. Through an extensive retrospective analysis, we have determined that gold markers tend not to affect the treatment outcome, except when located simultaneously close to the urethra and far from the target boundary, or when located directly on the target boundary itself. Clinically, gold markers had no apparent effect on treatment safety and efficacy compared to a control cohort without markers at the 12-month follow-up. Conversely, nitinol markers are generally problematic for TULSA. A second major challenge applies to all TULSA treatment indications. Immediately after TULSA therapy, MRI contrast agents are used to visualize the non-perfused volume, an objective measure of the ablation outcome. Unfortunately, even if undertreatment is observed, retreatment is not possible, forcing an additional treatment several months later, and with it the associated risks of a second intervention. By training a deep learning model with existing TULSA treatment-day, contrast-free MRI image sets, we have predicted the non-perfused volume with an accuracy comparable to modern-day deep learning prostate segmentation methods. Overall, this work will help daily clinical practice and increase the odds of a successful TULSA therapy.MRI-ohjatun eturauhasen ultraääniablaatiohoidon optimointi retrospektiivisten analyysien ja tekoälyn avulla Magneettikuvaus(MRI)-ohjattu virtsaputken kautta annettu ultraääniablaatio (TULSA) on uusi primaarin ja sädehoidon jälkeen paikallisesti uusiutuneen eturauhassyövän (PCa) hoitomuoto. Menetelmässä eturauhaskudosta koaguloidaan korkean intensiteetin ultraäänellä reaaliaikaisessa MRI-ohjauksessa, mikä parantaa hoidon tarkkuutta. Lupaavista kliinisistä tuloksista huolimatta MRI-ohjaus altistaa teknisille ja kliinisille haasteille, mitkä ovat hidastaneet TULSA-hoidon laajempaa käyttöönottoa. TULSA-hoidossa jokainen vaihe rekisteröidään MRI-kuvin. Koneoppimista hyödyntämällä voidaan retrospektiivisesti analysoida näitä MRI-kuvia TULSA-hoitotulosten optimoimiseksi. Sädehoidon ohjauksessa käytetyt eturauhaseen asetetut merkkijyvät saattavat vaikuttaa TULSA-hoidon tehoon ja turvallisuuteen uusiutuneessa PCa:ssä, koska ne voivat aiheuttaa artefaktoja MRI-kuvaan ja estää ultraäänen etenemisen. Laajassa retrospektiivisessa analyysissä todettiin, että kultamerkkijyvät eivät yleensä vaikuta hoitotulokseen, elleivät ne sijaitse samanaikaisesti lähellä virtsaputkea ja kaukana hoitokohteesta tai suoraan kohteen edessä. Kultamerkkijyvillä ei ollut ilmeistä vaikutusta hoidon turvallisuuteen ja tehokkuuteen verrattuna kontrolliryhmään ilman merkkijyviä 12 kuukauden seurannassa. Välittömästi TULSA-hoidon jälkeen hoitotulos varmistetaan merkkiainetehosteisilla MRI-kuvilla, joilla visualisoidaan verenkierroton alue, mikä korreloi akuuttiin kudosvaurioon eli onnistuneeseen hoitovasteeseen. Ongelmana on, että vaikka merkkiainetehosteisissa MRI-kuvissa todettaisiin riittämätön hoitovaste, uudelleenhoito ei ole samalla hoitokerralla mahdollista, koska eturauhaseen kerääntynyt merkkiaine estää hoidon. Tällöin tarvitaan uusi hoitokerta kuukausien kuluttua toimenpiteen sisältämineen riskeineen, mikä viivästyttää hoitoa ja kuormittaa potilasta. Tässä tutkimuksessa onnistuttiin tarkasti ennustamaan verenkierroton alue hoidonaikaisista merkkiainetehostamattomista MRI-kuvista hyödyntämällä syväoppimismallia. Näillä havainnoilla on tärkeä kliininen merkitys TULSA-hoitotulosten parantamisessa

Resting state network mapping in individuals using deep learning

INTRODUCTION: Resting state functional MRI (RS-fMRI) is currently used in numerous clinical and research settings. The localization of resting state networks (RSNs) has been utilized in applications ranging from group analysis of neurodegenerative diseases to individual network mapping for pre-surgical planning of tumor resections. Reproducibility of these results has been shown to require a substantial amount of high-quality data, which is not often available in clinical or research settings. METHODS: In this work, we report voxelwise mapping of a standard set of RSNs using a novel deep 3D convolutional neural network (3DCNN). The 3DCNN was trained on publicly available functional MRI data acquired in RESULTS: Our results indicate this method can be applied in individual subjects and is highly resistant to both noisy data and fewer RS-fMRI time points than are typically acquired. Further, our results show core regions within each network that exhibit high average probability and low STD. DISCUSSION: The 3DCNN algorithm can generate individual RSN localization maps, which are necessary for clinical applications. The similarity between 3DCNN mapping results and task-based fMRI responses supports the association of specific functional tasks with RSNs

Novel Image Acquisition and Reconstruction Methods: Towards Autonomous MRI

Magnetic Resonance Imaging (MR Imaging, or MRI) offers superior soft-tissue contrast compared to other medical imaging modalities. However, access to MRI across developing countries ranges from prohibitive to scarcely available. The lack of educational facilities and the excessive costs involved in imparting technical training have resulted in a lack of skilled human resources required to operate MRI systems in developing countries. While diagnostic medical imaging improves the utilization of facility-based rural health services and impacts management decisions, MRI requires technical expertise to set up the patient, acquire, visualize, and interpret data. The availability of such local expertise in underserved geographies is challenging. Inefficient workflows and usage of MRI result in challenges related to financial and temporal access in countries with higher scanner densities than the global average of 5.3 per million people. MRI is routinely employed for neuroimaging and, in particular, for dementia screening. Dementia affected 50 million people worldwide in 2018, with an estimated economic impact of US $1 trillion a year, and Alzheimer’s Disease (AD) accounts for up to 60–80% of dementia cases. However, AD-imaging using MRI is time-consuming, and protocol optimization to accelerate MR Imaging requires local expertise since each pulse sequence involves multiple configurable parameters that need optimization for acquisition time, image contrast, and image quality. The lack of this expertise contributes to the highly inefficient utilization of MRI services, diminishing their clinical value. Augmenting human capabilities can tackle these challenges and standardize the practice. Autonomous and time-efficient acquisition, reconstruction, and visualization schemes to maximize MRI hardware usage and solutions that reduce reliance on human operation of MRI systems could alleviate some of the challenges associated with the requirement/absence of skilled human resources. We first present a preliminary demonstration of AMRI that simplifies the end-to-end MRI workflow of registering the subject, setting up and invoking an imaging session, acquiring and reconstructing the data, and visualizing the images. Our initial implementation of AMRI separates the required intelligence and user interaction from the acquisition hardware. AMRI performs intelligent protocolling and intelligent slice planning. Intelligent protocolling optimizes contrast value while satisfying signal-to-noise ratio and acquisition time constraints. We acquired data from four healthy volunteers across three experiments that differed in acquisition time constraints. AMRI achieved comparable image quality across all experiments despite optimizing for acquisition duration, therefore indirectly optimizing for MR Value – a metric to quantify the value of MRI. We believe we have demonstrated the first Autonomous MRI of the brain. We also present preliminary results from a deep learning (DL) tool for generating first-read text-based radiological reports directly from input brain images. It can potentially alleviate the burden on radiologists who experience the seventh-highest levels of burnout among all physicians, according to a 2015 survey. Next, we accelerate the routine brain imaging protocol employed at the Columbia University Irving Medical Center and leverage DL methods to boost image quality via image-denoising. Since MR physics dictates that the volume of the object being imaged influences the amount of signal received, we also demonstrate subject-specific image-denoising. The accelerated protocol resulted in a factor of 1.94 gain in imaging throughput, translating to a 72.51% increase in MR Value. We also demonstrate that this accelerated protocol can potentially be employed for AD imaging. Finally, we present ArtifactID – a DL tool to identify Gibbs ringing in low-field (0.36 T) and high-field (1.5 T and 3.0 T) brain MRI. We train separate binary classification models for low-field and high-field data, and visual explanations are generated via the Grad-CAM explainable AI method to help develop trust in the models’ predictions. We also demonstrate detecting motion using an accelerometer in a low-field MRI scanner since low-field MRI is prone to artifacts. In conclusion, our novel contributions in this work include: i) a software framework to demonstrate an initial implementation of autonomous brain imaging; ii) an end-to-end framework that leverages intelligent protocolling and DL-based image-denoising that can potentially be employed for accelerated AD imaging; and iii) a DL-based tool for automated identification of Gibbs ringing artifacts that may interfere with diagnosis at the time of radiological reading. We envision AMRI augmenting human expertise to alleviate the challenges associated with the scarcity of skilled human resources and contributing to globally accessible MRI