Medical Image Analytics (Radiomics) with Machine/Deeping Learning for Outcome Modeling in Radiation Oncology

Image-based quantitative analysis (radiomics) has gained great attention recently. Radiomics possesses promising potentials to be applied in the clinical practice of radiotherapy and to provide personalized healthcare for cancer patients. However, there are several challenges along the way that this thesis will attempt to address. Specifically, this thesis focuses on the investigation of repeatability and reproducibility of radiomics features, the development of new machine/deep learning models, and combining these for robust outcomes modeling and their applications in radiotherapy. Radiomics features suffer from robustness issues when applied to outcome modeling problems, especially in head and neck computed tomography (CT) images. These images tend to contain streak artifacts due to patients’ dental implants. To investigate the influence of artifacts for radiomics modeling performance, we firstly developed an automatic artifact detection algorithm using gradient-based hand-crafted features. Then, comparing the radiomics models trained on ‘clean’ and ‘contaminated’ datasets. The second project focused on using hand-crafted radiomics features and conventional machine learning methods for the prediction of overall response and progression-free survival for Y90 treated liver cancer patients. By identifying robust features and embedding prior knowledge in the engineered radiomics features and using bootstrapped LASSO to select robust features, we trained imaging and dose based models for the desired clinical endpoints, highlighting the complementary nature of this information in Y90 outcomes prediction. Combining hand-crafted and machine learnt features can take advantage of both expert domain knowledge and advanced data-driven approaches (e.g., deep learning). Thus, we proposed a new variational autoencoder network framework that modeled radiomics features, clinical factors, and raw CT images for the prediction of intrahepatic recurrence-free and overall survival for hepatocellular carcinoma (HCC) patients in this third project. The proposed approach was compared with widely used Cox proportional hazard model for survival analysis. Our proposed methods achieved significant improvement in terms of the prediction using the c-index metric highlighting the value of advanced modeling techniques in learning from limited and heterogeneous information in actuarial prediction of outcomes. Advances in stereotactic radiation therapy (SBRT) has led to excellent local tumor control with limited toxicities for HCC patients, but intrahepatic recurrence still remains prevalent. As an extension of the third project, we not only hope to predict the time to intrahepatic recurrence, but also the location where the tumor might recur. This will be clinically beneficial for better intervention and optimizing decision making during the process of radiotherapy treatment planning. To address this challenging task, firstly, we proposed an unsupervised registration neural network to register atlas CT to patient simulation CT and obtain the liver’s Couinaud segments for the entire patient cohort. Secondly, a new attention convolutional neural network has been applied to utilize multimodality images (CT, MR and 3D dose distribution) for the prediction of high-risk segments. The results showed much improved efficiency for obtaining segments compared with conventional registration methods and the prediction performance showed promising accuracy for anticipating the recurrence location as well. Overall, this thesis contributed new methods and techniques to improve the utilization of radiomics for personalized radiotherapy. These contributions included new algorithm for detecting artifacts, a joint model of dose with image heterogeneity, combining hand-crafted features with machine learnt features for actuarial radiomics modeling, and a novel approach for predicting location of treatment failure.PHDApplied PhysicsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/163092/1/liswei_1.pd

Image Processing Algorithms for Detection of Anomalies in Orthopedic Surgery Implants

Orthopedic implant procedures for hip implants are performed on 300,000 patients annually in the United States, with 22.3 million procedures worldwide. While most such operations are successfully performed to relieve pain and restore joint function for the duration of the patient\u27s life, advances in medicine have enabled patients to outlive the life of their implant, increasing the likelihood of implant failure. There is significant advantage to the patient, the surgeon, and the medical community in early detection of implant failures.The research work presented in this thesis demonstrates a non-invasive digital image processing technique for the automated detection of specific arthroplasty failures before requiring revision surgery. This thesis studies hip implant loosening as the primary cause of failure. A combination of digital image segmentation, representation and numerical description is employed and validated on 2-D X-ray images of hip implant phantoms to detect 3-D rotations of the implant, with the support of radial basis function neural networks to accomplish this task. A successful clinical implementation of the methods developed in this thesis can eliminate the need for revision surgery and prolong the life of the orthopedic implant

Reduction of Limited Angle Artifacts in Medical Tomography via Image Reconstruction

Artifacts are unwanted effects in tomographic images that do not reflect the nature of the object. Their widespread occurrence makes their reduction and if possible removal an important subject in the development of tomographic image reconstruction algorithms. Limited angle artifacts are caused by the limited angular measurements, constraining the available tomographic information. This thesis focuses on reducing these artifacts via image reconstruction in two cases of incomplete measurements from: (1) the gaps left after the removal of high density objects such as dental fillings, screws and implants in computed tomography (CT) and (2) partial ring scanner configurations in positron emission tomography (PET). In order to include knowledge about the measurement and noise, prior terms were used within the reconstruction methods. Careful consideration was given to the trade-off between image blurring and noise reduction upon reconstruction of low-dose measurements.Development of reconstruction methods is an incremental process starting with testing on simple phantoms towards more clinically relevant ones by modeling the respective physical processes involved. In this work, phantoms were constructed to ensure that the proposed reconstruction methods addressed to the limited angle problem. The reconstructed images were assessed qualitatively and quantitatively in terms of noise reduction, edge sharpness and contrast recovery.Maximum a posteriori (MAP) estimation with median root prior (MRP) was selected for the reconstruction of limited angle measurements. MAP with MRP successfully reduced the artifacts caused by limited angle data in various datasets, tested with the reconstruction of both list-mode and projection data. In all cases, its performance was found to be superior to conventional reconstruction methods such as total-variation (TV) prior, maximum likelihood expectation maximization (MLEM) and filtered backprojection (FBP). MAP with MRP was also more robust with respect to parameter selection than MAP with TV prior.This thesis demonstrates the wide-range applicability of MAP with MRP in medical tomography, especially in low-dose imaging. Furthermore, we emphasize the importance of developing and testing reconstruction methods with application-specific phantoms, together with the properties and limitations of the measurements in mind

What scans we will read: imaging instrumentation trends in clinical oncology

Oncological diseases account for a significant portion of the burden on public healthcare systems with associated costs driven primarily by complex and long-lasting therapies. Through the visualization of patient-specific morphology and functional-molecular pathways, cancerous tissue can be detected and characterized non- invasively, so as to provide referring oncologists with essential information to support therapy management decisions. Following the onset of stand-alone anatomical and functional imaging, we witness a push towards integrating molecular image information through various methods, including anato-metabolic imaging (e.g., PET/ CT), advanced MRI, optical or ultrasound imaging. This perspective paper highlights a number of key technological and methodological advances in imaging instrumentation related to anatomical, functional, molecular medicine and hybrid imaging, that is understood as the hardware-based combination of complementary anatomical and molecular imaging. These include novel detector technologies for ionizing radiation used in CT and nuclear medicine imaging, and novel system developments in MRI and optical as well as opto-acoustic imaging. We will also highlight new data processing methods for improved non-invasive tissue characterization. Following a general introduction to the role of imaging in oncology patient management we introduce imaging methods with well-defined clinical applications and potential for clinical translation. For each modality, we report first on the status quo and point to perceived technological and methodological advances in a subsequent status go section. Considering the breadth and dynamics of these developments, this perspective ends with a critical reflection on where the authors, with the majority of them being imaging experts with a background in physics and engineering, believe imaging methods will be in a few years from now. Overall, methodological and technological medical imaging advances are geared towards increased image contrast, the derivation of reproducible quantitative parameters, an increase in volume sensitivity and a reduction in overall examination time. To ensure full translation to the clinic, this progress in technologies and instrumentation is complemented by progress in relevant acquisition and image-processing protocols and improved data analysis. To this end, we should accept diagnostic images as “data”, and – through the wider adoption of advanced analysis, including machine learning approaches and a “big data” concept – move to the next stage of non-invasive tumor phenotyping. The scans we will be reading in 10 years from now will likely be composed of highly diverse multi- dimensional data from multiple sources, which mandate the use of advanced and interactive visualization and analysis platforms powered by Artificial Intelligence (AI) for real-time data handling by cross-specialty clinical experts with a domain knowledge that will need to go beyond that of plain imaging

医用超音波における散乱体分布の高解像かつ高感度な画像化に関する研究

Ultrasound imaging as an effective method is widely used in medical diagnosis andNDT (non-destructive testing). In particular, ultrasound imaging plays an important role in medical diagnosis due to its safety, noninvasive, inexpensiveness and real-time compared with other medical imaging techniques. However, in general the ultrasound imaging has more speckles and is low definition than the MRI (magnetic resonance imaging) and X-ray CT (computerized tomography). Therefore, it is important to improve the ultrasound imaging quality. In this study, there are three newproposals. The first is the development of a high sensitivity transducer that utilizes piezoelectric charge directly for FET (field effect transistor) channel control. The second is a proposal of a method for estimating the distribution of small scatterers in living tissue using the empirical Bayes method. The third is a super-resolution imagingmethod of scatterers with strong reflection such as organ boundaries and blood vessel walls. The specific description of each chapter is as follows: Chapter 1: The fundamental characteristics and the main applications of ultrasound are discussed, then the advantages and drawbacks of medical ultrasound are high-lighted. Based on the drawbacks, motivations and objectives of this study are stated. Chapter 2: To overcome disadvantages of medical ultrasound, we advanced our studyin two directions: designing new transducer improves the acquisition modality itself, onthe other hand new signal processing improve the acquired echo data. Therefore, the conventional techniques related to the two directions are reviewed. Chapter 3: For high performance piezoelectric, a structure that enables direct coupling of a PZT (lead zirconate titanate) element to the gate of a MOSFET (metal-oxide semiconductor field-effect transistor) to provide a device called the PZT-FET that acts as an ultrasound receiver was proposed. The experimental analysis of the PZT-FET, in terms of its reception sensitivity, dynamic range and -6 dB reception bandwidth have been investigated. The proposed PZT-FET receiver offers high sensitivity, wide dynamic range performance when compared to the typical ultrasound transducer. Chapter 4: In medical ultrasound imaging, speckle patterns caused by reflection interference from small scatterers in living tissue are often suppressed by various methodologies. However, accurate imaging of small scatterers is important in diagnosis; therefore, we investigated influence of speckle pattern on ultrasound imaging by the empirical Bayesian learning. Since small scatterers are spatially correlated and thereby constitute a microstructure, we assume that scatterers are distributed according to the AR (auto regressive) model with unknown parameters. Under this assumption, the AR parameters are estimated by maximizing the marginal likelihood function, and the scatterers distribution is estimated as a MAP (maximum a posteriori) estimator. The performance of our method is evaluated by simulations and experiments. Through the results, we confirmed that the band limited echo has sufficient information of the AR parameters and the power spectrum of the echoes from the scatterers is properly extrapolated. Chapter 5: The medical ultrasound imaging of strong reflectance scatterers based on the MUSIC algorithm is the main subject of Chapter 5. Previously, we have proposed a super-resolution ultrasound imaging based on multiple TRs (transmissions/receptions) with different carrier frequencies called SCM (super resolution FM-chirp correlation method). In order to reduce the number of required TRs for the SCM, the method has been extended to the SA (synthetic aperture) version called SA-SCM. However, since super-resolution processing is performed for each line data obtained by the RBF (reception beam forming) in the SA-SCM, image discontinuities tend to occur in the lateral direction. Therefore, a new method called SCM-weighted SA is proposed, in this version the SCM is performed on each transducer element, and then the SCM result is used as the weight for RBF. The SCM-weighted SA can generate multiple B-mode images each of which corresponds to each carrier frequency, and the appropriate low frequency images among them have no grating lobes. For a further improvement, instead of simple averaging, the SCM applied to the result of the SCM-weighted SA for all frequencies again, which is called SCM-weighted SA-SCM. We evaluated the effectiveness of all the methods by simulations and experiments. From the results, it can be confirmed that the extension of the SCM framework can help ultrasound imaging reduce grating lobes, perform super-resolution and better SNR(signal-to-noise ratio). Chapter 6: A discussion of the overall content of the thesis as well as suggestions for further development together with the remaining problems are summarized.首都大学東京, 2019-03-25, 博士（工学）首都大学東

Artificial Intelligence in Radiation Therapy

Artificial intelligence (AI) has great potential to transform the clinical workflow of radiotherapy. Since the introduction of deep neural networks, many AI-based methods have been proposed to address challenges in different aspects of radiotherapy. Commercial vendors have started to release AI-based tools that can be readily integrated to the established clinical workflow. To show the recent progress in AI-aided radiotherapy, we have reviewed AI-based studies in five major aspects of radiotherapy including image reconstruction, image registration, image segmentation, image synthesis, and automatic treatment planning. In each section, we summarized and categorized the recently published methods, followed by a discussion of the challenges, concerns, and future development. Given the rapid development of AI-aided radiotherapy, the efficiency and effectiveness of radiotherapy in the future could be substantially improved through intelligent automation of various aspects of radiotherapy

Machine learning-based automated segmentation with a feedback loop for 3D synchrotron micro-CT

Die Entwicklung von Synchrotronlichtquellen der dritten Generation hat die Grundlage für die Untersuchung der 3D-Struktur opaker Proben mit einer Auflösung im Mikrometerbereich und höher geschaffen. Dies führte zur Entwicklung der Röntgen-Synchrotron-Mikro-Computertomographie, welche die Schaffung von Bildgebungseinrichtungen zur Untersuchung von Proben verschiedenster Art förderte, z.B. von Modellorganismen, um die Physiologie komplexer lebender Systeme besser zu verstehen. Die Entwicklung moderner Steuerungssysteme und Robotik ermöglichte die vollständige Automatisierung der Röntgenbildgebungsexperimente und die Kalibrierung der Parameter des Versuchsaufbaus während des Betriebs. Die Weiterentwicklung der digitalen Detektorsysteme führte zu Verbesserungen der Auflösung, des Dynamikbereichs, der Empfindlichkeit und anderer wesentlicher Eigenschaften. Diese Verbesserungen führten zu einer beträchtlichen Steigerung des Durchsatzes des Bildgebungsprozesses, aber auf der anderen Seite begannen die Experimente eine wesentlich größere Datenmenge von bis zu Dutzenden von Terabyte zu generieren, welche anschließend manuell verarbeitet wurden. Somit ebneten diese technischen Fortschritte den Weg für die Durchführung effizienterer Hochdurchsatzexperimente zur Untersuchung einer großen Anzahl von Proben, welche Datensätze von besserer Qualität produzierten. In der wissenschaftlichen Gemeinschaft besteht daher ein hoher Bedarf an einem effizienten, automatisierten Workflow für die Röntgendatenanalyse, welcher eine solche Datenlast bewältigen und wertvolle Erkenntnisse für die Fachexperten liefern kann. Die bestehenden Lösungen für einen solchen Workflow sind nicht direkt auf Hochdurchsatzexperimente anwendbar, da sie für Ad-hoc-Szenarien im Bereich der medizinischen Bildgebung entwickelt wurden. Daher sind sie nicht für Hochdurchsatzdatenströme optimiert und auch nicht in der Lage, die hierarchische Beschaffenheit von Proben zu nutzen. Die wichtigsten Beiträge der vorliegenden Arbeit sind ein neuer automatisierter Analyse-Workflow, der für die effiziente Verarbeitung heterogener Röntgendatensätze hierarchischer Natur geeignet ist. Der entwickelte Workflow basiert auf verbesserten Methoden zur Datenvorverarbeitung, Registrierung, Lokalisierung und Segmentierung. Jede Phase eines Arbeitsablaufs, die eine Trainingsphase beinhaltet, kann automatisch feinabgestimmt werden, um die besten Hyperparameter für den spezifischen Datensatz zu finden. Für die Analyse von Faserstrukturen in Proben wurde eine neue, hochgradig parallelisierbare 3D-Orientierungsanalysemethode entwickelt, die auf einem neuartigen Konzept der emittierenden Strahlen basiert und eine präzisere morphologische Analyse ermöglicht. Alle entwickelten Methoden wurden gründlich an synthetischen Datensätzen validiert, um ihre Anwendbarkeit unter verschiedenen Abbildungsbedingungen quantitativ zu bewerten. Es wurde gezeigt, dass der Workflow in der Lage ist, eine Reihe von Datensätzen ähnlicher Art zu verarbeiten. Darüber hinaus werden die effizienten CPU/GPU-Implementierungen des entwickelten Workflows und der Methoden vorgestellt und der Gemeinschaft als Module für die Sprache Python zur Verfügung gestellt. Der entwickelte automatisierte Analyse-Workflow wurde erfolgreich für Mikro-CT-Datensätze angewandt, die in Hochdurchsatzröntgenexperimenten im Bereich der Entwicklungsbiologie und Materialwissenschaft gewonnen wurden. Insbesondere wurde dieser Arbeitsablauf für die Analyse der Medaka-Fisch-Datensätze angewandt, was eine automatisierte Segmentierung und anschließende morphologische Analyse von Gehirn, Leber, Kopfnephronen und Herz ermöglichte. Darüber hinaus wurde die entwickelte Methode der 3D-Orientierungsanalyse bei der morphologischen Analyse von Polymergerüst-Datensätzen eingesetzt, um einen Herstellungsprozess in Richtung wünschenswerter Eigenschaften zu lenken

Advancements and Breakthroughs in Ultrasound Imaging

Ultrasonic imaging is a powerful diagnostic tool available to medical practitioners, engineers and researchers today. Due to the relative safety, and the non-invasive nature, ultrasonic imaging has become one of the most rapidly advancing technologies. These rapid advances are directly related to the parallel advancements in electronics, computing, and transducer technology together with sophisticated signal processing techniques. This book focuses on state of the art developments in ultrasonic imaging applications and underlying technologies presented by leading practitioners and researchers from many parts of the world

Quantitative imaging in radiation oncology

Artificially intelligent eyes, built on machine and deep learning technologies, can empower our capability of analysing patients’ images. By revealing information invisible at our eyes, we can build decision aids that help our clinicians to provide more effective treatment, while reducing side effects. The power of these decision aids is to be based on patient tumour biologically unique properties, referred to as biomarkers. To fully translate this technology into the clinic we need to overcome barriers related to the reliability of image-derived biomarkers, trustiness in AI algorithms and privacy-related issues that hamper the validation of the biomarkers. This thesis developed methodologies to solve the presented issues, defining a road map for the responsible usage of quantitative imaging into the clinic as decision support system for better patient care