Jekyll: Attacking Medical Image Diagnostics using Deep Generative Models

Advances in deep neural networks (DNNs) have shown tremendous promise in the medical domain. However, the deep learning tools that are helping the domain, can also be used against it. Given the prevalence of fraud in the healthcare domain, it is important to consider the adversarial use of DNNs in manipulating sensitive data that is crucial to patient healthcare. In this work, we present the design and implementation of a DNN-based image translation attack on biomedical imagery. More specifically, we propose Jekyll, a neural style transfer framework that takes as input a biomedical image of a patient and translates it to a new image that indicates an attacker-chosen disease condition. The potential for fraudulent claims based on such generated 'fake' medical images is significant, and we demonstrate successful attacks on both X-rays and retinal fundus image modalities. We show that these attacks manage to mislead both medical professionals and algorithmic detection schemes. Lastly, we also investigate defensive measures based on machine learning to detect images generated by Jekyll.Comment: Published in proceedings of the 5th European Symposium on Security and Privacy (EuroS&P '20

3D Medical Image Segmentation based on multi-scale MPU-Net

The high cure rate of cancer is inextricably linked to physicians' accuracy in diagnosis and treatment, therefore a model that can accomplish high-precision tumor segmentation has become a necessity in many applications of the medical industry. It can effectively lower the rate of misdiagnosis while considerably lessening the burden on clinicians. However, fully automated target organ segmentation is problematic due to the irregular stereo structure of 3D volume organs. As a basic model for this class of real applications, U-Net excels. It can learn certain global and local features, but still lacks the capacity to grasp spatial long-range relationships and contextual information at multiple scales. This paper proposes a tumor segmentation model MPU-Net for patient volume CT images, which is inspired by Transformer with a global attention mechanism. By combining image serialization with the Position Attention Module, the model attempts to comprehend deeper contextual dependencies and accomplish precise positioning. Each layer of the decoder is also equipped with a multi-scale module and a cross-attention mechanism. The capability of feature extraction and integration at different levels has been enhanced, and the hybrid loss function developed in this study can better exploit high-resolution characteristic information. Moreover, the suggested architecture is tested and evaluated on the Liver Tumor Segmentation Challenge 2017 (LiTS 2017) dataset. Compared with the benchmark model U-Net, MPU-Net shows excellent segmentation results. The dice, accuracy, precision, specificity, IOU, and MCC metrics for the best model segmentation results are 92.17%, 99.08%, 91.91%, 99.52%, 85.91%, and 91.74%, respectively. Outstanding indicators in various aspects illustrate the exceptional performance of this framework in automatic medical image segmentation.Comment: 37 page

Personalized prostate cancer management : AI-assisted prostate pathology and improved active surveillance

Prostate cancer is a major global health concern and is the most common cancer-related cause of death in Sweden. Prostate cancer screening using PSA has been shown to reduce prostate cancer mortality but also leads to significant overdiagnosis and overtreatment of low-risk cancers. Improved risk stratification and effective active surveillance are crucial to balancing the benefits of screening with the risk of overdiagnosis and overtreatment. In Study I, we studied the uptake and the follow-up of active surveillance using a retrospective cohort of patients who were diagnosed with low-risk prostate cancer between 2008 and 2017 in Stockholm County. Our results showed that only 50% of eligible active surveillance patients received active surveillance as their primary treatment choice at diagnosis. Most men that enrolled in active surveillance remained on surveillance during the first years after diagnosis (82% during a median 3.5 years), but did not receive a follow up according to guidelines with regard to repeat biopsies and PSA tests. Current clinical practice has seen an increase in the use of magnetic resonance imaging (MRI) and the incorporation of risk prediction models to select men with the highest suspicion of clinically significant prostate cancer for prostate biopsy. However, the effectiveness and how MRI and risk prediction models should be incorporated into active surveillance follow-up have yet to be established. Study II evaluated the performance of MRI-targeted biopsies and a blood-based risk prediction model (the Stockholm3 test) for monitoring disease progression in patients on active surveillance and compared this to the conventional follow-up using PSA and systematic biopsies. When MRI-targeted and systematic biopsies were combined, the detection rate of clinically significant prostate cancer increased when compared to conventional systematic biopsies. Biopsies performed in MRI-positive men resulted in a 49% reduction in performed biopsies, at the expense of failing to diagnose 1.4% clinically significant prostate cancer in MRInegative men. The incorporation of the Stockholm3 test showed a 27% reduction in required MRI investigations and a 57% reduction in performed biopsies compared to performing only systematic biopsies. In Study III, we digitized biopsy cores from STHLM3 participants to develop an artificial intelligence (AI) for prostate cancer diagnostics. The AI system demonstrated clinically useful performance that was comparable to that of the study pathologist for cancer detection (AUC of 0.986) and for predictions of cancer length (correlation of 0.87) and grading performance that was on par with that of expert prostate pathologists. In Study IV, we developed a conformal predictor to estimate the uncertainty of the predictions for the model in Study III. The uncertainty estimates were used to control the error rate so that only predictions with high confidence are accepted and unreliable predictions can be detected. The conformal predictor was able to identify unreliable predictions as a result of variations in digital pathology scanners, preparation of tissue in different pathology laboratories, and the existence of unusual prostate tissue that the AI model was not exposed to during training. Little is known about the relationships between prostate cancer genetic risk factors and the morphology of prostate tissue. In Study V:, we investigated whether weakly supervised deep learning can learn to detect such possible associations. The findings in this paper imply relationships between prostatic tissue morphology and genetic risk factors for prostate cancer, particularly in young men. These results provide proof of principle for exploring the use of morphological information in multi-modal prostate cancer risk prediction algorithms. In conclusion, the purpose of this thesis was to describe possible extensions to improve prostate cancer active surveillance management, as well as to develop prediction models for improved prostate cancer diagnostics

IoT Health Devices: Exploring Security Risks in the Connected Landscape

The concept of the Internet of Things (IoT) spans decades, and the same can be said for its inclusion in healthcare. The IoT is an attractive target in medicine; it offers considerable potential in expanding care. However, the application of the IoT in healthcare is fraught with an array of challenges, and also, through it, numerous vulnerabilities that translate to wider attack surfaces and deeper degrees of damage possible to both consumers and their confidence within health systems, as a result of patient-specific data being available to access. Further, when IoT health devices (IoTHDs) are developed, a diverse range of attacks are possible. To understand the risks in this new landscape, it is important to understand the architecture of IoTHDs, operations, and the social dynamics that may govern their interactions. This paper aims to document and create a map regarding IoTHDs, lay the groundwork for better understanding security risks in emerging IoTHD modalities through a multi-layer approach, and suggest means for improved governance and interaction. We also discuss technological innovations expected to set the stage for novel exploits leading into the middle and latter parts of the 21st century

Machine learning models for diagnosis and prognosis of Parkinson's disease using brain imaging: general overview, main challenges, and future directions

Parkinson’s disease (PD) is a progressive and complex neurodegenerative disorder associated with age that affects motor and cognitive functions. As there is currently no cure, early diagnosis and accurate prognosis are essential to increase the effectiveness of treatment and control its symptoms. Medical imaging, specifically magnetic resonance imaging (MRI), has emerged as a valuable tool for developing support systems to assist in diagnosis and prognosis. The current literature aims to improve understanding of the disease’s structural and functional manifestations in the brain. By applying artificial intelligence to neuroimaging, such as deep learning (DL) and other machine learning (ML) techniques, previously unknown relationships and patterns can be revealed in this high-dimensional data. However, several issues must be addressed before these solutions can be safely integrated into clinical practice. This review provides a comprehensive overview of recent ML techniques analyzed for the automatic diagnosis and prognosis of PD in brain MRI. The main challenges in applying ML to medical diagnosis and its implications for PD are also addressed, including current limitations for safe translation into hospitals. These challenges are analyzed at three levels: disease-specific, task- specific, and technology-specific. Finally, potential future directions for each challenge and future perspectives are discusse

Diagnóstico de esclerosis múltiple mediante análisis de registros de tomografía de coherencia óptica y redes neuronales convolucionales entrenadas con imágenes sintéticas

Antecedentes: La Esclerosis Múltiple (EM) es una enfermedad del sistema nervioso central altamente discapacitante y que se presenta con frecuencia en adultos jóvenes. Para su diagnóstico se utilizan los criterios de McDonald, basados principalmente en evidencias de resonancia magnética, estudio del líquido cefalorraquídeo y el estado clínico del paciente. Sin embargo es conveniente investigar nuevos biomarcadores que permitan un diagnóstico fiable y no invasivo en las primeras fases de la enfermedad, permitiendo de este modo el uso de tratamientos modificadores de la enfermedad, ya que puede suponer una mejor evolución de los pacientes. Objetivos: El objetivo general de la presente tesis doctoral es investigar nuevos métodos de procesamiento y clasificación de imágenes de espesores de diferentes estructuras de la retina, obtenidas mediante Tomografía de Coherencia Óptica de fuente de barrido (SS-OCT) para conseguir un diagnóstico precoz de EM. Métodos: Se dispone de imágenes de espesores de las siguientes estructuras de la retina: retina completa, RNFL, GCL+, GCL++ y coroides, adquiridas por un equipo SS-OCT, en una base de datos formada por 48 sujetos de control y 48 pacientes con EM de diagnóstico reciente. Para la identificación de las estructuras y de las regiones con mayor capacidad discriminante se utiliza el método Relieff de categorización de características. Como clasificador, se utiliza una Red Neuronal Convolucional (RNC), y para evitar problemas de sobreajuste, se generan imágenes sintéticas con Redes Generativas Antagónicas. La comprobación de los métodos de clasificación se realiza mediante validación cruzada dejando uno fuera. Resultados: No existe diferencia significativa entre el grupo de control y el grupo de pacientes ni en edad ni en distribución entre sexos. Los pacientes han tenido un diagnóstico reciente (7,35 ± 1,95 meses). La aplicación del método Relieff detecta que las tres estructuras con mayor capacidad discriminante son GCL+, GCL++ y el espesor de la retina completa. Mediante las Redes Generativas Antagónicas se generan 100 imágenes SS-OCT sintéticas de sujetos de control y 100 imágenes SS-OCT de pacientes EM. Utilizando las imágenes originales en el clasificador RNC se obtiene una precisión de 0,968; en imágenes filtradas con el método Relieff la precisión de es 1,0 y utilizando las imágenes sintéticas para el entrenamiento de la RNC también es 1,0. Si se dispone únicamente del 50% de las imágenes originales, se comprueba la ventaja de disponer datos sintéticos para el entrenamiento de la RNC: la precisión aumenta de 0,66 a 0,96. Conclusiones: Las alteraciones estructurales neurorretinianas en las primeras fases de la EM son adecuadas para implementar un sistema de ayuda al diagnóstico mediante una red neuronal convolucional con un excelente nivel de precisión

Artificial intelligence in cancer imaging: Clinical challenges and applications

Judgement, as one of the core tenets of medicine, relies upon the integration of multilayered data with nuanced decision making. Cancer offers a unique context for medical decisions given not only its variegated forms with evolution of disease but also the need to take into account the individual condition of patients, their ability to receive treatment, and their responses to treatment. Challenges remain in the accurate detection, characterization, and monitoring of cancers despite improved technologies. Radiographic assessment of disease most commonly relies upon visual evaluations, the interpretations of which may be augmented by advanced computational analyses. In particular, artificial intelligence (AI) promises to make great strides in the qualitative interpretation of cancer imaging by expert clinicians, including volumetric delineation of tumors over time, extrapolation of the tumor genotype and biological course from its radiographic phenotype, prediction of clinical outcome, and assessment of the impact of disease and treatment on adjacent organs. AI may automate processes in the initial interpretation of images and shift the clinical workflow of radiographic detection, management decisions on whether or not to administer an intervention, and subsequent observation to a yet to be envisioned paradigm. Here, the authors review the current state of AI as applied to medical imaging of cancer and describe advances in 4 tumor types (lung, brain, breast, and prostate) to illustrate how common clinical problems are being addressed. Although most studies evaluating AI applications in oncology to date have not been vigorously validated for reproducibility and generalizability, the results do highlight increasingly concerted efforts in pushing AI technology to clinical use and to impact future directions in cancer care

Explainable deep learning classifiers for disease detection based on structural brain MRI data

In dieser Doktorarbeit wird die Frage untersucht, wie erfolgreich deep learning bei der Diagnostik von neurodegenerativen Erkrankungen unterstützen kann. In 5 experimentellen Studien wird die Anwendung von Convolutional Neural Networks (CNNs) auf Daten der Magnetresonanztomographie (MRT) untersucht. Ein Schwerpunkt wird dabei auf die Erklärbarkeit der eigentlich intransparenten Modelle gelegt. Mit Hilfe von Methoden der erklärbaren künstlichen Intelligenz (KI) werden Heatmaps erstellt, die die Relevanz einzelner Bildbereiche für das Modell darstellen. Die 5 Studien dieser Dissertation zeigen das Potenzial von CNNs zur Krankheitserkennung auf neurologischen MRT, insbesondere bei der Kombination mit Methoden der erklärbaren KI. Mehrere Herausforderungen wurden in den Studien aufgezeigt und Lösungsansätze in den Experimenten evaluiert. Über alle Studien hinweg haben CNNs gute Klassifikationsgenauigkeiten erzielt und konnten durch den Vergleich von Heatmaps zur klinischen Literatur validiert werden. Weiterhin wurde eine neue CNN Architektur entwickelt, spezialisiert auf die räumlichen Eigenschaften von Gehirn MRT Bildern.Deep learning and especially convolutional neural networks (CNNs) have a high potential of being implemented into clinical decision support software for tasks such as diagnosis and prediction of disease courses. This thesis has studied the application of CNNs on structural MRI data for diagnosing neurological diseases. Specifically, multiple sclerosis and Alzheimer’s disease were used as classification targets due to their high prevalence, data availability and apparent biomarkers in structural MRI data. The classification task is challenging since pathology can be highly individual and difficult for human experts to detect and due to small sample sizes, which are caused by the high acquisition cost and sensitivity of medical imaging data. A roadblock in adopting CNNs to clinical practice is their lack of interpretability. Therefore, after optimizing the machine learning models for predictive performance (e.g. balanced accuracy), we have employed explainability methods to study the reliability and validity of the trained models. The deep learning models achieved good predictive performance of over 87% balanced accuracy on all tasks and the explainability heatmaps showed coherence with known clinical biomarkers for both disorders. Explainability methods were compared quantitatively using brain atlases and shortcomings regarding their robustness were revealed. Further investigations showed clear benefits of transfer-learning and image registration on the model performance. Lastly, a new CNN layer type was introduced, which incorporates a prior on the spatial homogeneity of neuro-MRI data. CNNs excel when used on natural images which possess spatial heterogeneity, and even though MRI data and natural images share computational similarities, the composition and orientation of neuro-MRI is very distinct. The introduced patch-individual filter (PIF) layer breaks the assumption of spatial invariance of CNNs and reduces convergence time on different data sets without reducing predictive performance. The presented work highlights many challenges that CNNs for disease diagnosis face on MRI data and defines as well as tests strategies to overcome those