Datengestützte Ableitung von rheumatologischen Krankheitsmerkmalen aus medizinischen Bildern

Rheumatologists have various tools at hand that allow characterizing the diseases of their patients. A multitude of those tools requires human annotation, which in some cases is time-intensive and error-prone. This poses a unique challenge, that can be addressed by modern pattern recognition advancements. In this work, a brief overview of rheumatic diseases was provided. Thereafter, imaging methods commonly used in rheumatology were described. Lastly, the fundamentals of deep learning and neural networks were explained, which build the foundation of the methods applied during the works, that are part of this thesis. As a first step to reducing time and increasing consistency, the workflow to estimate the density of patient bones using computed tomography (CT) was addressed. Tracking the bone density of patients over time provides necessary information for clinicians to adequately adapt patient therapy. As this requires manual annotation of the bone in CT images, it is time-intensive and error-prone. By using a segmentation network to assist humans in this step, the clinical workflow could be considerably accelerated while still generating high-quality results. The second step of this thesis regarded the classification of patients based on magnetic resonance images (MRI) and CT scans to different groups of diseases. As rheumatic diseases specifically target the soft tissue of patients and the bone structures, those imaging techniques are of great utility in the rheumatological practice. However, disease mechanisms are still not fully understood. Thus, in this thesis, the ability to classify patients into the different disease entities of rheumatic diseases using the information present only in MRI and CT was investigated. Based on both imaging techniques, with the help of neural networks, classifiers with solid performance were trained. Interestingly, certain features in the shape of the bones, that have been described in previous works, influenced the predictions of the network to a high degree. In the last work of this thesis, the classification and prediction of the severity of nail psoriasis, a condition that occurs during severe disease stages of psoriasis, was studied. Currently, the grading of nail psoriasis is mostly based on the nail psoriasis severity index (NAPSI). However, even though this can help clinicians in their decision process, it is not clinically used due to the high time-intensiveness of the grading process. By acquiring a dataset of hand photos and the corresponding labels, a neural network was trained to predict the NAPSI automatically. By providing access to the complete system online, the applicability of the approach for other clinicians was greatly improved and enables for the first time the usage of the NAPSI not just in clinical studies but also in clinical practice. At the end of this thesis, the limitations of the proposed works were discussed. A common aspect across all works was the necessity for a greater amount of training data. Specifically, the disease entity classification works reached a solid performance, but would likely benefit greatly from an increasing amount of training data. The nail psoriasis classification work reached very good performance, but future work, that would increase utility for patients would be the development of a mobile application that runs the proposed method and tracks the estimated disease severity over time. Briefly, rheumatic diseases and their representation in CT, MRI, and hand photos were investigated in this thesis. With the help of neural networks, time-consuming workflows could be considerably accelerated and new insights into the disease mechanics were gained.Kliniker in der Rheumatologie verfügen über verschiedene Instrumente, mit denen sie die Krankheiten ihrer Patienten charakterisieren können. Eine Vielzahl dieser Instrumente erfordert eine menschliche Annotation, die in einigen Fällen zeitintensiv und fehleranfällig ist. Dies stellt eine einzigartige Herausforderung dar, die durch moderne Fortschritte in der Mustererkennung bewältigt werden kann. In dieser Arbeit wurde zunächst ein kurzer Überblick über rheumatische Erkrankungen gegeben. Danach wurden die in der Rheumatologie üblichen bildgebenden Verfahren beschrieben. Schließlich wurden die Grundlagen des Deep Learning und der neuronalen Netze erläutert, die die Grundlage für die in dieser Arbeit verwendeten Methoden bilden. Als erster Schritt zur Zeitersparnis und Erhöhung der Konsistenz wurde der Arbeitsablauf zur Schätzung der Knochendichte von Patienten mit Hilfe der Computertomographie (CT) behandelt. Die Verfolgung der Knochendichte von Patienten im Laufe der Zeit liefert den Klinikern die notwendigen Informationen, um die Therapie der Patienten angemessen anzupassen. Da dies eine manuelle Beschriftung des Knochens in CT-Bildern erfordert, ist dies zeitintensiv und fehleranfällig. Durch den Einsatz eines Segmentierungsnetzes, das den Menschen bei diesem Schritt unterstützt, könnte der klinische Arbeitsablauf erheblich beschleunigt werden, während gleichzeitig qualitativ hochwertige Ergebnisse erzielt werden. Der zweite Schritt dieser Arbeit betraf die Klassifizierung von Patienten anhand von Magnetresonanzbildern (MRT) und CT-Scans in verschiedene Krankheitsgruppen. Da rheumatische Erkrankungen speziell die Weichteile der Patienten und die Knochenstrukturen betreffen, sind diese bildgebenden Verfahren von großem Nutzen für die Rheumatologie. Die Krankheitsmechanismen sind jedoch noch immer nicht vollständig verstanden. In dieser Arbeit wurde daher untersucht, inwieweit sich Patienten anhand der Informationen aus MRT und CT in die verschiedenen rheumatischen Krankheitsbilder einordnen lassen. Auf der Grundlage beider bildgebender Verfahren wurden mit Hilfe neuronaler Netze Klassifikatoren mit solider Leistung trainiert. Interessanterweise beeinflussten bestimmte Merkmale in der Form der Knochen, die in früheren Arbeiten beschrieben worden waren, die Vorhersagen des Netzes in hohem Maße. In der letzten Arbeit dieser Dissertation wurde die Klassifizierung und Vorhersage des Schweregrads der Nagelpsoriasis untersucht, die bei schweren Krankheitsstadien der Psoriasis auftritt. Derzeit basiert die Einstufung der Nagelpsoriasis meist auf dem Nagelpsoriasis-Schweregradindex (NAPSI). Obwohl dieser Index Ärzten bei der Entscheidungsfindung helfen kann, wird er aufgrund des hohen Zeitaufwands für die Einstufung nicht klinisch eingesetzt. Durch die Erfassung eines Datensatzes von Handfotos und der entsprechenden Beschriftungen wurde ein neuronales Netz zur automatischen Vorhersage des NAPSI trainiert. Durch den Online-Zugriff auf das komplette System wurde die Anwendbarkeit des Ansatzes für andere Kliniker erheblich verbessert und ermöglicht erstmals den Einsatz des NAPSI nicht nur in klinischen Studien, sondern auch in der klinischen Praxis. Am Ende dieser Arbeit wurden die Grenzen der präsentierten Arbeiten diskutiert. Ein gemeinsamer Aspekt aller Arbeiten war die Notwendigkeit einer größeren Menge an Trainingsdaten. Insbesondere die Arbeiten zur Klassifizierung von Krankheitsgruppen erreichten eine solide Leistung, würden aber wahrscheinlich stark von einer größeren Menge an Trainingsdaten profitieren. Die Arbeit zur Klassifizierung der Nagelpsoriasis erreichte eine sehr gute Leistung, aber zukünftige Arbeiten, die den Nutzen für die Patienten erhöhen würden, wären die Entwicklung einer mobilen Anwendung, die die vorgeschlagene Methode ausführt und den geschätzten Krankheitsschweregrad im Laufe der Zeit verfolgt. Im Rahmen dieser Arbeit wurden rheumatische Erkrankungen und ihre Darstellung in CT-, MRT- und Handfotos untersucht. Mit Hilfe von neuronalen Netzen konnten zeitaufwändige Arbeitsabläufe erheblich beschleunigt und neue Erkenntnisse über die Krankheitsmechanismen gewonnen werden

Generation of Anonymous Chest Radiographs Using Latent Diffusion Models for Training Thoracic Abnormality Classification Systems

The availability of large-scale chest X-ray datasets is a requirement for developing well-performing deep learning-based algorithms in thoracic abnormality detection and classification. However, biometric identifiers in chest radiographs hinder the public sharing of such data for research purposes due to the risk of patient re-identification. To counteract this issue, synthetic data generation offers a solution for anonymizing medical images. This work employs a latent diffusion model to synthesize an anonymous chest X-ray dataset of high-quality class-conditional images. We propose a privacy-enhancing sampling strategy to ensure the non-transference of biometric information during the image generation process. The quality of the generated images and the feasibility of serving as exclusive training data are evaluated on a thoracic abnormality classification task. Compared to a real classifier, we achieve competitive results with a performance gap of only 3.5% in the area under the receiver operating characteristic curve.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

A gradient-based approach to fast and accurate head motion compensation in cone-beam CT

Cone-beam computed tomography (CBCT) systems, with their portability, present a promising avenue for direct point-of-care medical imaging, particularly in critical scenarios such as acute stroke assessment. However, the integration of CBCT into clinical workflows faces challenges, primarily linked to long scan duration resulting in patient motion during scanning and leading to image quality degradation in the reconstructed volumes. This paper introduces a novel approach to CBCT motion estimation using a gradient-based optimization algorithm, which leverages generalized derivatives of the backprojection operator for cone-beam CT geometries. Building on that, a fully differentiable target function is formulated which grades the quality of the current motion estimate in reconstruction space. We drastically accelerate motion estimation yielding a 19-fold speed-up compared to existing methods. Additionally, we investigate the architecture of networks used for quality metric regression and propose predicting voxel-wise quality maps, favoring autoencoder-like architectures over contracting ones. This modification improves gradient flow, leading to more accurate motion estimation. The presented method is evaluated through realistic experiments on head anatomy. It achieves a reduction in reprojection error from an initial average of 3mm to 0.61mm after motion compensation and consistently demonstrates superior performance compared to existing approaches. The analytic Jacobian for the backprojection operation, which is at the core of the proposed method, is made publicly available. In summary, this paper contributes to the advancement of CBCT integration into clinical workflows by proposing a robust motion estimation approach that enhances efficiency and accuracy, addressing critical challenges in time-sensitive scenarios.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Deep Learning-Based Classification of Inflammatory Arthritis by Identification of Joint Shape Patterns—How Neural Networks Can Tell Us Where to “Deep Dive” Clinically

Objective: We investigated whether a neural network based on the shape of joints can differentiate between rheumatoid arthritis (RA), psoriatic arthritis (PsA), and healthy controls (HC), which class patients with undifferentiated arthritis (UA) are assigned to, and whether this neural network is able to identify disease-specific regions in joints. Methods We trained a novel neural network on 3D articular bone shapes of hand joints of RA and PsA patients as well as HC. Bone shapes were created from high-resolution peripheral-computed-tomography (HR-pQCT) data of the second metacarpal bone head. Heat maps of critical spots were generated using GradCAM. After training, we fed shape patterns of UA into the neural network to classify them into RA, PsA, or HC. Results Hand bone shapes from 932 HR-pQCT scans of 617 patients were available. The network could differentiate the classes with an area-under-receiver-operator-curve of 82% for HC, 75% for RA, and 68% for PsA. Heat maps identified anatomical regions such as bare area or ligament attachments prone to erosions and bony spurs. When feeding UA data into the neural network, 86% were classified as “RA,” 11% as “PsA,” and 3% as “HC” based on the joint shape. Conclusion We investigated neural networks to differentiate the shape of joints of RA, PsA, and HC and extracted disease-specific characteristics as heat maps on 3D joint shapes that can be utilized in clinical routine examination using ultrasound. Finally, unspecific diseases such as UA could be grouped using the trained network based on joint shape

Deep learning methods allow fully automated segmentation of metacarpal bones to quantify volumetric bone mineral density

Arthritis patients develop hand bone loss, which leads to destruction and functional impairment of the affected joints. High resolution peripheral quantitative computed tomography (HR-pQCT) allows the quantification of volumetric bone mineral density (vBMD) and bone microstructure in vivo with an isotropic voxel size of 82 micrometres. However, image-processing to obtain bone characteristics is a time-consuming process as it requires semi-automatic segmentation of the bone. In this work, a fully automatic vBMD measurement pipeline for the metacarpal (MC) bone using deep learning methods is introduced. Based on a dataset of HR-pQCT volumes with MC measurements for 541 patients with arthritis, a segmentation network is trained. The best network achieves an intersection over union as high as 0.94 and a Dice similarity coefficient of 0.97 while taking only 33 s to process a whole patient yielding a speedup between 2.5 and 4.0 for the whole workflow. Strong correlation between the vBMD measurements of the expert and of the automatic pipeline are achieved for the average bone density with 0.999 (Pearson) and 0.996 (Spearman’s rank) with p<0.001 for all correlations. A qualitative assessment of the network predictions and the manual annotations yields a 65.9% probability that the expert favors the network predictions. Further, the steps to integrate the pipeline into the clinical workflow are shown. In order to make these workflow improvements available to others, we openly share the code of this work

Image quality assessment using deep learning in high b-value diffusion-weighted breast MRI

AbstractThe objective of this IRB approved retrospective study was to apply deep learning to identify magnetic resonance imaging (MRI) artifacts on maximum intensity projections (MIP) of the breast, which were derived from diffusion weighted imaging (DWI) protocols. The dataset consisted of 1309 clinically indicated breast MRI examinations of 1158 individuals (median age [IQR]: 50 years [16.75 years]) acquired between March 2017 and June 2020, in which a DWI sequence with a high b-value equal to 1500 s/mm2 was acquired. From these, 2D MIP images were computed and the left and right breast were cropped out as regions of interest (ROI). The presence of MRI image artifacts on the ROIs was rated by three independent observers. Artifact prevalence in the dataset was 37% (961 out of 2618 images). A DenseNet was trained with a fivefold cross-validation to identify artifacts on these images. In an independent holdout test dataset (n = 350 images) artifacts were detected by the neural network with an area under the precision-recall curve of 0.921 and a positive predictive value of 0.981. Our results show that a deep learning algorithm is capable to identify MRI artifacts in breast DWI-derived MIPs, which could help to improve quality assurance approaches for DWI sequences of breast examinations in the future.</jats:p

DeepNAPSI multi-reader nail psoriasis prediction using deep learning

Abstract Nail psoriasis occurs in about every second psoriasis patient. Both, finger and toe nails can be affected and also severely destroyed. Furthermore, nail psoriasis is associated with a more severe course of the disease and the development of psoriatic arthritis. User independent quantification of nail psoriasis, however, is challenging due to the heterogeneous involvement of matrix and nail bed. For this purpose, the nail psoriasis severity index (NAPSI) has been developed. Experts grade pathological changes of each nail of the patient leading to a maximum score of 80 for all nails of the hands. Application in clinical practice, however, is not feasible due to the time-intensive manual grading process especially if more nails are involved. In this work we aimed to automatically quantify the modified NAPSI (mNAPSI) of patients using neuronal networks retrospectively. First, we performed photographs of the hands of patients with psoriasis, psoriatic arthritis, and rheumatoid arthritis. In a second step, we collected and annotated the mNAPSI scores of 1154 nail photos. Followingly, we extracted each nail automatically using an automatic key-point-detection system. The agreement among the three readers with a Cronbach’s alpha of 94% was very high. With the nail images individually available, we trained a transformer-based neural network (BEiT) to predict the mNAPSI score. The network reached a good performance with an area-under-receiver-operator-curve of 88% and an area-under precision-recall-curve (PR-AUC) of 63%. We could compare the results with the human annotations and achieved a very high positive Pearson correlation of 90% by aggregating the predictions of the network on the test set to the patient-level. Lastly, we provided open access to the whole system enabling the use of the mNAPSI in clinical practice

Imaging in inflammatory arthritis: progress towards precision medicine

International audienceImaging techniques such as ultrasonography and MRI have gained ground in the diagnosis and management of inflammatory arthritis, as these imaging modalities allow a sensitive assessment of musculoskeletal inflammation and damage. However, these techniques cannot discriminate between disease subsets and are currently unable to deliver an accurate prediction of disease progression and therapeutic response in individual patients. This major shortcoming of today’s technology hinders a targeted and personalized patient management approach. Technological advances in the areas of high-resolution imaging (for example, high-resolution peripheral quantitative computed tomography and ultra-high field MRI), functional and molecular-based imaging (such as chemical exchange saturation transfer MRI, positron emission tomography, fluorescence optical imaging, optoacoustic imaging and contrast-enhanced ultrasonography) and artificial intelligence-based data analysis could help to tackle these challenges. These new imaging approaches offer detailed anatomical delineation and an in vivo and non-invasive evaluation of the immunometabolic status of inflammatory reactions, thereby facilitating an in-depth characterization of inflammation. By means of these developments, the aim of earlier diagnosis, enhanced monitoring and, ultimately, a personalized treatment strategy looms closer