Git workflow for active learning - a development methodology proposal for data-centric AI projects

As soon as Artificial Intelligence (AI) projects grow from small feasibility studies to mature projects, developers and data scientists face new challenges, such as collaboration with other developers, versioning data, or traceability of model metrics and other resulting artifacts. This paper suggests a data-centric AI project with an Active Learning (AL) loop from a developer perspective and presents ”Git Workflow for AL”: A methodology proposal to guide teams on how to structure a project and solve implementation challenges. We introduce principles for data, code, as well as automation, and present a new branching workflow. The evaluation shows that the proposed method is an enabler for fulfilling established best practices

LIFEDATA - a framework for traceable active learning projects

Active Learning has become a popular method for iteratively improving data-intensive Artificial Intelligence models. However, it often presents a significant challenge when dealing with large volumes of volatile data in projects, as with an Active Learning loop. This paper introduces LIFEDATA, a Python- based framework designed to assist developers in implementing Active Learning projects focusing on traceability. It supports seamless tracking of all artifacts, from data selection and labeling to model interpretation, thus promoting transparency throughout the entire model learning process and enhancing error debugging efficiency while ensuring experiment reproducibility. To showcase its applicability, we present two life science use cases. Moreover, the paper proposes an algorithm that combines query strategies to demonstrate LIFEDATA’s ability to reduce data labeling effort

Automatic three-dimensional reconstruction of the oesophagus in achalasia patients undergoing POEM: an innovative approach for evaluating treatment outcomes

Background and aims Peroral endoscopic myotomy (POEM) is a standard treatment option for achalasia patients. Treatment response varies due to factors such as achalasia type, degree of dilatation, pressure and distensibility indices. We present an innovative approach for treatment response prediction based on an automatic three-dimensional (3-D) reconstruction of the tubular oesophagus (TE) and the lower oesophageal sphincter (LES) in patients undergoing POEM for achalasia. Methods A software was developed, integrating data from high-resolution manometry, timed barium oesophagogram and endoscopic images to automatically generate 3-D reconstructions of the TE and LES. Novel normative indices for TE (volume×pressure) and LES (volume/pressure) were automatically integrated, facilitating pre-POEM and post-POEM comparisons. Treatment response was evaluated by changes in volumetric and pressure indices for the TE and the LES before as well as 3 and 12 months after POEM. In addition, these values were compared with normal value indices of non-achalasia patients. Results 50 treatment-naive achalasia patients were enrolled prospectively. The mean TE index decreased significantly (p<0.0001) and the mean LES index increased significantly 3 months post-POEM (p<0.0001). In the 12-month follow-up, no further significant change of value indices between 3 and 12 months post-POEM was seen. 3 months post-POEM mean LES index approached the mean LES of the healthy control group (p=0.077). Conclusion 3-D reconstruction provides an interactive, dynamic visualisation of the oesophagus, serving as a comprehensive tool for evaluating treatment response. It may contribute to refining our approach to achalasia treatment and optimising treatment outcomes

Federated medical data - how much can deep learning models benefit? [Poster]

Many machine learning algorithms, like supervised Deep Learning, assume that Training Data are available in a single database. Federated Learning trains a model at each client locally, aggregates and share only the model, not the (patient-) data. Using the TensorFlow Federated Framework and data from the MIT-BIH Electrocardiogram database, we simulate two scenarios of an arrhythmia classifier (hospital and smartwatches as clients in a federated learning domain). The model quality is measured via the F1 score on a validation data set. We define the metrics Privacy Costs and Federated Benefit to evaluate the benefit of Federated Medical Data for the Deep Learning Models

Towards domain-specific explainable AI: model interpretation of a skin image classifier using a human approach

Machine Learning models have started to outperform medical experts in some classification tasks. Meanwhile, the question of how these classifiers produce certain results is attracting increasing research attention. Current interpretation methods provide a good starting point in investigating such questions, but they still massively lack the relation to the problem domain. In this work, we present how explanations of an AI system for skin image analysis can be made more domain-specific. We apply the synthesis of Local Interpretable Model-agnostic Explanations (LIME) with the ABCD-rule, a diagnostic approach of dermatologists, and present the results using a Deep Neural Network (DNN) based skin image classifier

Vertrauenswürdige KI in der Medizin - von effizienter Datenannotation bis intuitiver Modellerklärung

Die zunehmende Digitalisierung in der Medizin eröffnet durch die seit Jahren stark steigende Menge an verfügbaren Daten neue Möglichkeiten für medizinische KI-Anwendungen. So können beispielsweise Ärzt:innen bei der Diagnostik mit Clinical Decision Support Systemen unterstützt oder die Nachbeobachtung von Medizinprodukten zur Qualitätssicherung im Feld effizienter gestaltet werden. Um die Vorteile von medizinischen KI-Anwendungen optimal zu nutzen, stellen sich jedoch grundlegende Herausforderungen: So ist beispielsweise für eine sinnvolle Nutzung meist eine ausreichend große Menge annotierter, d.h. befundeter Daten notwendig, deren Annotation selbst sich insbesondere im medizinischen Bereich als sehr zeit- und kostenintensiv manifestiert. Weiter hängt die Akzeptanz und somit die Anwendung in der Breite stark von einer intuitiven Modellerklärung ab, da im medizinischen Umfeld Stakeholder:innen mit sehr unterschiedlicher Expertise aufeinandertreffen. Im Rahmen des Forschungsprojekts LIFEDATA werden diese zwei Problemstellungen für den Anwendungsfall von EKG-Daten betrachtet. Ein neuartiges Open-Source-Framework soll durch Aktives Lernen den Annotationsprozess effizient gestalten. Das wissenschaftliche Konzept kombiniert dazu mehrere Ansätze, um selbstständig Datenpunkte auszuwählen, die bei der anschließenden Annotation durch medizinische Expert:innen den größten Informationsgewinn mit sich bringen. Weiter werden unterschiedliche EKG-Interpretationsalgorithmen erforscht, um eine intuitive Modellerklärung zu erreichen und somit die Akzeptanz und das Vertrauen in KI-Vorhersagemodelle zu verbessern. Welchen Beitrag kann das Framework zur Entwicklung einer vertrauenswürdiger KI leisten? Die ganzheitliche Integration von Domänenexpert:innen ermöglicht neue Ansätze bei der Entwicklung von KI-gestützten Systemen in der Medizin. Im Rahmen des Workshops wollen wir mit Ihnen verschiedene Perspektiven entlang des Entwicklungsprozesses erörtern