Perspectives and Best Practices for Artificial Intelligence and Continuously Learning Systems in Healthcare

Goals of this paper Healthcare is often a late adopter when it comes to new techniques and technologies; this works to our advantage in the development of this paper as we relied on lessons learned from CLS in other industries to help guide the content of this paper. Appendix V includes a number of example use cases of AI in Healthcare and other industries. This paper focuses on identifying unique attributes, constraints and potential best practices towards what might represent “good” development for Continuously Learning Systems (CLS) AI systems with applications ranging from pharmaceutical applications for new drug development and research to AI enabled smart medical devices. It should be noted that although the emphasis of this paper is on CLS, some of these issues are common to all AI products in healthcare. Additionally, there are certain topics that should be considered when developing CLS for healthcare, but they are outside of the scope of this paper. These topics will be briefly touched upon, but will not be explored in depth. Some examples include: Human Factors – this is a concern in the development of any product – what are the unique usability challenges that arise when collecting data and presenting the results? Previous efforts at generating automated alerts have often created problems (e.g. alert fatigue.) CyberSecurity and Privacy – holding a massive amount of patient data is an attractive target for hackers, what steps should be taken to protect data from misuse? How does the European Union’s General Data Protection Regulation (GDPR) impact the use of patient data? Legal liability – if a CLS system recommends action that is then reviewed and approved by a doctor, where does the liability lie if the patient is negatively affected? Regulatory considerations – medical devices are subject to regulatory oversight around the world; in fact, if a product is considered a medical device depends on what country you are in. AI provides an interesting challenge to traditional regulatory models. Additionally, some organizations like the FTC regulate non-medical devices. This paper is not intended to be a standard, nor is this paper trying to advocate for one and only one method of developing, verifying, and validating CLS systems – this paper highlights best practices from other industries and suggests adaptation of those processes for healthcare. This paper is also not intended to evaluate existing or developing regulatory, legal, ethical, or social consequences of CLS systems. This is a rapidly evolving subject with many companies, and now some countries, establishing their own AI Principles or Code of Conduct which emphasize legal and ethical considerations including goals and principles of fairness, reliability and safety, transparency around how the results of these learning systems are explained to the people using those systems5 . The intended audience of this paper are Developers, Researchers, Quality Assurance and Validation personnel, Business Managers and Regulators across both Medical Device and Pharmaceutical industries that would like to learn more about CLS best practices, and CLS practitioners wanting to learn more about medical device software development

Validation of artificial intelligence containing products across the regulated healthcare industries

Purpose: The introduction of artificial intelligence / machine learning (AI/ML) products to the regulated fields of pharmaceutical research and development (R&D) and drug manufacture, and medical devices (MD) and in-vitro diagnostics (IVD), poses new regulatory problems: a lack of a common terminology and understanding leads to confusion, delays and product failures. Validation as a key step in product development, common to each of these sectors including computerized systems and AI/ML development, offers an opportune point of comparison for aligning people and processes for cross-sectoral product development. Methods: A comparative approach, built upon workshops and a subsequent written sequence of exchanges, summarized in a look-up table suitable for mixed-teams work. Results: 1. A bottom-up, definitions led, approach which leads to a distinction between broad vs narrow validation, and their relationship to regulatory regimes. 2. Common basis introduction to the primary methodologies for AI-containing software validation. 3. Pharmaceutical drug development and MD/IVD specific perspectives on compliant AI software development, as a basis for collaboration. Conclusions: Alignment of the terms and methodologies used in validation of software products containing artificial intelligence / machine learning (AI/ML) components across the regulated industries of human health is a vital first step in streamlining processes and improving workflows.Comment: 33 pages, 3 figures, 1 table. A version of this preprint with messy formatting is available from https://www.researchsquare.com/article/rs-2153749/v

ArCo: the Italian Cultural Heritage Knowledge Graph

ArCo is the Italian Cultural Heritage knowledge graph, consisting of a network of seven vocabularies and 169 million triples about 820 thousand cultural entities. It is distributed jointly with a SPARQL endpoint, a software for converting catalogue records to RDF, and a rich suite of documentation material (testing, evaluation, how-to, examples, etc.). ArCo is based on the official General Catalogue of the Italian Ministry of Cultural Heritage and Activities (MiBAC) - and its associated encoding regulations - which collects and validates the catalogue records of (ideally) all Italian Cultural Heritage properties (excluding libraries and archives), contributed by CH administrators from all over Italy. We present its structure, design methods and tools, its growing community, and delineate its importance, quality, and impact

Holistic analysis of the effectiveness of a software engineering teaching approach

To provide the best training in software engineering, several approaches and strategies are carried out. Some of them are more theoretical, learned through books and manuals, while others have a practical focus and often done in collaboration with companies. In this paper, we share an approach based on a balanced mix to foster the assimilation of knowledge, the approximation with what is done in software companies and student motivation. Two questionnaires were also carried out, one involving students, who had successfully completed the subject in past academic years (some had already graduated, and others are still students), and other questionnaire involving companies, in the field of software development, which employ students from our school. The analysis of the perspectives of the different stakeholders allows an overall and holistic) view, and a general understanding, of the effectiveness of the software engineering teaching approach. We analyse the results of the questionnaires and share some of the experiences and lessons learned.info:eu-repo/semantics/publishedVersio

Software engineering for AI-based systems: A survey

AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image-, speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state-of-the-art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.This work has been partially funded by the “Beatriz Galindo” Spanish Program BEAGAL18/00064 and by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00)Peer ReviewedPostprint (author's final draft