Stakeholder Perspectives on Clinical Decision Support Tools to Inform Clinical Artificial Intelligence Implementation: Protocol for a Framework Synthesis for Qualitative Evidence

BACKGROUND: Quantitative systematic reviews have identified clinical artificial intelligence (AI)-enabled tools with adequate performance for real-world implementation. To our knowledge, no published report or protocol synthesizes the full breadth of stakeholder perspectives. The absence of such a rigorous foundation perpetuates the "AI chasm," which continues to delay patient benefit. OBJECTIVE: The aim of this research is to synthesize stakeholder perspectives of computerized clinical decision support tools in any health care setting. Synthesized findings will inform future research and the implementation of AI into health care services. METHODS: The search strategy will use MEDLINE (Ovid), Scopus, CINAHL (EBSCO), ACM Digital Library, and Science Citation Index (Web of Science). Following deduplication, title, abstract, and full text screening will be performed by 2 independent reviewers with a third topic expert arbitrating. The quality of included studies will be appraised to support interpretation. Best-fit framework synthesis will be performed, with line-by-line coding completed by 2 independent reviewers. Where appropriate, these findings will be assigned to 1 of 22 a priori themes defined by the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability framework. New domains will be inductively generated for outlying findings. The placement of findings within themes will be reviewed iteratively by a study advisory group including patient and lay representatives. RESULTS: Study registration was obtained from PROSPERO (CRD42021256005) in May 2021. Final searches were executed in April, and screening is ongoing at the time of writing. Full text data analysis is due to be completed in October 2021. We anticipate that the study will be submitted for open-access publication in late 2021. CONCLUSIONS: This paper describes the protocol for a qualitative evidence synthesis aiming to define barriers and facilitators to the implementation of computerized clinical decision support tools from all relevant stakeholders. The results of this study are intended to expedite the delivery of patient benefit from AI-enabled clinical tools. TRIAL REGISTRATION: PROSPERO CRD42021256005; https://tinyurl.com/r4x3thvp. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/33145

Stakeholder Perspectives on Clinical Decision Support Tools to Inform Clinical Artificial Intelligence Implementation: Protocol for a Framework Synthesis for Qualitative Evidence

Quantitative systematic reviews have identified clinical artificial intelligence (AI)-enabled tools with adequate performance for real-world implementation. To our knowledge, no published report or protocol synthesizes the full breadth of stakeholder perspectives. The absence of such a rigorous foundation perpetuates the "AI chasm," which continues to delay patient benefit. The aim of this research is to synthesize stakeholder perspectives of computerized clinical decision support tools in any health care setting. Synthesized findings will inform future research and the implementation of AI into health care services. The search strategy will use MEDLINE (Ovid), Scopus, CINAHL (EBSCO), ACM Digital Library, and Science Citation Index (Web of Science). Following deduplication, title, abstract, and full text screening will be performed by 2 independent reviewers with a third topic expert arbitrating. The quality of included studies will be appraised to support interpretation. Best-fit framework synthesis will be performed, with line-by-line coding completed by 2 independent reviewers. Where appropriate, these findings will be assigned to 1 of 22 a priori themes defined by the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability framework. New domains will be inductively generated for outlying findings. The placement of findings within themes will be reviewed iteratively by a study advisory group including patient and lay representatives. Study registration was obtained from PROSPERO (CRD42021256005) in May 2021. Final searches were executed in April, and screening is ongoing at the time of writing. Full text data analysis is due to be completed in October 2021. We anticipate that the study will be submitted for open-access publication in late 2021. This paper describes the protocol for a qualitative evidence synthesis aiming to define barriers and facilitators to the implementation of computerized clinical decision support tools from all relevant stakeholders. The results of this study are intended to expedite the delivery of patient benefit from AI-enabled clinical tools. PROSPERO CRD42021256005; https://tinyurl.com/r4x3thvp. DERR1-10.2196/33145. [Abstract copyright: ©Mohaimen Al-Zubaidy, HD Jeffry Hogg, Gregory Maniatopoulos, James Talks, Marion Dawn Teare, Pearse A Keane, Fiona R Beyer. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 01.04.2022.

Stakeholder Perspectives of Clinical Artificial Intelligence Implementation: Systematic Review of Qualitative Evidence

Background: The rhetoric surrounding clinical artificial intelligence (AI) often exaggerates its effect on real-world care. Limited understanding of the factors that influence its implementation can perpetuate this. Objective: In this qualitative systematic review, we aimed to identify key stakeholders, consolidate their perspectives on clinical AI implementation, and characterize the evidence gaps that future qualitative research should target. Methods: Ovid-MEDLINE, EBSCO-CINAHL, ACM Digital Library, Science Citation Index-Web of Science, and Scopus were searched for primary qualitative studies on individuals’ perspectives on any application of clinical AI worldwide (January 2014-April 2021). The definition of clinical AI includes both rule-based and machine learning–enabled or non–rule-based decision support tools. The language of the reports was not an exclusion criterion. Two independent reviewers performed title, abstract, and full-text screening with a third arbiter of disagreement. Two reviewers assigned the Joanna Briggs Institute 10-point checklist for qualitative research scores for each study. A single reviewer extracted free-text data relevant to clinical AI implementation, noting the stakeholders contributing to each excerpt. The best-fit framework synthesis used the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework. To validate the data and improve accessibility, coauthors representing each emergent stakeholder group codeveloped summaries of the factors most relevant to their respective groups. Results: The initial search yielded 4437 deduplicated articles, with 111 (2.5%) eligible for inclusion (median Joanna Briggs Institute 10-point checklist for qualitative research score, 8/10). Five distinct stakeholder groups emerged from the data: health care professionals (HCPs), patients, carers and other members of the public, developers, health care managers and leaders, and regulators or policy makers, contributing 1204 (70%), 196 (11.4%), 133 (7.7%), 129 (7.5%), and 59 (3.4%) of 1721 eligible excerpts, respectively. All stakeholder groups independently identified a breadth of implementation factors, with each producing data that were mapped between 17 and 24 of the 27 adapted Nonadoption, Abandonment, Scale-up, Spread, and Sustainability subdomains. Most of the factors that stakeholders found influential in the implementation of rule-based clinical AI also applied to non–rule-based clinical AI, with the exception of intellectual property, regulation, and sociocultural attitudes. Conclusions: Clinical AI implementation is influenced by many interdependent factors, which are in turn influenced by at least 5 distinct stakeholder groups. This implies that effective research and practice of clinical AI implementation should consider multiple stakeholder perspectives. The current underrepresentation of perspectives from stakeholders other than HCPs in the literature may limit the anticipation and management of the factors that influence successful clinical AI implementation. Future research should not only widen the representation of tools and contexts in qualitative research but also specifically investigate the perspectives of all stakeholder HCPs and emerging aspects of non–rule-based clinical AI implementation. Trial Registration: PROSPERO (International Prospective Register of Systematic Reviews) CRD42021256005; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=25600

Experimental data sets.

a<p>: <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0028636#pone.0028636-Rutter1" target="_blank">[32]</a>.</p>b<p>: <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0028636#pone.0028636-Mei1" target="_blank">[33]</a>.</p

Observed allelic expression ratios measured at rs5854, a transcribed polymorphism at the 3′ end of the MMP1 gene grouped according to the genotype for rs11292517, a polymorphism in the promoter region of the gene.

<p>Observed allelic expression ratios measured at rs5854, a transcribed polymorphism at the 3′ end of the MMP1 gene grouped according to the genotype for rs11292517, a polymorphism in the promoter region of the gene.</p

Summary of tests used.

a<p>: Pattern of disequilibrium, as represented in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0028636#pone-0028636-g003" target="_blank">Figure 3</a>, for which the test is most appropriate.</p>b<p>: Assumes that given the genotype AERs follow a log normal distribution.</p

A visualisation of different approaches for testing an association between allelic expression and a biallelic polymorphism.

<p>The distribution of allelic expression ratios across a population is represented. We consider here two polymorphisms: a transcribed one, with alleles m and M, used to measure allelic expression; and a <i>cis</i> acting one with alleles c and C. Each elongated diamond represents the mean and the spread of the AEI measurements by specific genotypes. A) The general situation. B) Perfect disequilibrium (D′ = 1, R² = 1) between the <i>cis</i> acting and the transcribed polymorphism, only two distinct haplotypes exist. C) Complete disequilibrium (D′ = 1, R²<1), only three distinct haplotypes exist. D) Situation when the phase between alleles at both sites is known.</p

Additional sites affecting the expression in <i>cis</i>.

<p>The graph represents the influence of the number of sites upon the power to detect the SNP with the largest effect. All polymorphisms are assumed to be in linkage disequilibrium. Simulation parameters: .</p

Power comparisons when data are simulated assuming a log normal distribution for the allelic expression ratios.

<p>For all simulations: . Panel A: Effect of sample size assuming transcribed and <i>cis</i> acting polymorphism are in linkage equilibrium (Simulation parameters: and ). Panel B: The influence of the extent of disequilibrium (Simulation parameters: ); Panels C and D: The influence of effect size (Panel C for and panel D for other simulation parameters ). Panels E and F: The influence of allele frequency for the transcribed polymorphism (Panel E for and panel F for , othersimulation parameters: ). Panels G and H: The influence of allele frequency for the <i>cis</i> acting variant (Panel G for and Panel H for ,other parameters: ).</p

Diagrammatic representation of the effect of a <i>cis</i> acting polymorphism upon allelic expression.

<p>Depicted is the situation for an individual who is heterozygous for a <i>cis</i> acting polymorphism with alleles A and C and is also heterozygous for a polymorphism within the affected transcript.</p