Artificial Intelligence in Clinical Decision Support : Challenges for Evaluating AI and Practical Implications

OBJECTIVES This paper draws attention to: i) key considerations for evaluating artificial intelligence (AI) enabled clinical decision support; and ii) challenges and practical implications of AI design, development, selection, use, and ongoing surveillance. METHOD A narrative review of existing research and evaluation approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Informatics and the European Federation for Medical Informatics (EFMI) Working Group for Assessment of Health Information Systems. RESULTS There is a rich history and tradition of evaluating AI in healthcare. While evaluators can learn from past efforts, and build on best practice evaluation frameworks and methodologies, questions remain about how to evaluate the safety and effectiveness of AI that dynamically harness vast amounts of genomic, biomarker, phenotype, electronic record, and care delivery data from across health systems. This paper first provides a historical perspective about the evaluation of AI in healthcare. It then examines key challenges of evaluating AI-enabled clinical decision support during design, development, selection, use, and ongoing surveillance. Practical aspects of evaluating AI in healthcare, including approaches to evaluation and indicators to monitor AI are also discussed. CONCLUSION Commitment to rigorous initial and ongoing evaluation will be critical to ensuring the safe and effective integration of AI in complex sociotechnical settings. Specific enhancements that are required for the new generation of AI-enabled clinical decision support will emerge through practical application

The Factors Associated With Nonuse of and Dissatisfaction With the National Patient Portal in Finland in the Era of COVID-19 : Population-Based Cross-sectional Survey

Funding Information: This study was supported by the Strategic Research Council at the Academy of Finland (projects 327145 and 327147), the Ministry of Social Affairs and Health (project 414919001), and NordForsk (project 100477). The authors wish to thank all the respondents for their contribution. Publisher Copyright: © Emma Kainiemi, Tuulikki Vehko, Maiju Kyytsönen, Iiris Hörhammer, Sari Kujala, Vesa Jormanainen, Tarja Heponiemi.Background: In the abnormal circumstances caused by the COVID-19 pandemic, patient portals have supported patient empowerment and engagement by providing patients with access to their health care documents and medical information. However, the potential benefits of patient portals cannot be utilized unless the patients accept and use the services. Disparities in the use of patient portals may exacerbate the already existing inequalities in health care access and health outcomes, possibly increasing the digital inequality in societies. Objective: The aim of this study is to examine the factors associated with nonuse of and dissatisfaction with the Finnish nationwide patient portal My Kanta Pages among the users of health care services during the COVID-19 outbreak. Several factors related to sociodemographic characteristics, health, and the use of health care services; experiences of guidance concerning electronic services; and digital skills and attitudes were evaluated. Methods: A national population survey was sent using stratified sampling to 13,200 Finnish residents who had reached the age of 20 years. Data were collected from September 2020 to February 2021 during the COVID-19 pandemic. Respondents who had used health care services and the internet for transactions or for searching for information in the past 12 months were included in the analyses. Bivariate logistic regression analyses were used to examine the adjusted associations of respondent characteristics with the nonuse of My Kanta Pages and dissatisfaction with the service. The inverse probability weighting (IPW) method was applied in all statistical analyses to correct for bias. Results: In total, 3919 (64.9%) of 6034 respondents were included in the study. Most respondents (3330/3919, 85.0%) used My Kanta Pages, and 2841 (85.3%) of them were satisfied. Nonusers (589/3919, 15%) were a minority among all respondents, and only 489 (14.7%) of the 3330 users were dissatisfied with the service. Especially patients without a long-term illness (odds ratio[OR] 2.14, 95% CI 1.48-3.10), those who were not referred to electronic health care services by a professional (OR 2.51, 95% CI 1.70-3.71), and those in need of guidance using online social and health care services (OR 2.26, 95% CI 1.41-3.65) were more likely nonusers of the patient portal. Perceptions of poor health (OR 2.10, 95% CI 1.51-2.93) and security concerns (OR 1.87, 95% CI 1.33-2.62) were associated with dissatisfaction with the service. Conclusions: Patients without long-term illnesses, those not referred to electronic health care services, and those in need of guidance on the use of online social and health care services seemed to be more likely nonusers of the Finnish nationwide patient portal. Moreover, poor health and security concerns appeared to be associated with dissatisfaction with the service. Interventions to promote referral to electronic health care services by professionals are needed. Attention should be targeted to information security of the service and promotion of the public's confidence in the protection of their confidential data.Peer reviewe