Online Guide for Electronic Health Evaluation Approaches: Systematic Scoping Review and Concept Mapping Study

BACKGROUND: Despite the increase in use and high expectations of digital health solutions, scientific evidence about the effectiveness of electronic health (eHealth) and other aspects such as usability and accuracy is lagging behind. eHealth solutions are complex interventions, which require a wide array of evaluation approaches that are capable of answering the many different questions that arise during the consecutive study phases of eHealth development and implementation. However, evaluators seem to struggle in choosing suitable evaluation approaches in relation to a specific study phase. OBJECTIVE: The objective of this project was to provide a structured overview of the existing eHealth evaluation approaches, with the aim of assisting eHealth evaluators in selecting a suitable approach for evaluating their eHealth solution at a specific evaluation study phase. METHODS: Three consecutive steps were followed. Step 1 was a systematic scoping review, summarizing existing eHealth evaluation approaches. Step 2 was a concept mapping study asking eHealth researchers about approaches for evaluating eHealth. In step 3, the results of step 1 and 2 were used to develop an "eHealth evaluation cycle" and subsequently compose the online "eHealth methodology guide." RESULTS: The scoping review yielded 57 articles describing 50 unique evaluation approaches. The concept mapping study questioned 43 eHealth researchers, resulting in 48 unique approaches. After removing duplicates, 75 unique evaluation approaches remained. Thereafter, an "eHealth evaluation cycle" was developed, consisting of six evaluation study phases: conceptual and planning, design, development and usability, pilot (feasibility), effectiveness (impact), uptake (implementation), and all phases. Finally, the "eHealth methodology guide" was composed by assigning the 75 evaluation approaches to the specific study phases of the "eHealth evaluation cycle." CONCLUSIONS: Seventy-five unique evaluation approaches were found in the literature and suggested by eHealth researchers, which served as content for the online "eHealth methodology guide." By assisting evaluators in selecting a suitable evaluation approach in relation to a specific study phase of the "eHealth evaluation cycle," the guide aims to enhance the quality, safety, and successful long-term implementation of novel eHealth solutions

Risico's van het gebruik van eHealth-technologie : Een verkennende studie

Voor de risico's van eHealth-technologie is meer aandacht nodig. In de media, vakbladen, en wetenschappelijke tijdschriften is een overvloed aan informatie beschikbaar over de mogelijkheden van (mobiele) informatie- en communicatietechnologieën in de zorg. Voorbeelden zijn het 'op afstand' monitoren van diabetes in de thuiszorg, internethulp bij depressie, of digitale ondersteuning (PDA) bij stoppen met roken. Er is echter weinig bekend over de risico's van dergelijke technologieën. Als aanvulling op bestaande, veelal positieve, eHealth-evaluaties zouden de risico's daarom structureel en stelselmatig in kaart moeten worden gebracht. Dat is een voorwaarde om eHealth-technologie succesvol en veilig te kunnen gebruiken. Dit zijn de belangrijkste bevindingen van een verkennend literatuuronderzoek van het RIVM, uitgevoerd in opdracht van de Inspectie voor de Gezondheidszorg (IGZ). Voorbeelden van risico's In de wetenschappelijke literatuur zijn geen systematische studies (randomized controlled trials) gevonden die risico's van eHealth-technologie als hoofdonderwerp hebben. Wèl worden talloze, onbedoelde gevolgen van het gebruik van eHealth gemeld die raken aan de patiëntveiligheid of aan de kwaliteit van zorg. Risico's doen zich voor bij de gebruiker (patiënt), de technologie zelf en de organisatie die eHealth inzet. Voor de patiënt gaat het om gebruiksonvriendelijke technologie, onnadenkend gebruik ervan of beperkte toegang ertoe. Patiënten kunnen hierdoor vastlopen, gedemotiveerd raken of de therapie staken. Hierdoor kan de behandeling niet het beoogde effect hebben of de klacht zelfs verergeren. Bij de technologie komen de risico's vooral voort uit slecht functionerende apparaten. Op organisatieniveau ontstaan risico's wanneer eHealth onvoldoende is ingebed in het zorgproces. De aangetroffen bewijzen voor de risico's zijn hoofdzakelijk anecdotisch van aard. Over de omvang ervan is weinig bekend. Zowel onderzoek van online 'grijze' bronnen, zoals databases en websites, als gezaghebbende publicaties laten deze uitkomsten zien. Risicomanagement en meldsysteem Omdat in Nederland steeds meer eHealth-technologie wordt gebruikt, is het belangrijk dat in de gezondheidszorg bestaande procedures voor risicomanagement ook voor eHealth worden ingezet. Een betrouwbaar systeem waar incidenten structureel kunnen worden gemeld, geïdentificeerd, gedocumenteerd en gemonitord zou daarbij helpen.More awareness is needed about the risks of e-Health technology. While information regarding its potential is abundant, the risks associated with the use of information (including mobile) and communication technology in health care have scarcely been addressed. In order to implement e-Health technology successfully and safely, the evaluation of their benefits should be integrated into and complemented with systematic risk assessment. This is the main recommendation resulting from an exploratory literature study that was performed at the request of the Dutch Health Care Inspectorate. A review of scientific literature identified no systematic studies (randomized controlled trials) that directly investigated the risks of e-Health technology. However, many unintended, 'secondary', outcomes have been reported that indicate risks for patient safety or quality of care at the level of the technology, the end-user (patient, professional) or the organization. They vary from high time consumption, adverse effects, usability problems, limited server access and malfunctioning devices due to improper use or financial issues. Similar outcomes were found through searching 'grey' sources accessed through the internet. From the combined scientific and grey sources, we found anecdotal evidence for a wide variety of risks in e-Health, of which the magnitude is largely unknown. Confirmation of these findings was obtained from several other recent, authoritative reports. E-Health interventions are being increasingly used in Dutch health care. It is, therefore, important that tools currently used for risk management are applied to e-Health as well. A reliable system to report, identify, document and monitor risks would help to increase transparency in this field.IG

Business Models for Sustained EHealth Implementation: Lesson from Two Continents

IngenieursweseBedryfsingenieurswesePlease help us populate SUNScholar with the post print version of this article. It can be e-mailed to: [email protected]