A New Standards-based Grammar for Linking Aggregate Datasets

The theme of this session is the linking and cross-referencing of disparate aggregate datasets that need to be combined for pruporses of reporting and/or analysis. The session leverages, as a global case study, the US Government's President's Emergency Plan for AIDS Relief (PEPFAR) programme. PEPFAR is a $7 billion per year programme supporting the delivery of HIV-related services, medicines, and commodities in 58 low and middle-income countries (www.pepfar.gov). PEPFAR has an immense datastore of monitoring, evaluation and reporting (MER) indicators that have been collected from all its supported countries over the course of its 15 years of operations. The goal of the session is to describe for attendees a newly-developed, standards-based grammar for describing interoperable aggregate data exchange and the message schemas needed to support it. The session facilitators are the primary authors of this new standard. Using the PEPFAR case study as a working example, the session explores how disparate HIV data elements and indicators from PEPFAR-supported countries are cross-referenced to each other and collected into a single central datastore to support analysis, management and reporting across the global programme. The specific HIV example will be elaborated upon to illustrate generalizable techniques that can be applied to linking aggregate datasets in other use cases (e.g. reporting to the annual WHO global health observatory, multiple provinces reporting to a federal health data institute, etc.). The session will be facilitated by Xenophon Santas and James Kariuki of the US CDC, Bob Jolliffe of the University of Oslo's Health Information Systems Programme (HISP) and Derek Ritz of ecGroup Inc (a Canadian health informatics consultancy). All four facilitators are members of the Quality, Research and Publich Health (QRPH) technical committee of the international digital health standards body, Integrating the Healthcare Enterprise (IHE; www.ihe.net). The session's content and examples will leverage the facilitators' first-hand experience working on HIV-related projects in low and middle-income countries (e.g. South Africa, Rwanda, Kenya, Malawi, Zimbabwe, Uganda, Sierre Leone, Vietnam, the Philippines and elsewhere). It is intended that the session will be conducted using an interactive workshop style. Attendees who wish it will have an opportunity to engage in participative (hands-on) learning. To get started, information will be provided about the standards-based grammar and how it works. Then, results from the facilitators' efforts leveraging this method to link multiple disparate HIV-related datasets will be presented. As a hands-on activity, attendees who have notebook computers will be able to connect to an open source software solution (www.dhis2.org) and "play in a sandbox" to try for themselves some of the techniques that have been described. As learning objectives, it is expected that attendees will: • Be introduced to data linking use cases outside of their everyday experience • Learn about a new technique for expressing aggregate content schema that supports interoperable data exhange • Apply new skills in a hands-on, worked example

Development of standard indicators to assess use of electronic health record systems implemented in low-and medium-income countries

Background Electronic Health Record Systems (EHRs) are being rolled out nationally in many low- and middle-income countries (LMICs) yet assessing actual system usage remains a challenge. We employed a nominal group technique (NGT) process to systematically develop high-quality indicators for evaluating actual usage of EHRs in LMICs. Methods An initial set of 14 candidate indicators were developed by the study team adapting the Human Immunodeficiency Virus (HIV) Monitoring, Evaluation, and Reporting indicators format. A multidisciplinary team of 10 experts was convened in a two-day NGT workshop in Kenya to systematically evaluate, rate (using Specific, Measurable, Achievable, Relevant, and Time-Bound (SMART) criteria), prioritize, refine, and identify new indicators. NGT steps included introduction to candidate indicators, silent indicator ranking, round-robin indicator rating, and silent generation of new indicators. 5-point Likert scale was used in rating the candidate indicators against the SMART components. Results Candidate indicators were rated highly on SMART criteria (4.05/5). NGT participants settled on 15 final indicators, categorized as system use (4); data quality (3), system interoperability (3), and reporting (5). Data entry statistics, systems uptime, and EHRs variable concordance indicators were rated highest. Conclusion This study describes a systematic approach to develop and validate quality indicators for determining EHRs use and provides LMICs with a multidimensional tool for assessing success of EHRs implementations.Funding Agencies|MCW Norwegian Programme for Capacity Development in Higher Education and Research for Development (NORAD: Project) through the HITRAIN program [QZA-0484]</p

Development of standard indicators to assess use of electronic health record systems implemented in low-and medium-income countries

Background: Electronic Health Record Systems (EHRs) are being rolled out nationally in many low- and middle-income countries (LMICs) yet assessing actual system usage remains a challenge. We employed a nominal group technique (NGT) process to systematically develop high-quality indicators for evaluating actual usage of EHRs in LMICs. Methods: An initial set of 14 candidate indicators were developed by the study team adapting the Human Immunodeficiency Virus (HIV) Monitoring, Evaluation, and Reporting indicators format. A multidisciplinary team of 10 experts was convened in a two-day NGT workshop in Kenya to systematically evaluate, rate (using Specific, Measurable, Achievable, Relevant, and Time-Bound (SMART) criteria), prioritize, refine, and identify new indicators. NGT steps included introduction to candidate indicators, silent indicator ranking, round-robin indicator rating, and silent generation of new indicators. 5-point Likert scale was used in rating the candidate indicators against the SMART components. Results: Candidate indicators were rated highly on SMART criteria (4.05/5). NGT participants settled on 15 final indicators, categorized as system use (4); data quality (3), system interoperability (3), and reporting (5). Data entry statistics, systems uptime, and EHRs variable concordance indicators were rated highest. Conclusion: This study describes a systematic approach to develop and validate quality indicators for determining EHRs use and provides LMICs with a multidimensional tool for assessing success of EHRs implementations

The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: A systematic review

Background: It is estimated that one million people infected with HIV initiate anti-retroviral therapy (ART) in resource-constrained countries annually. This occurs against a background of overburdened health workers with limited skills to handle rapidly changing treatment standards and guidelines hence compromising quality of care. Electronic medical record (EMR)-based clinical decision support systems (CDSS) are considered a solution to improve quality of care. Little evidence, however, exists on the effectiveness of EMR-based CDSS on quality of HIV care and treatment in resource-constrained settings. Objective: The aim of this systematic review was to identify original studies on EMR-based CDSS describing process and outcome measures as well as reported barriers to their implementation in resource-constrained settings. We characterized the studies by guideline adherence, data and process, and barriers to CDSS implementation. Methods: Two reviewers independently assessed original articles from a search of the MEDLINE, EMBASE, CINAHL and Global Health Library databases until January 2012. The included articles were those that evaluated or described the implementation of EMR-based CDSS that were used in HIV care in low-income countries. Results: A total of 12 studies met the inclusion criteria, 10 of which were conducted in sub-Saharan Africa and 2 in the Caribbean. None of the papers described a strong (randomized controlled) evaluation design. Guideline adherence: One study showed that ordering rates for CD4 tests were significantly higher when reminders were used. Data and process: Studies reported reduction in data errors, reduction in missed appointments, reduction in missed CD4 results and reduction in patient waiting time. Two studies showed a significant increase in time spent by clinicians on direct patient care. Barriers to CDSS implementation: Technical infrastructure problems such as unreliable electric power and erratic Internet connectivity, clinicians' limited computer skills and failure by providers to comply with the reminders are key impediments to the implementation and effective use of CDSS. The limited number of evaluation studies, the basic and heterogeneous study designs, and varied outcome measures make it difficult to meaningfully conclude on the effectiveness of CDSS on quality of HIV care and treatment in resource-limited settings. High quality evaluation studies are needed. Factors specific to implementation of EMR-based CDSS in resource-limited setting should be addressed before such countries can demonstrate its full benefits. More work needs to be done to overcome the barriers to EMR and CDSS implementation in developing countries such as technical infrastructure and care providers' computer illiteracy. However, simultaneously evaluating and describing CDSS implementation strategies that work can further guide wise investments in their wider rollout. Published by Elsevier Ireland Lt

Automating indicator data reporting from health facility EMR to a national aggregate data system in Kenya: An Interoperability field-test using OpenMRS and DHIS2

Introduction: Developing countries are increasingly strengthening national health information systems (HIS) for evidence-based decision-making. However, the inability to report indicator data automatically from electronic medical record systems (EMR) hinders this process. Data are often printed and manually re-entered into aggregate reporting systems. This affects data completeness, accuracy, reporting timeliness, and burdens staff who support routine indicator reporting from patient-level data.  Method: After conducting a feasibility test to exchange indicator data from Open Medical Records System (OpenMRS) to District Health Information System version 2 (DHIS2), we conducted a field test at a health facility in Kenya. We configured a field-test DHIS2 instance, similar to the Kenya Ministry of Health (MOH) DHIS2, to receive HIV care and treatment indicator data and the KenyaEMR, a customized version of OpenMRS, to generate and transmit the data from a health facility. After training facility staff how to send data using the module, we compared completeness, accuracy and timeliness of automated indicator reporting with facility monthly reports manually entered into MOH DHIS2.Results: All 45 data values in the automated reporting process were 100% complete and accurate while in manual entry process, data completeness ranged from 66.7% to 100% and accuracy ranged from 33.3% to 95.5% for seven months (July 2013-January 2014). Manual tally and entry process required at least one person to perform each of the five reporting activities, generating data from EMR and manual entry required at least one person to perform each of the three reporting activities, while automated reporting process had one activity performed by one person. Manual tally and entry observed in October 2013 took 375 minutes. Average time to generate data and manually enter into DHIS2 was over half an hour (M=32.35 mins, SD=0.29) compared to less than a minute for automated submission (M=0.19 mins, SD=0.15).Discussion and Conclusion: The results indicate that indicator data sent electronically from OpenMRS-based EMR at a health facility to DHIS2 improves data completeness, eliminates transcription errors and delays in reporting, and reduces the reporting burden on human resources. This increases availability of quality indicator data using available resources to facilitate monitoring service delivery and measuring progress towards set goals

Automating indicator data reporting from health facility EMR to a national aggregate data system in Kenya: An Interoperability field-test using OpenMRS and DHIS2

Introduction:Developing countries are increasingly strengthening national health information systems (HIS) for evidence-based decision-making. However, the inability to report indicator data automatically from electronic medical record systems (EMR) hinders this process. Data are often printed and manually re-entered into aggregate reporting systems. This affects data completeness, accuracy, reporting timeliness, and burdens staff who support routine indicator reporting from patient-level data. Method: After conducting a feasibility test to exchange indicator data from Open Medical Records System (OpenMRS) to District Health Information System version 2 (DHIS2), we conducted a field test at a health facility in Kenya. We configured a field-test DHIS2 instance, similar to the Kenya Ministry of Health (MOH) DHIS2, to receive HIV care and treatment indicator data and the KenyaEMR, a customized version of OpenMRS, to generate and transmit the data from a health facility. After training facility staff how to send data using DHIS2 reporting module, we compared completeness, accuracy and timeliness of automated indicator reporting with facility monthly reports manually entered into MOH DHIS2. Results: All 45 data values in the automated reporting process were 100% complete and accurate while in manual entry process, data completeness ranged from 66.7% to 100% and accuracy ranged from 33.3% to 95.6% for seven months (July 2013-January 2014). Manual tally and entry process required at least one person to perform each of the five reporting activities, generating data from EMR and manual entry required at least one person to perform each of the three reporting activities, while automated reporting process had one activity performed by one person. Manual tally and entry observed in October 2013 took 375 minutes. Average time to generate data and manually enter into DHIS2 was over half an hour (M=32.35 mins, SD=0.29) compared to less than a minute for automated submission (M=0.19 mins, SD=0.15). Discussion and Conclusion: The results indicate that indicator data sent electronically from OpenMRS-based EMR at a health facility to DHIS2 improves data completeness, eliminates transcription errors and delays in reporting, and reduces the reporting burden on human resources. This increases availability of quality indicator data using available resources to facilitate monitoring service delivery and measuring progress towards set goals

User Perceptions and Use of an Enhanced Electronic Health Record in Rwanda With and Without Clinical Alerts: Cross-sectional Survey

BackgroundElectronic health records (EHRs) have been implemented in many low-resource settings but lack strong evidence for usability, use, user confidence, scalability, and sustainability. ObjectiveThis study aimed to evaluate staff use and perceptions of an EHR widely used for HIV care in >300 health facilities in Rwanda, providing evidence on factors influencing current performance, scalability, and sustainability. MethodsA randomized, cross-sectional, structured interview survey of health center staff was designed to assess functionality, use, and attitudes toward the EHR and clinical alerts. This study used the associated randomized clinical trial study sample (56/112, 50% sites received an enhanced EHR), pulling 27 (50%) sites from each group. Free-text comments were analyzed thematically using inductive coding. ResultsOf the 100 participants, 90 (90% response rate) were interviewed at 54 health centers: 44 (49%) participants were clinical and 46 (51%) were technical. The EHR top uses were to access client data easily or quickly (62/90, 69%), update patient records (56/89, 63%), create new patient records (49/88, 56%), generate various reports (38/85, 45%), and review previous records (43/89, 48%). In addition, >90% (81/90) of respondents agreed that the EHR made it easier to make informed decisions, was worth using, and has improved patient information quality. Regarding availability, (66/88) 75% said they could always or almost always count on the EHR being available, whereas (6/88) 7% said never/almost never. In intervention sites, staff were significantly more likely to update existing records (P=.04), generate summaries before (P<.001) or during visits (P=.01), and agree that “the EHR provides useful alerts, and reminders” (P<.01). ConclusionsMost users perceived the EHR as well accepted, appropriate, and effective for use in low-resource settings despite infrastructure limitation in 25% (22/88) of the sites. The implementation of EHR enhancements can improve the perceived usefulness and use of key functions. Successful scale-up and use of EHRs in small health facilities could improve clinical documentation, care, reporting, and disease surveillance in low- and middle-income countries

Effect of a clinical decision support system on early action on immunological treatment failure in patients with HIV in Kenya: a cluster randomised controlled trial

A clinical decision support system (CDSS) is a computer program that applies a set of rules to data stored in electronic health records to offer actionable recommendations. We aimed to establish whether a CDSS that supports detection of immunological treatment failure among patients with HIV taking antiretroviral therapy (ART) would improve appropriate and timely action. We did this prospective, cluster randomised controlled trial in adults and children (aged ≥18 months) who were eligible for, and receiving, ART at HIV clinics in Siaya County, western Kenya. Health facilities were randomly assigned (1:1), via block randomisation (block size of two) with a computer-generated random number sequence, to use electronic health records either alone (control) or with CDSS (intervention). Facilities were matched by type and by number of patients enrolled in HIV care. The primary outcome measure was the difference between groups in the proportion of patients who experienced immunological treatment failure and had a documented clinical action. We used generalised linear mixed models with random effects to analyse clustered data. This trial is registered with ClinicalTrials.gov, number NCT01634802. Between Sept 1, 2012, and Jan 31, 2014, 13 clinics, comprising 41,062 patients, were randomly assigned to the control group (n=6) or the intervention group (n=7). Data collection at each site took 12 months. Among patients eligible for ART, 10,358 (99%) of 10,478 patients were receiving ART at control sites and 10,991 (99%) of 11,028 patients were receiving ART at intervention sites. Of these patients, 1125 (11%) in the control group and 1342 (12%) in the intervention group had immunological treatment failure, of whom 332 (30%) and 727 (54%), respectively, received appropriate action. The likelihood of clinicians taking appropriate action on treatment failure was higher with CDSS alerts than with no decision support system (adjusted odds ratio 3·18, 95% CI 1·02-9·87). CDSS significantly improved the likelihood of appropriate and timely action on immunological treatment failure. We expect our findings will be generalisable to virological monitoring of patients with HIV receiving ART once countries implement the 2015 WHO recommendation to scale up viral load monitoring. US President's Emergency Plan for AIDS Relief (PEPFAR), through the US Centers for Disease Control and Preventio