Data for local decision-making, not a mere reporting requirement: development of an index to measure facility-level use of HMIS data

# Background Well-functioning health management information systems (HMIS) enable decision-making at all health system levels. This study develops an index to measure the use of HMIS data at the facility level. # Methods We used two rounds of cross-sectional data collected from 305 health facilities in Ethiopia in 2019 (pre-COVID-19) and 2020 (post-COVID-19). We constructed a summative, 10-item index using exploratory factor analysis and 2019 index development data; and used Cronbach's alpha to assess reliability. To examine content validity, we mapped items against a previously published conceptual framework and consulted Ethiopian experts. We then employed one-way ANOVA and t-tests comparing the mean index scores overall and by key facility characteristics between 2019 and 2020. # Results The 10-item index loaded on one factor (Cronbach's alpha=0.74), and the index scores did not differ significantly by facility characteristics in 2019. The mean index score increased from 7.2 in 2019 to 7.9 in 2020 (*P*\<0.01). During this period, more facilities received feedback on HMIS reports from facility leadership (19.3% difference); received actionable recommendations on performance targets and resource allocation (7.5% and 12.3% difference, respectively); and reviewed maternal deaths (15.1% difference); conversely, the proportion of facilities that held participatory performance review meetings monthly or more often decreased by 13.8% (all *P* \<0.05). # Conclusions We propose a facility-level HMIS data use index and document an upward trend in HMIS data use in Ethiopia immediately after the COVID-19 pandemic was declared. Future research should further evaluate and refine the proposed index to support the measurement of HMIS data quality and utilization in Ethiopia and like settings

Lessons Learned From the Capacity-Building and Mentorship Program to Improve Health Information Systems in 11 Districts of Ethiopia

Introduction: Health information systems (HIS) performance in Ethiopia is currently insufficient, and improvements are required to ensure that decision making is data driven. We share our experiences from the early-stage implementation of a package of HIS capacity-strengthening interventions as part of an innovative academic-government collaboration that addresses challenges in HIS performance.Methods: We used routine program data to assess HIS performance using the Performance of Routine Information System Management (PRISM) assessment tools. The assessment employed a pre-post design and was conducted in a total of 24 selected health facilities (6 hospitals and 18 health centers) from 11 districts in Ethiopia at project baseline (2018) and midline (2020).Results: Source document completeness rate reached less than 80% for the majority of the assessed data elements. Improvements were observed in quarterly report completeness (26% vs. 83%) and timeliness (17% vs. 48%). Though data inaccuracies are noted for all assessed data elements in 2020, the majority (83%) of skilled birth attendance and HIV reports (68%) fall within the acceptable range of reporting accuracy. The identification of performance-related problems, using performance monitoring team (PMT) meetings, improved between 2018 and 2020 (67% vs 89%). Similar improvements were also observed in developing action plans to solve identified problems via the PMT (52% in 2018 vs. 89% in 2020). Data use for planning and target setting (65% in 2018 vs. 90% in 2020), reviewing performance (58% in 2018 vs. 60% in 2020), and supervision (51% in 2018 vs. 53% in 2020) all improved among assessed health facilities.Discussion: This study showed that a capacity-building and mentorship program that engages experts from multiple disciplines and sectors can improve the quality and use of health data. This partnership enabled engagement between government and academic stakeholders and allowed for a more robust exchange of resources and expertise toward HIS improvement

Current and Future Needs for Human Resources for Ethiopia’s National Health Information System: Survey and Forecasting Study

BackgroundStrengthening the national health information system is one of Ethiopia’s priority transformation agendas. A well-trained and competent workforce is the essential ingredient to a strong health information system. However, this workforce has neither been quantified nor characterized well, and there is no roadmap of required human resources to enhance the national health information system. ObjectiveWe aimed to determine the current state of the health information system workforce and to forecast the human resources needed for the health information system by 2030. MethodsWe conducted a survey to estimate the current number of individuals employed in the health information system unit and the turnover rate. Document review and key-informant interviews were used to collect current human resources and available health information system position data from 110 institutions, including the Ministry of Health, federal agencies, regional health bureaus, zonal health departments, district health offices, and health facilities. The Delphi technique was used to forecast human resources required for the health information system in the next ten years: 3 rounds of workshops with experts from the Ministry of Health, universities, agencies, and regional health bureaus were held. In the first expert meeting, we set criteria, which was followed by expert suggestions and feedback. ResultsAs of April 2020, there were 10,344 health information system professionals working in the governmental health system. Nearly 95% (20/21) of district health offices and 86.7% (26/30) of health centers reported that the current number of health information system positions was inadequate. In the period from June 2015 to June 2019, health information technicians had high turnover (48/244, 19.7%) at all levels of the health system. In the next ten years, we estimate that 50,656 health information system professionals will be needed to effectively implement the Ethiopia's national health information system. ConclusionsCurrent health information system–related staffing levels were found to be inadequate. To meet the estimated need of 50,656 multidisciplinary health information system professionals by 2030, the Ministry of Health and regional health bureaus, in collaboration with partners and academic institutions, need to work on retaining existing and training additional health information system professionals

Maternal Service Coverage and Its Relationship To Health Information System Performance: A Linked Facility and Population-Based Survey in Ethiopia

Background: Studies in Ethiopia show an increasing trend in maternal health service use, such as having at least 4 visits of antenatal care (ANC4+) and skilled birth attendance (SBA). Improving the health information system (HIS) is an intervention that can improve service uptake and quality. We conducted a baseline study to measure current maternal service coverage, HIS performance status, and their relationship.Methods: We conducted a linked health facility-level and population-based survey from September 2020 to October 2020. The study covers all regions of Ethiopia. For the population-based survey, 3,016 mothers were included. Overall, 81 health posts, 71 health centers, and 15 hospitals were selected for the facility survey. A two-stage sampling procedure was applied to select target households. The study used modified Performance of Routine Information System Management tools for the facility survey and a structured questionnaire for the household survey. Multilevel logistic regression was employed to account for clustering and control for likely confounders.Results: Maternal service indicators, ANC4+ visits (54.0%), SBA (75.8%), postnatal care (70.6%), and cesarean delivery (9%) showed good service uptake. All data quality and use indicators showed lower performance compared to the national target of 90%. Maternal education and higher levels of wealth index were significantly and positively associated with all selected maternal service indicators. Longer distance from health facilities was significantly and negatively associated with SBA and the maternal care composite indicator. Among HIS-related indicators, availability of electronic HIS tools was significantly associated with maternal care composite indicator and ANC4+.Conclusions: Maternal service indicators showed promising performance. However, current HIS performance is suboptimal. Both service user and HIS-related factors were associated with maternal service uptake. Conducting similar research outside of the project sites will be helpful to have a wider understanding and better coverage

Maturity Assessment of District Health Information System Version 2 Implementation in Ethiopia: Current Status and Improvement Pathways

BackgroundAlthough Ethiopia has made remarkable progress in the uptake of the District Health Information System version 2 (DHIS2) for national aggregate data reporting, there has been no comprehensive assessment of the maturity level of the system. ObjectiveThis study aims to assess the maturity level of DHIS2 implementation in Ethiopia and propose a road map that could guide the progress toward a higher level of maturity. We also aim to assess the current maturity status, implementation gaps, and future directions of DHIS2 implementation in Ethiopia. The assessment focused on digital health system governance, skilled human resources, information and communication technology (ICT) infrastructure, interoperability, and data quality and use. MethodsA collaborative assessment was conducted with the engagement of key stakeholders through consultative workshops using the Stages of Continuous Improvement tool to measure maturity levels in 5 core domains, 13 components, and 39 subcomponents. A 5-point scale (1=emerging, 2=repeatable, 3=defined, 4=managed, and 5=optimized) was used to measure the DHIS2 implementation maturity level. ResultsThe national DHIS2 implementation’s maturity level is currently at the defined stage (score=2.81) and planned to move to the manageable stage (score=4.09) by 2025. The domain-wise maturity score indicated that except for ICT infrastructure, which is at the repeatable stage (score=2.14), the remaining 4 domains are at the defined stage (score=3). The development of a standardized and basic DHIS2 process at the national level, the development of a 10-year strategic plan to guide the implementation of digital health systems including DHIS2, and the presence of the required competencies at the facility level to accomplish specific DHIS2-related tasks are the major strength of the Ministry of Health of Ethiopia so far. The lack of workforce competency guidelines to support the implementation of DHIS2; the unavailability of core competencies (knowledge, skills, and abilities) required to accomplish DHIS2 tasks at all levels of the health system; and ICT infrastructures such as communication network and internet connectivity at the district, zonal, and regional levels are the major hindrances to effective DHIS2 implementation in the country. ConclusionsOn the basis of the Stages of Continuous Improvement maturity model toolkit, the implementation status of DHIS2 in Ethiopia is at the defined stage, with the ICT infrastructure domain being at the lowest stage as compared to the other 4 domains. By 2025, the maturity status is planned to move from the defined stage to the managed stage by improving the identified gaps. Various action points are suggested to address the identified gaps and reach the stated maturity level. The responsible body, necessary resources, and methods of verification required to reach the specified maturity level are also listed