Can local staff reliably assess their own programs? A confirmatory test-retest study of Lot Quality Assurance Sampling data collectors in Uganda

Background Data collection techniques that routinely provide health system information at the local level are in demand and needed. LQAS is intended for use by local health teams to collect data at the district and sub-district levels. Our question is whether local health staff produce biased results as they are responsible for implementing the programs they also assess. Methods This test-retest study replicates on a larger scale an earlier LQAS reliability assessment in Uganda. We conducted in two districts an LQAS survey using 15 local health staff as data collectors. A week later, the data collectors swapped districts, where they acted as disinterested non-local data collectors, repeating the LQAS survey with the same respondents. We analysed the resulting two data sets for agreement using Cohens’ Kappa. Results The average Kappa score for the knowledge indicators was κ=0.43 (SD=0.16) and for practice indicators κ=0.63 (SD=0.17). These scores show moderate agreement for knowledge indicators and substantial agreement for practice indicators. Analyses confirm that respondents were more knowledgeable on retest; no evidence of bias was found for practices indicators. Conclusion The findings of this study are remarkably similar to those produced in the first reliability study. There is no evidence that using local healthcare staff to collect LQAS data biases data collection in an LQAS study. The bias observed in the knowledge indicators was most likely due to a ‘practice effect’, whereby respondents increased their knowledge as a result of completing the first survey; no corresponding effect was seen in the practice indicators

Levels, trends and determinants of technical efficiency of general hospitals in Uganda: data envelopment analysis and Tobit regression analysis

General hospitals provide a wide range of primary and secondary healthcare services. They accounted for 38% of government funding to health facilities, 8.8% of outpatient department visits and 28% of admissions in Uganda in the financial year 2016/17. We assessed the levels, trends and determinants of technical efficiency of general hospitals in Uganda from 2012/13 to 2016/17. Methods We undertook input-oriented data envelopment analysis to estimate technical efficiency of 78 general hospitals using data abstracted from the Annual Health Sector Performance Reports for 2012/13, 2014/15 and 2016/17. Trends in technical efficiency was analysed using Excel while determinants of technical efficiency were analysed using Tobit Regression Model in STATA 15.1. Results The average constant returns to scale, variable returns to scale and scale efficiency of general hospitals for 2016/17 were 49% (95% CI, 44–54%), 69% (95% CI, 65–74%) and 70% (95% CI, 65–75%) respectively. There was no statistically significant difference in the efficiency scores of public and private hospitals. Technical efficiency generally increased from 2012/13 to 2014/15, and dropped by 2016/17. Some hospitals were persistently efficient while others were inefficient over this period. Hospital size, geographical location, training status and average length of stay were statistically significant determinants of efficiency at 5% level of significance. Conclusion The 69% average variable returns to scale technical efficiency indicates that the hospitals could generate the same volume of outputs using 31% (3439) less staff and 31% (3539) less beds. Benchmarking performance of the efficient hospitals would help to guide performance improvement in the inefficient ones. There is need to incorporate hospital size, geographical location, training status and average length of stay in the resource allocation formula and adopt annual hospital efficiency assessments

Continuous quality improvement as a tool to implement evidence-informed problem solving: experiences from the district and health facility level in Uganda

Background: Continuous quality improvement processes in health care were developed for use at health facility level, and that is where they have been used the most, often addressing defined care processes. However, in different settings different factors have been important to support institutionalization. This study explores how continuous quality improvement processes were institutionalized at the district level and at the health facility level in Uganda. Methods: This qualitative study was carried out in seven districts in Uganda. Semi-structured interviews with key informants from the district health management teams and document review were conducted. Thematic analysis was used to analyze the data. Results: All districts that participated in the study formed Continuous Quality Improvement (CQI) teams both at the district level and at the health facilities. The district CQI teams comprised of members from different departments within the district health office. District level CQI teams were mandated to take the lead in addressing management gaps and follow up CQI activities at the health facility level. Acceptability of quality improvement processes by the district leadership was identified across districts as supporting the successful implementation of CQI. However, high turnover of staff at health facility level was also reported as a detrimental to the successful implementation of quality improvement processes. Also the district health management teams did not engage much in addressing their own roles using continuous quality improvement. Conclusion: The leadership and management provided by the district health management team was an important factor for the use of Continuous Quality Improvement principles within the district. The key roles of the district health team revolved around the institutionalisation of CQI at different levels of the health system, monitoring results of continuous quality improvement implementation, mobilising resources and health care delivery hence promoting the culture of quality, direct implementation of CQI, and creating an enabling environment for the lower-level health facilities to engage in CQI. High turnover of staff at health facility level was also reported as one of the challenges to the successful implementation of continuous quality improvement. The DHT did not engage much in addressing gaps in their own roles using continuous quality improvement. Keywords: Continuous quality improvement, District health management team, Quality improvement, Uganda, Health facilit

Additional file 1: of Can local staff reliably assess their own programs? A confirmatory test-retest study of Lot Quality Assurance Sampling data collectors in Uganda

0 to 5 months test. (XLSX 156 kb

Additional file 2: of Can local staff reliably assess their own programs? A confirmatory test-retest study of Lot Quality Assurance Sampling data collectors in Uganda

0 to 5 months retest. (XLSX 155 kb