Can local staff reliably assess their own programs? A confirmatory test-retest study of Lot Quality Assurance Sampling data collectors in Uganda

Background Data collection techniques that routinely provide health system information at the local level are in demand and needed. LQAS is intended for use by local health teams to collect data at the district and sub-district levels. Our question is whether local health staff produce biased results as they are responsible for implementing the programs they also assess. Methods This test-retest study replicates on a larger scale an earlier LQAS reliability assessment in Uganda. We conducted in two districts an LQAS survey using 15 local health staff as data collectors. A week later, the data collectors swapped districts, where they acted as disinterested non-local data collectors, repeating the LQAS survey with the same respondents. We analysed the resulting two data sets for agreement using Cohens’ Kappa. Results The average Kappa score for the knowledge indicators was κ=0.43 (SD=0.16) and for practice indicators κ=0.63 (SD=0.17). These scores show moderate agreement for knowledge indicators and substantial agreement for practice indicators. Analyses confirm that respondents were more knowledgeable on retest; no evidence of bias was found for practices indicators. Conclusion The findings of this study are remarkably similar to those produced in the first reliability study. There is no evidence that using local healthcare staff to collect LQAS data biases data collection in an LQAS study. The bias observed in the knowledge indicators was most likely due to a ‘practice effect’, whereby respondents increased their knowledge as a result of completing the first survey; no corresponding effect was seen in the practice indicators

Continuous quality improvement as a tool to implement evidence-informed problem solving: experiences from the district and health facility level in Uganda

Background: Continuous quality improvement processes in health care were developed for use at health facility level, and that is where they have been used the most, often addressing defined care processes. However, in different settings different factors have been important to support institutionalization. This study explores how continuous quality improvement processes were institutionalized at the district level and at the health facility level in Uganda. Methods: This qualitative study was carried out in seven districts in Uganda. Semi-structured interviews with key informants from the district health management teams and document review were conducted. Thematic analysis was used to analyze the data. Results: All districts that participated in the study formed Continuous Quality Improvement (CQI) teams both at the district level and at the health facilities. The district CQI teams comprised of members from different departments within the district health office. District level CQI teams were mandated to take the lead in addressing management gaps and follow up CQI activities at the health facility level. Acceptability of quality improvement processes by the district leadership was identified across districts as supporting the successful implementation of CQI. However, high turnover of staff at health facility level was also reported as a detrimental to the successful implementation of quality improvement processes. Also the district health management teams did not engage much in addressing their own roles using continuous quality improvement. Conclusion: The leadership and management provided by the district health management team was an important factor for the use of Continuous Quality Improvement principles within the district. The key roles of the district health team revolved around the institutionalisation of CQI at different levels of the health system, monitoring results of continuous quality improvement implementation, mobilising resources and health care delivery hence promoting the culture of quality, direct implementation of CQI, and creating an enabling environment for the lower-level health facilities to engage in CQI. High turnover of staff at health facility level was also reported as one of the challenges to the successful implementation of continuous quality improvement. The DHT did not engage much in addressing gaps in their own roles using continuous quality improvement. Keywords: Continuous quality improvement, District health management team, Quality improvement, Uganda, Health facilit

Additional file 1: of Can local staff reliably assess their own programs? A confirmatory test-retest study of Lot Quality Assurance Sampling data collectors in Uganda

0 to 5 months test. (XLSX 156 kb

Additional file 2: of Can local staff reliably assess their own programs? A confirmatory test-retest study of Lot Quality Assurance Sampling data collectors in Uganda

0 to 5 months retest. (XLSX 155 kb