Evaluating inter-rater reliability of indicators to assess performance of medicines management in health facilities in Uganda

Background: To build capacity in medicines management, the Uganda Ministry of Health introduced a nationwide supervision, performance assessment and recognition strategy (SPARS) in 2012. Medicines management supervisors (MMS) assess performance using 25 indicators to identify problems, focus supervision, and monitor improvement in medicines stock and storage management, ordering and reporting, and prescribing and dispensing. Although the indicators are well-recognized and used internationally, little was known about the reliability of these indicators. An initial assessment of inter-rater reliability (IRR), which measures agreement among raters (i.e., MMS), showed poor IRR; subsequently, we implemented efforts to improve IRR. The aim of this study was to assess IRR for SPARS indicators at two subsequent time points to determine whether IRR increased following efforts to improve reproducibility. Methods: IRR was assessed in 2011 and again after efforts to improve IRR in 2012 and 2013. Efforts included targeted training, providing detailed guidelines and job aids, and refining indicator definitions and response categories. In the assessments, teams of three MMS measured 24 SPARS indicators in 26 facilities. We calculated IRR as a team agreement score (i.e., percent of the MMS teams in which all three MMS had the same score). Two sample tests for proportions were used to compare IRR scores for each indicator, domain, and overall for the initial assessment and the following two assessments. We also compared the IRR scores for indicators classified as simple (binary) versus complex (multi-component). Logistic regression was used to identify supervisor group characteristics associated with domain-specific and overall IRR scores. Results: Initially only five (21%) indicators had acceptable reproducibility, defined as an IRR score ≥ 75%. At the initial assessment, prescribing quality indicators had the lowest and stock management indicators had the highest IRR. By the third IRR assessment, 12 (50%) indicators had acceptable reproducibility, and the overall IRR score improved from 57% to 72%. The IRR of simple indicators was consistently higher than that of complex indicators in the three assessment periods. We found no correlation between IRR scores and MMS experience or professional background. Conclusions: Assessments of indicator reproducibility are needed to improve IRR. Using simple indicators is recommended. Electronic supplementary material The online version of this article (10.1186/s40545-018-0137-y) contains supplementary material, which is available to authorized users

Improving Potable Water Accessibility And Sustainability Through Efficient Management Of Pipe Water Supply System

This paper discusses how to improve potable water accessibility and sustainability through efficient management of pipe water supply system a case study of Uganda, Kampala region. Kampala the capital city of Uganda still faces a challenge to access clean potable water. Water supply coverage is 77.5 % showing at least 22.5 % of the total population has limited access to potable drinking water causing a gap between water supply and water demand. Hypotheses of the paper were that the city's population is increasing rapidly with an annual urbanization rate of 4.8% this has put pressure on the pipe water infrastructure in terms of expansion and maintenance. Also the functioning pipes in place are too few to provide enough water for the growing population demand. The paper aimed at identifying and prioritizing possible policies on how the gap between the water supply and consumer water demand could be reduced .In that respect a review of both the current pipe water supply system was modelled using a system dynamics approach to allow model and policy testing through computer simulations as well as to validate the different hypotheses. The results showed that the increase in functioning pipes would greatly improve potable water supply however this can only be done with enough available funds. The water board management can achieve this through donations from non-government organizations, loans and customer support in terms of paying their water bills

Additional file 3: of Evaluating inter-rater reliability of indicators to assess performance of medicines management in health facilities in Uganda

Illustrative example of IRR score calculation for SPARS indicators, sub-indicators, and domains across the rater teams. (PDF 99 kb

Additional file 4: of Evaluating inter-rater reliability of indicators to assess performance of medicines management in health facilities in Uganda

IRR score for the rational drug use sub indicators assessments 1–3. (PDF 502 kb