First regulatory inspections measuring adherence to Good Pharmacy Practices in the public sector in Uganda: a cross-sectional comparison of performance between supervised and unsupervised facilities

GPP inspection indicators with classification (critical, major, and minor) and overlap with SPARS indicators indicated with*, partly overlap **. (DOCX 145 kb

Evaluating inter-rater reliability of indicators to assess performance of medicines management in health facilities in Uganda

Background: To build capacity in medicines management, the Uganda Ministry of Health introduced a nationwide supervision, performance assessment and recognition strategy (SPARS) in 2012. Medicines management supervisors (MMS) assess performance using 25 indicators to identify problems, focus supervision, and monitor improvement in medicines stock and storage management, ordering and reporting, and prescribing and dispensing. Although the indicators are well-recognized and used internationally, little was known about the reliability of these indicators. An initial assessment of inter-rater reliability (IRR), which measures agreement among raters (i.e., MMS), showed poor IRR; subsequently, we implemented efforts to improve IRR. The aim of this study was to assess IRR for SPARS indicators at two subsequent time points to determine whether IRR increased following efforts to improve reproducibility. Methods: IRR was assessed in 2011 and again after efforts to improve IRR in 2012 and 2013. Efforts included targeted training, providing detailed guidelines and job aids, and refining indicator definitions and response categories. In the assessments, teams of three MMS measured 24 SPARS indicators in 26 facilities. We calculated IRR as a team agreement score (i.e., percent of the MMS teams in which all three MMS had the same score). Two sample tests for proportions were used to compare IRR scores for each indicator, domain, and overall for the initial assessment and the following two assessments. We also compared the IRR scores for indicators classified as simple (binary) versus complex (multi-component). Logistic regression was used to identify supervisor group characteristics associated with domain-specific and overall IRR scores. Results: Initially only five (21%) indicators had acceptable reproducibility, defined as an IRR score ≥ 75%. At the initial assessment, prescribing quality indicators had the lowest and stock management indicators had the highest IRR. By the third IRR assessment, 12 (50%) indicators had acceptable reproducibility, and the overall IRR score improved from 57% to 72%. The IRR of simple indicators was consistently higher than that of complex indicators in the three assessment periods. We found no correlation between IRR scores and MMS experience or professional background. Conclusions: Assessments of indicator reproducibility are needed to improve IRR. Using simple indicators is recommended. Electronic supplementary material The online version of this article (10.1186/s40545-018-0137-y) contains supplementary material, which is available to authorized users

Inter-rater reliability and validity of good pharmacy practices measures in inspection of public sector health facility pharmacies in Uganda

Abstract Background The National Drug Authority (NDA) inspects and certifies private and public sector pharmacies in Uganda using an indicator-based inspection tool that measures adherence to good pharmacy practices (GPP). 67 measures identify the situation in the domains of premises, dispensing quality, stores management, and operating requirements. Although the GPP measures are well-recognized and used internationally, little is known about their validity and reliability. The study aimed to assess validity, which measures agreement of GPP measures between a gold standard inspector and NDA inspector and inter-rater reliability (IRR), which measures agreement among NDA inspectors, of GPP measures. Methods We assessed validity and IRR by four teams of inspectors in eight government health facilities that represent three levels of care. Each team inspected two facilities, resulting in 24 total inspections. Each team comprised one central-level inspector, one district-level inspector, and one gold-standard inspector (i.e., a very experienced central-level inspector). We calculated median validity and IRR for each GPP measure, overall, indicator categorized as either critical, major, or minor, by domains, by the inspection decision (i.e., certified or not certified) and by adequatevalidity and IRR score (i.e., score ≥ 75%). Results The median validity for all GPP measures was 69%, with 29 (43%) measures having an adequate validity of ≥75%. The median IRR for all GPP measures was 71%, with 31 (46%) having an adequate IRR measure of ≥75%. Validity did not differ significantly by indicator category, domain or level of care. Adequate IRR and validity score (≥75%) was lowest for critical measures, which are key determinants of the certification decision, scoring 20 and 40% respectively. District inspectors had lower median validity for critical indicators and premises and higher validity for store management. Compared to central inspectors, the validity of district inspectors’certification decisions was lower; in the eight facilities, three district inspectors agreed with gold standard inspector vs. all eight central inspectors. Conclusions Our findings question the validity and reliability of many GPP inspection measures, particularly critical measures that greatly impact certification decision. This study demonstrates the need for assessments of, and interventions to improve, validity and reproducibility of GPP measures and inspections

Competency in supportive supervision: a study of public sector medicines management supervisors in Uganda

Background: Supportive supervision has been found to be more effective than corrective fault-oriented inspections. Uganda’s Ministry of Health in 2012 implemented a comprehensive strategy (SPARS) to build medicines management capacity in public sector health facilities. The approach includes supportive supervision. This structured observational study assesses supportive supervision competency among medicines management supervisors (MMS). Method The study used structured observations of two groups of five purposely selected MMS—one group supervising facilities with greater medicines management improvement during one year of SPARS and one group with less improvement, based on quantitative metrics. We observed and scored behaviors and skills of supervisors in 11 categories deemed critical for effective and supportive supervision. Results: Supportive supervision was not evenly or adequately implemented, with the median supportive supervision competency score for all observed supervisors being 38%. Supervisors’ main strengths were problem identification, data interpretation, education, and providing constructive feedback (45%–47%). Their weakest areas were assuring continuity and setting targets (17%), and most MMS were fair to strong in effective communication, use of tools, and problem solving. MMS of facilities with little improvement in medicines management over time were weak in setting targets and promoting participation. There was a 33 percentage point difference in the median supportive supervision competency scores between MMS of facilities with more versus less improvement (57%–24%) and a 77 percentage point difference in competency between the highest and lowest scoring MMS (77%–0%). We did not find a relationship between MMS experience (number of visits implemented) and their supportive supervision competency or facility improvement in medicines management. However, there is a likely relationship between supportive supervision competency and facility improvement. Conclusion: Competency of MMS in supportive supervision among the sampled MMS was generally weak, but with much individual variation. Our results suggest that MMS’ supportive supervision competency is positively related to the SPARS effectiveness scores of the facilities they supervise. We recommend strategies to strengthen supportive supervision behaviors and skills. Electronic supplementary material The online version of this article (10.1186/s40545-017-0121-y) contains supplementary material, which is available to authorized users

Pharmaceutical system strengthening in Uganda: implementing a holistic, evidence-informed, long-term strategy

Abstract A strong pharmaceutical sector is a precondition for effective and efficient health care and financing systems, and thus for achieving the best possible health of a population. Supported by visionary, long-term donor funds, in conjunction with mutual trust, the USAID-funded Securing Ugandans Rights to Essential Medicines (SURE) and Uganda Health Supply Chain (UHSC) program engaged in a close, more than 10 year-long (in 2018) collaboration with the Ministry of Health of Uganda. Over time, the partnership implemented numerous multi-pronged comprehensive changes in the pharmaceutical sector and conducted research to document successes and failures. We describe the evolution and key characteristics of the SURE/UHSC interventions

Additional file 1: of An exploratory study on equity in funding allocation for essential medicines and health supplies in Uganda’s public sector

The data collection tool is attached in Additional file 1. (DOCX 169 kb