How to and how not to develop a theory of change to evaluate a complex intervention: reflections on an experience in the Democratic Republic of Congo.

Theories of change (ToCs) describe how interventions can bring about long-term outcomes through a logical sequence of intermediate outcomes and have been used to design and measure the impact of public health programmes in several countries. In recognition of their capacity to provide a framework for monitoring and evaluation, they are being increasingly employed in the development sector. The construction of a ToC typically occurs through a consultative process, requiring stakeholders to reflect on how their programmes can bring about change. ToCs help make explicit any underlying assumptions, acknowledge the role of context and provide evidence to justify the chain of causal pathways. However, while much literature exists on how to develop a ToC with respect to interventions in theory, there is comparatively little reflection on applying it in practice to complex interventions in the health sector. This paper describes the initial process of developing a ToC to inform the design of an evaluation of a complex intervention aiming to improve government payments to health workers in the Democratic Republic of Congo. Lessons learnt include: the need for the ToC to understand how the intervention produces effects on the wider system and having broad stakeholder engagement at the outset to maximise chances of the intervention's success and ensure ownership. Power relationships between stakeholders may also affect the ToC discourse but can be minimised by having an independent facilitator. We hope these insights are of use to other global public health practitioners using this approach to evaluate complex interventions

A cross-sectional study of the income sources of primary care health workers in the Democratic Republic of Congo.

BACKGROUND: In the Democratic Republic of Congo (DRC), the state system to remunerate health workers is poorly functional, encouraging diversification of income sources and corruption. Given the central role that health workers play in health systems, policy-makers need to ensure health workers are remunerated in a way which best incentivises them to provide effective and good quality services. This study describes the different sources and quantities of income paid to primary care health workers in Equateur, Maniema, Kasai Occidental, Province Orientale and Kasai Oriental provinces. It also explores characteristics associated with the receipt of different sources of income. METHODS: Quantitative data on the income received by health workers were collected through baseline surveys. Descriptive statistics explored the demographic characteristics of health workers surveyed, and types and amounts of incomes received. A series of regression models were estimated to examine the health worker and facility-level determinants of receiving each income source and of levels received. Qualitative data collection was carried out in Kasai Occidental province to explore perceptions of each income source and reasons for receiving each. RESULTS: Nurses made up the majority of workers in primary care. Only 31% received a government salary, while 75% reported compensation from user fees. Almost half of all nurses engaged in supplemental non-clinical activities. Receipt of government payments was associated with income from private practice and non-clinical activities. Male nurses were more likely to receive per diems, performance payments, and higher total remuneration compared to females. Contextual factors such as provincial location, presence of externally financed health programmes and local user fee policy also influenced the extent to which nurses received many income sources. CONCLUSIONS: The receipt of government payments was unreliable and had implications for receipt of other income sources. A mixture of individual, facility and geographical factors were associated with the receipt of various income sources. Greater co-ordination is needed between partners involved in health worker remuneration to design more effective financial incentive packages, reduce the fragmentation of incomes and improve transparency in the payment of workers in the DRC

What Happens When Donors Pull Out? Examining Differences in Motivation Between Health Workers Who Recently Had Performance-Based Financing (PBF) Withdrawn With Workers Who Never Received PBF in the Democratic Republic of Congo

Background: A motivated workforce is necessary to ensure the delivery of high quality health services. In developing countries, performance-based financing (PBF) is often employed to increase motivation by providing financial incentives linked to performance. However, given PBF schemes are usually funded by donors, their long-term financing is not always assured, and the effects of withdrawing PBF on motivation are largely unknown. This cross-sectional study aimed to identify differences in motivation between workers who recently had donor-funded PBF withdrawn, with workers who had not received PBF. Methods: Quantitative data were collected from 485 health workers in 5 provinces using a structured survey containing questions on motivation which were based on an established motivation framework. Confirmatory factor analysis was used to verify dimensions of motivation, and multiple regression to assess differences in motivation scores between workers who had previously received PBF and those who never had. Qualitative interviews were also carried out in Kasai Occidental province with 16 nurses who had previously or never received PBF. Results: The results indicated that workers in facilities where PBF had been removed scored significantly lower on most dimensions of motivation compared to workers who had never received PBF. The removal of the PBF scheme was blamed for an exodus of staff due to the dramatic reduction in income, and negatively impacted on relationships between staff and the local community. Conclusion: Donors and governments unable to sustain PBF or other donor-payments should have clear exit strategies and institute measures to mitigate any adverse effects on motivation following withdrawal

What Happens When Donors Pull Out? Examining Differences in Motivation Between Health Workers Who Recently Had Performance-Based Financing (PBF) Withdrawn With Workers Who Never Received PBF in the Democratic Republic of Congo.

BACKGROUND: A motivated workforce is necessary to ensure the delivery of high quality health services. In developing countries, performance-based financing (PBF) is often employed to increase motivation by providing financial incentives linked to performance. However, given PBF schemes are usually funded by donors, their long-term financing is not always assured, and the effects of withdrawing PBF on motivation are largely unknown. This cross-sectional study aimed to identify differences in motivation between workers who recently had donor-funded PBF withdrawn, with workers who had not received PBF. METHODS: Quantitative data were collected from 485 health workers in 5 provinces using a structured survey containing questions on motivation which were based on an established motivation framework. Confirmatory factor analysis was used to verify dimensions of motivation, and multiple regression to assess differences in motivation scores between workers who had previously received PBF and those who never had. Qualitative interviews were also carried out in Kasai Occidental province with 16 nurses who had previously or never received PBF. RESULTS: The results indicated that workers in facilities where PBF had been removed scored significantly lower on most dimensions of motivation compared to workers who had never received PBF. The removal of the PBF scheme was blamed for an exodus of staff due to the dramatic reduction in income, and negatively impacted on relationships between staff and the local community. CONCLUSION: Donors and governments unable to sustain PBF or other donor-payments should have clear exit strategies and institute measures to mitigate any adverse effects on motivation following withdrawal

A proposal for further developing fatigue-related post COVID-19 health states for burden of disease studies

Previous efforts to estimate the burden of fatigue-related symptoms due to long COVID have a very high threshold for inclusion of cases, relative to the proposed definition from the World Health Organization. In practice this means that milder cases, that may be occurring very frequently, are not included in estimates of the burden of long COVID which will result in underestimation. A more comprehensive approach to modelling the disease burden from long COVID, in relation to fatigue, can ensure that we do not only focus on what is easiest to measure; which risks losing focus of less severe health states that may be more difficult to measure but are occurring very frequently. Our proposed approach provides a means to better understand the scale of challenge from long COVID, for consideration when preventative and mitigative action is being planned

Prevalence and risk factors for long COVID among adults in Scotland using electronic health records : a national, retrospective, observational cohort study

Acknowledgements This work was supported by the Chief Scientist Office, grant number COV/LTE/20/15. EAVE II is supported by a grant (MC_PC_19075) from the Medical Research Council; and a grant (MC_PC_19004) from BREATHE–The Health Data Research Hub for Respiratory Health, funded through the UK Research and Innovation Industrial Strategy Challenge Fund. LD was supported by a post-doctoral clinical fellowship from the Asthma UK Centre for Applied Research. SVK acknowledges funding from a NRS Senior Clinical Fellowship (SCAF/15/02), the Medical Research Council (MC_UU_00022/2) and the Scottish Government Chief Scientist Office (SPHSU17). The authors would like to acknowledge the support of Dave Kelly and Lamorna Brown of Albasoft Ltd., and Sharon Kennedy, Mike Birnie, Safraj Shahul Hameed and Elliott Hall of Public Health Scotland for their involvement in obtaining approvals, provisioning, and linking data and the use of the secure analytical platform within the National Safe Haven. Funding Chief Scientist Office (Scotland), Medical Research Council, and BREATHE.Peer reviewe

Prevalence and risk factors for long COVID among adults in Scotland using electronic health records : a national, retrospective, observational cohort study

Background: Long COVID is a debilitating multisystem condition. The objective of this study was to estimate the prevalence of long COVID in the adult population of Scotland, and to identify risk factors associated with its development. Methods: In this national, retrospective, observational cohort study, we analysed electronic health records (EHRs) for all adults (≥18 years) registered with a general medical practice and resident in Scotland between March 1, 2020, and October 26, 2022 (98–99% of the population). We linked data from primary care, secondary care, laboratory testing and prescribing. Four outcome measures were used to identify long COVID: clinical codes, free text in primary care records, free text on sick notes, and a novel operational definition. The operational definition was developed using Poisson regression to identify clinical encounters indicative of long COVID from a sample of negative and positive COVID-19 cases matched on time-varying propensity to test positive for SARS-CoV-2. Possible risk factors for long COVID were identified by stratifying descriptive statistics by long COVID status. Findings: Of 4,676,390 participants, 81,219 (1.7%) were identified as having long COVID. Clinical codes identified the fewest cases (n = 1,092, 0.02%), followed by free text (n = 8,368, 0.2%), sick notes (n = 14,469, 0.3%), and the operational definition (n = 64,193, 1.4%). There was limited overlap in cases identified by the measures; however, temporal trends and patient characteristics were consistent across measures. Compared with the general population, a higher proportion of people with long COVID were female (65.1% versus 50.4%), aged 38–67 (63.7% versus 48.9%), overweight or obese (45.7% versus 29.4%), had one or more comorbidities (52.7% versus 36.0%), were immunosuppressed (6.9% versus 3.2%), shielding (7.9% versus 3.4%), or hospitalised within 28 days of testing positive (8.8% versus 3.3%%), and had tested positive before Omicron became the dominant variant (44.9% versus 35.9%). The operational definition identified long COVID cases with combinations of clinical encounters (from four symptoms, six investigation types, and seven management strategies) recorded in EHRs within 4–26 weeks of a positive SARS-CoV-2 test. These combinations were significantly (p < 0.0001) more prevalent in positive COVID-19 patients than in matched negative controls. In a case-crossover analysis, 16.4% of those identified by the operational definition had similar healthcare patterns recorded before testing positive. Interpretation:The prevalence of long COVID presenting in general practice was estimated to be 0.02–1.7%, depending on the measure used. Due to challenges in diagnosing long COVID and inconsistent recording of information in EHRs, the true prevalence of long COVID is likely to be higher. The operational definition provided a novel approach but relied on a restricted set of symptoms and may misclassify individuals with pre-existing health conditions. Further research is needed to refine and validate this approach

Prevalence and risk factors for long COVID among adults in Scotland using electronic health records: a national, retrospective, observational cohort study.

Background: Long COVID is a debilitating multisystem condition. The objective of this study was to estimate the prevalence of long COVID in the adult population of Scotland, and to identify risk factors associated with its development. Methods: In this national, retrospective, observational cohort study, we analysed electronic health records (EHRs) for all adults (≥18 years) registered with a general medical practice and resident in Scotland between March 1, 2020, and October 26, 2022 (98–99% of the population). We linked data from primary care, secondary care, laboratory testing and prescribing. Four outcome measures were used to identify long COVID: clinical codes, free text in primary care records, free text on sick notes, and a novel operational definition. The operational definition was developed using Poisson regression to identify clinical encounters indicative of long COVID from a sample of negative and positive COVID-19 cases matched on time-varying propensity to test positive for SARS-CoV-2. Possible risk factors for long COVID were identified by stratifying descriptive statistics by long COVID status. Findings: Of 4,676,390 participants, 81,219 (1.7%) were identified as having long COVID. Clinical codes identified the fewest cases (n = 1,092, 0.02%), followed by free text (n = 8,368, 0.2%), sick notes (n = 14,469, 0.3%), and the operational definition (n = 64,193, 1.4%). There was limited overlap in cases identified by the measures; however, temporal trends and patient characteristics were consistent across measures. Compared with the general population, a higher proportion of people with long COVID were female (65.1% versus 50.4%), aged 38–67 (63.7% versus 48.9%), overweight or obese (45.7% versus 29.4%), had one or more comorbidities (52.7% versus 36.0%), were immunosuppressed (6.9% versus 3.2%), shielding (7.9% versus 3.4%), or hospitalised within 28 days of testing positive (8.8% versus 3.3%%), and had tested positive before Omicron became the dominant variant (44.9% versus 35.9%). The operational definition identified long COVID cases with combinations of clinical encounters (from four symptoms, six investigation types, and seven management strategies) recorded in EHRs within 4–26 weeks of a positive SARS-CoV-2 test. These combinations were significantly (p &lt; 0.0001) more prevalent in positive COVID-19 patients than in matched negative controls. In a case-crossover analysis, 16.4% of those identified by the operational definition had similar healthcare patterns recorded before testing positive. Interpretation: The prevalence of long COVID presenting in general practice was estimated to be 0.02–1.7%, depending on the measure used. Due to challenges in diagnosing long COVID and inconsistent recording of information in EHRs, the true prevalence of long COVID is likely to be higher. The operational definition provided a novel approach but relied on a restricted set of symptoms and may misclassify individuals with pre-existing health conditions. Further research is needed to refine and validate this approach