Using theories of change to design monitoring and evaluation of community engagement in research: experiences from a research institute in Malawi.

Background: Evaluation of community and public engagement in research is important to deepen understanding of how engagement works and to enhance its effectiveness. Theories of change have been recommended for evaluating community engagement, for their ability to make explicit intended outcomes and understandings of how engagement activities contribute to these outcomes. However, there are few documented examples of using theories of change for evaluation of engagement. This article reports experience of using theories of change to develop a framework for evaluating community engagement in research at a clinical research organisation in Malawi. We describe the steps used to develop theories of change, and the way theories of change were used to design data collection plans. Based on our experience, we reflect on the advantages and challenges of the theory of change approach. Methods: The theories of change and evaluation framework were developed through a series of workshops and meetings between engagement practitioners, monitoring and evaluation staff, and researchers. We first identified goals for engagement, then used 'so that' chains to clarify pathways and intermediate outcomes between engagement activities and goals. Further meetings were held to refine initial theories of change, identify priority information needs, and define feasible evaluation methods. Results: The theory of change approach had several benefits. In particular, it helped to construct an evaluation framework focused on relevant outcomes and not just activities. The process of reflecting on intended goals and pathways also helped staff to review the design of engagement activities. Challenges included practical considerations around time to consider evaluation plans among practitioners (a challenge for evaluation more generally regardless of method), and more fundamental difficulties related to identifying feasible and agreed outcomes. Conclusions: These experiences from Malawi provide lessons for other research organisations considering use of theories of change to support evaluation of community engagement

'Whose failure counts?' A critical reflection on definitions of failure for community health volunteers providing HIV self-testing in a community-based HIV/TB intervention study in urban Malawi.

The category of community health worker applied within the context of health intervention trials has been promoted as a cost-effective approach to meeting study objectives across large populations, relying on the promotion of the concept of 'community belonging' to encourage altruistic volunteerism from community members to promote health. This community-based category of individuals is recruited to facilitate externally driven priorities defined by large research teams, outside of the target research environment. An externally defined intervention is then 'brought to' the community through locally recruited community volunteers who form a bridge between the researchers and participants. The specific role of these workers is context-driven and responsive to the needs of the intervention. This paper is based on the findings from an annual evaluation of community health worker performance employed as community counsellors to deliver semi-supervised HIV self-testing (HIVST) at community level of a large HIV/TB intervention trial conducted in urban Blantyre, Malawi. A performance evaluation was conducted to appraise individual service delivery and assess achievements in meeting pre-defined targets for uptake of HIVST with the aim of improving overall uptake of HIVST. Through an empirical 'evaluation of the evaluation' this paper critically reflects on the position of the community volunteer through the analytical lens of 'failure', exploring the tensions in communication and interpretation of intervention delivery between researchers and community volunteers and the differing perspectives on defining failure. It is concluded that community interventions should be developed in collaboration with the population and that information guiding success should be clearly defined

"Not just dogs, but rabid dogs": tensions and conflicts amongst research volunteers in Malawi.

Building trust between researchers and communities involved in research is one goal of community engagement. This paper examines the implications of community engagement for trust within communities, including trust among community volunteers who assist with research and between these volunteers and other community members. We describe the experiences of two groups of community volunteers recruited as part of an HIV and TB intervention trial in Malawi: cluster representatives, recruited both to act as key informants for TB suspects and mortality reporting and to identify and report community concerns, and community counsellors, recruited to provide semi-supervised HIV self-testing. We examine tensions experienced due to playing multiple roles, and the implications of volunteer responsibilities for short- and long-term community relationships. Data was collected through a workshop, in-depth interviews and focus group discussions with volunteers and community members. While the volunteer system initially enhanced trust among volunteers and with the community, relationships deteriorated when cluster representatives assumed an additional supervisory role part-way through the trial. Combined with challenging recruitment targets and unequal power relations between volunteers, this new role damaged trust, with implications for volunteer well-being and social relationships. These experiences suggest researchers should consider potential social implications when designing community engagement systems

Stakeholder views on the acceptability of human infection studies in Malawi.

BACKGROUND: Human infection studies (HIS) are valuable in vaccine development. Deliberate infection, however, creates challenging questions, particularly in low and middle-income countries (LMICs) where HIS are new and ethical challenges may be heightened. Consultation with stakeholders is needed to support contextually appropriate and acceptable study design. We examined stakeholder perceptions about the acceptability and ethics of HIS in Malawi, to inform decisions about planned pneumococcal challenge research and wider understanding of HIS ethics in LMICs. METHODS: We conducted 6 deliberative focus groups and 15 follow-up interviews with research staff, medical students, and community representatives from rural and urban Blantyre. We also conducted 5 key informant interviews with clinicians, ethics committee members, and district health government officials. RESULTS: Stakeholders perceived HIS research to have potential population health benefits, but they also had concerns, particularly related to the safety of volunteers and negative community reactions. Acceptability depended on a range of conditions related to procedures for voluntary and informed consent, inclusion criteria, medical care or support, compensation, regulation, and robust community engagement. These conditions largely mirror those in existing guidelines for HIS and biomedical research in LMICs. Stakeholder perceptions pointed to potential tensions, for example, balancing equity, safety, and relevance in inclusion criteria. CONCLUSIONS: Our findings suggest HIS research could be acceptable in Malawi, provided certain conditions are in place. Ongoing assessment of participant experiences and stakeholder perceptions will be required to strengthen HIS research during development and roll-out

‘We are the eyes and ears of researchers and community’: understanding the role of community advisory groups in representing researchers and communities in Malawi

Community engagement to protect and empower participating individuals and communities is an ethical requirement in research. There is however limited evidence on effectiveness or relevance of some of the approaches used to improve ethical practice. We conducted a study to understand the rationale, relevance and benefits of community engagement in health research. This paper draws from this wider study and focuses on factors that shaped Community Advisory Group (CAG) members’ selection processes and functions in Malawi. A qualitative research design was used; two participatory workshops were conducted with CAG members to understand their roles in research. Workshop findings were triangulated with insights from ethnographic field notes, key informant interviews with stakeholders, focus group discussions with community members and document reviews. Data were coded manually and thematic content analysis was used to identify main issues. Results have shown that democratic selection of CAG members presented challenges in both urban and rural settings. We also noted that CAG members perceived their role as a form of employment which potentially led to ineffective representation of community interests. We conclude that democratic voting is not enough to ensure effective representation of community's interests of ethical relevance. CAG members’ abilities to understand research ethics, identify potential harms to community and communicate feedback to researchers is critical to optimise engagement of lay community and avoid tokenistic engagement.</p

"Are we getting the biometric bioethics right?" - the use of biometrics within the healthcare system in Malawi.

Biometrics is the science of establishing the identity of an individual based on their physical attributes. Ethical concerns surrounding the appropriate use of biometrics have been raised, especially in resource-poor settings. A qualitative investigation was conducted to explore biometrics clients (n = 14), implementers (n = 12) and policy makers as well as bioethicists (n = 4) perceptions of the ethical aspects of implementing biometrics within the healthcare system in Malawi. Informed use, privacy and confidentiality as well as perceptions of benefits and harms were identified as major issues in the application of biometrics. Implementation of biometrics within the healthcare system in Malawi poses a range of potential ethical issues and practical challenges that impact on equitable uptake. There is a need for more research to explore the benefits and harms of biometrics in practice. Improved community engagement and sensitization should be a required component of biometrics introduction in Malawi

Strengthening ethical community engagement in contemporary Malawi.

Although community engagement is increasingly promoted in global health research to improve ethical research practice, there is sometimes a disconnect between the broader moral ambitions for community engagement in the literature and guidelines on the one hand and its rather narrower practical application in health research on the other. In practice, less attention is paid to engaging communities for the 'intrinsic' value of showing respect and ensuring inclusive participation of community partners in research design. Rather, more attention is paid to the use of community engagement for 'instrumental' purposes to improve community understanding of research and ensure successful study implementation. Against this backdrop, we reviewed the literature and engaged various research stakeholders at a workshop to discuss ways of strengthening ethical engagement of communities and to develop context-relevant guidelines for community engagement in health research in Malawi

Uptake, Accuracy, Safety, and Linkage into Care over Two Years of Promoting Annual Self-Testing for HIV in Blantyre, Malawi: A Community-Based Prospective Study

Background Home-based HIV testing and counselling (HTC) achieves high uptake, but is difficult and expensive to implement and sustain. We investigated a novel alternative based on HIV self-testing (HIVST). The aim was to evaluate the uptake of testing, accuracy, linkage into care, and health outcomes when highly convenient and flexible but supported access to HIVST kits was provided to a well-defined and closely monitored population. Methods and Findings Following enumeration of 14 neighbourhoods in urban Blantyre, Malawi, trained resident volunteer-counsellors offered oral HIVST kits (OraQuick ADVANCE Rapid HIV-1/2 Antibody Test) to adult (≥16 y old) residents (n = 16,660) and reported community events, with all deaths investigated by verbal autopsy. Written and demonstrated instructions, pre- and post-test counselling, and facilitated HIV care assessment were provided, with a request to return kits and a self-completed questionnaire. Accuracy, residency, and a study-imposed requirement to limit HIVST to one test per year were monitored by home visits in a systematic quality assurance (QA) sample. Overall, 14,004 (crude uptake 83.8%, revised to 76.5% to account for population turnover) residents self-tested during months 1–12, with adolescents (16–19 y) most likely to test. 10,614/14,004 (75.8%) participants shared results with volunteer-counsellors. Of 1,257 (11.8%) HIV-positive participants, 26.0% were already on antiretroviral therapy, and 524 (linkage 56.3%) newly accessed care with a median CD4 count of 250 cells/μl (interquartile range 159–426). HIVST uptake in months 13–24 was more rapid (70.9% uptake by 6 mo), with fewer (7.3%, 95% CI 6.8%–7.8%) positive participants. Being “forced to test”, usually by a main partner, was reported by 2.9% (95% CI 2.6%–3.2%) of 10,017 questionnaire respondents in months 1–12, but satisfaction with HIVST (94.4%) remained high. No HIVST-related partner violence or suicides were reported. HIVST and repeat HTC results agreed in 1,639/1,649 systematically selected (1 in 20) QA participants (99.4%), giving a sensitivity of 93.6% (95% CI 88.2%–97.0%) and a specificity of 99.9% (95% CI 99.6%–100%). Key limitations included use of aggregate data to report uptake of HIVST and being unable to adjust for population turnover. Conclusions Community-based HIVST achieved high coverage in two successive years and was safe, accurate, and acceptable. Proactive HIVST strategies, supported and monitored by communities, could substantially complement existing approaches to providing early HIV diagnosis and periodic repeat testing to adolescents and adults in high-HIV settings

Community Engagement Before Initiation of Typhoid Conjugate Vaccine Trial in Schools in Two Urban Townships in Blantyre, Malawi: Experience and Lessons

Background To determine the efficacy of a new typhoid conjugate vaccine in an endemic setting in sub-Saharan Africa, the Typhoid Vaccine Acceleration Consortium is conducting a phase-3 randomized controlled trial in Blantyre, Malawi. This article describes community and stakeholder engagement activities before and during the trial, challenges, and lessons learned. Methods In October 2017, Malawi-Liverpool Wellcome Trust (MLW) organized a wide range of community engagement activities, including meetings with Ministry of Health and Education officials at the district and facility level, local community leadership, and parent teacher association groups. We engaged media outlets to include local and international television, radio, and print media. Community members were informed directly through a study jingle played via loudspeaker from a van and by community-based activities. To review engagement activity effectiveness: The MLW team met to discuss progress and challenges; and a focus group discussion (FGD), consisting of trial staff, sought feedback from the community on each engagement modality. Results The school-based vaccine campaign increased community participation exceeding recruitment targets to date (on average, >200 children/day). Conclusions The FGD concluded that the van and local activities improved awareness and turnout for the trial, but prior engagement with local government and community leadership is an essential mechanism to provide details of the study, answer questions, communicate the value of the study, and address safety concerns. Effective community engagement is essential in a large intervention trial. Multiple channels of communication are required to reach the community and deliver information needed for participation and provide opportunity for dialogue with the trial team

Discordance, Disclosure and Normative Gender Roles: Barriers to Couple Testing Within a Community-Level HIV Self-Testing Intervention in Urban Blantyre, Malawi

A community-based HIV self-testing study in Blantyre, Malawi demonstrated that not all individuals living in couples tested with their partner. We describe factors dissuading individuals in couples from self-testing with their partner. Data were drawn from qualitative study exploring consequences of HIV self-testing within couples. In-depth interviews were conducted with 33 individuals living in couples who tested alone. Participants expressed fear of dealing with HIV-discordant relationships. Failure to self-test with a partner was correlated with gender, with more men than women overtly declining or unconsciously unable to have joint HIV self-test. Men feared exposure of infidelity and were often not available at home for economic reasons. Barriers to uptake of couple HIV self-testing seemed to be shaped by gendered dichotomies of social-relationships. To help achieve the first 90% of the UNAIDS 90:90:90 goals, it is important to overcome structural barriers to realise the full potential of HIV self-testing