Using theories of change to design monitoring and evaluation of community engagement in research: experiences from a research institute in Malawi.

Background: Evaluation of community and public engagement in research is important to deepen understanding of how engagement works and to enhance its effectiveness. Theories of change have been recommended for evaluating community engagement, for their ability to make explicit intended outcomes and understandings of how engagement activities contribute to these outcomes. However, there are few documented examples of using theories of change for evaluation of engagement. This article reports experience of using theories of change to develop a framework for evaluating community engagement in research at a clinical research organisation in Malawi. We describe the steps used to develop theories of change, and the way theories of change were used to design data collection plans. Based on our experience, we reflect on the advantages and challenges of the theory of change approach. Methods: The theories of change and evaluation framework were developed through a series of workshops and meetings between engagement practitioners, monitoring and evaluation staff, and researchers. We first identified goals for engagement, then used 'so that' chains to clarify pathways and intermediate outcomes between engagement activities and goals. Further meetings were held to refine initial theories of change, identify priority information needs, and define feasible evaluation methods. Results: The theory of change approach had several benefits. In particular, it helped to construct an evaluation framework focused on relevant outcomes and not just activities. The process of reflecting on intended goals and pathways also helped staff to review the design of engagement activities. Challenges included practical considerations around time to consider evaluation plans among practitioners (a challenge for evaluation more generally regardless of method), and more fundamental difficulties related to identifying feasible and agreed outcomes. Conclusions: These experiences from Malawi provide lessons for other research organisations considering use of theories of change to support evaluation of community engagement

Enhancing media communication of health research: an evaluation of the Journalist in Residence Programme at the Malawi Liverpool Wellcome Trust Clinical Research Programme

Background: In 2012, the Science Communication department at the Malawi-Liverpool Wellcome Trust Clinical Research Programme (MLW) established the Journalist in Residence Programme (JIR) to train journalists and give them support when reporting health research stories for MLW. However, the programme had not been evaluated since its inception. We assessed the impact of media training and support through this programme on the development of health research reporting in Malawi. Methods: Qualitative approaches were used to conduct in-depth interviews with five journalists, two editors, two consultants, five researchers, and three Ministry of Health officials. Two focus group discussions were also conducted with selected Community Advisory Group members in Blantyre and Chikwawa. All interviews and focus group discussions were recorded and transcribed verbatim. Data analysis was conducted using Nvivo11. Results: The JIR programme has a positive impact on the development of health research reporting in Malawi. The quantity and quality of health research stories generated from Malawi Liverpool Wellcome Trust research have increased after introducing the programme. Additionally, journalists are motivated to write health research stories as they get training and support through the programme. The health research work being implemented by Malawi-Liverpool Wellcome Trust is appreciated by policy makers and the public at large through stories published by journalists in the programme. Conclusions: This evaluation found that the JIR programme is a powerful tool for achieving the development of health research reporting in Malawi.</ns3:p