The systems evaluation network: building capability and capacity in the use of systems science across public health

Abstract

Background: The Systems Evaluation Network (SEN) aims to build capability and capacity regarding the use of systems science in public health evaluation. The SEN was established in June 2021 and 3 years from its inception, we undertook a member survey to understand the engagement with, and impact of, the SEN. Methods: An 18-item cross-sectional survey captured quantitative and qualitative responses regarding SEN member perspectives, centring around their experience of the SEN, associated impacts, and future requirements. We analysed quantitative data descriptively and qualitative data through content analysis. Sub-group analyses explored differences between those working in academia vs practice/policy. Results: Seventy-three participants completed the survey, with 60% working in academia and 40% in practice/policy. Considering experiences of the SEN, participants felt the SEN has shared information about innovative methods and evaluation approaches (94.0% agreed), has provided the opportunity to share and learn with other members (86.0% agreed), and has improved knowledge of systems evaluation methods (86.2% agreed). Regarding impacts of the SEN, participants stated that the SEN has increased their capability to apply systems-oriented methods and evaluation of systems approaches (76% agreed) and has facilitated relationships with others (56.9% agreed). Participants shared future capability requirements for evaluation, which focused on methods (e.g. systems dynamics modelling and ripple effects mapping), approaches (e.g. developmental evaluation and embedded researchers), and other ways in which capability could be increased (e.g. by using case studies). Conclusion: This paper illustrates the experiences and impacts of the SEN, identifying its strengths such as the wide range of topics/content and the flexible and accessible delivery format, but contrast against the difficulties of fostering new relationships in an online setting. These findings can help inform the future direction of the SEN and provide insight to other online communities of practice