Developing a methodology and tool for the monitoring and evaluation of social initiatives: a systems-thinking approach

Abstract

Hitchcock, O. E. J. 2025. Developing a Methodology and Tool for the Monitoring and Evaluation of Social Initiatives: A Systems-Thinking Approach. Unpublished masters thesis. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/c4fe4856-dfd9-44bb-8e3f-d04441395efdThesis (MEng)--Stellenbosch University, 2025.ENGLISH ABSTRACT: This thesis presents a conceptual Monitoring and Evaluation (M&E) approach with a supporting framework designed for the M&E of Social Initiatives (SIs), addressing critical challenges in accurately assessing complex, context dependent interventions. Rooted in a Systems- Thinking (ST) approach and guided by the principles of Realist Evaluation (RE), the framework is designed to capture both the nuanced context and the dynamic mechanisms underpinning SIs. While traditional M&E methods often lack the adaptability required to account for contextspecific variables, this integrated methodology advances the field by offering a comprehensive, adaptable tool for practitioners and evaluators. The research is structured in two primary phases. Phase 1 involves a scoping review, systematically analysing existing literature on M&E, ST, and RE to establish a theoretical foundation for the framework. This phase identifies the limitations of current evaluation methodologies in capturing the interplay between context and mechanism within SIs, thereby establishing the need for a more holistic approach. Phase 2 then details the iterative development of the framework, integrating insights from ST and RE to enable evaluators to systematically link SI outputs, outcomes, and impacts to underlying causal mechanisms. The framework is tested through a case study application to Unjani Clinics, a South African Non-Profit Company (NPC) providing low-cost, nurse-led Primary Health Care (PHC) services to underserved communities. This case study demonstrates the application of the coneptual methodology and supporting framework, applying ST tools such as Causal Loop Diagrams (CLDs) and Behaviour over time graphs (BOTGs) to map and measure the complexities within the healthcare system and Unjani’s role within it. Outputs are examined as tangible, measurable results, such as clinic expansion and increased patient consultations, while outcomes reflect broader, long-term impacts, including improved health metrics and economic empowerment within local communities. The findings underscore the framework’s effectiveness in identifying and mapping the mechanisms through which Unjani Clinics impact community health outcomes, enabling a clearer understanding of both direct and indirect contributions. Furthermore, the study highlights the importance of contextual factors—such as public-private partnerships, community trust, and healthcare accessibility—in shaping the success and sustainability of SIs. Limitations arising from data availability and eHealth Research Ethics Committee (HREC) constraints are acknowledged, and recommendations for future research include expanded longitudinal studies and the incorporation of additional metrics to validate the framework’s applicability across varied social sectors. In conclusion, this thesis contributes to the field of M&E by offering a validated, theoretically grounded framework that bridges the gap between M&E theory and practical application to the M&E of SIs. By incorporating ST and RE, the framework allows evaluators to comprehensively assess SIs in real-world, complex environments, ultimately supporting more informed decision-making and enhancing the long-term effectiveness of social interventions. The framework’s adaptability and usablility make it a valuable asset for stakeholders engaged in the planning, implementation, and evaluation of SIs, particularly in contexts where traditional M&E approaches may fall short.AFRIKAANSE OPSOMMING: Hierdie tesis bied ’n konseptuele benadering tot monitering en evaluering met ’n ondersteunende raamwerk vir sosiale intiative wat belangrike uitdagings in die akkuraatheid van konteksafhanklike evaluering aanspreek. Gegrond op ’n sisteemdenke-benadering en gelei deur die beginsels van realistiese evaluering, is die raamwerk ontwerp om sowel die konteks as die dinamiese meganismes wat sosiale intervensies onderlˆe, vas te vang. Hierdie ge¨ıntegreerde metodologie bied ’n omvattende, aanpasbare instrument vir praktisyns en evalueerders. Die navorsing is gestruktureer in twee fases. Fase 1 behels omvang hersiening, sowel as ’n literatuuroorsig oor monitering en evaluering, sisteemdenke en realistiese evaluering om ’n teoretiese grondslag vir die raamwerk te skep. Hierdie fase beklemtoon die tekortkominge van huidige metodologie¨e om die wisselwerking tussen konteks en meganisme in sosiale intervensies vas te vang. Fase 2 fokus op die iteratiewe ontwikkeling van die raamwerk om uitsette, uitkomste en impakte sistematies aan oorsaaklike meganismes te koppel. Die raamwerk word getoets deur ’n gevallestudie van Unjani-klinieke, ’n Suid-Afrikaanse maatskappy sonder winsoogmerk wat bekostigbare, verpleegster-geleide primˆere gesondheidsorgdienste aan onderbediende gemeenskappe verskaf. Hierdie gevallestudie demonstreer die raamwerk se operasionalisering deur sisteemdenke-instrumente soos oorsaaklike terugvoerlusse en gedragsoor-tyd-grafieke te gebruik om die kompleksiteite in die gesondheidstelsel te karteer en te meet. Tasbare, meetbare uitsette, soos kliniekuitbreiding en verhoogde konsultasies, asook langtermyn-uitkomste, soos verbeterde gesondheid en ekonomiese bemagtiging, word ondersoek. Die bevindinge beklemtoon die raamwerk se doeltreffenheid in die identifisering en kartering van die maganismes waardeur Unjni Klinieke ’n impak op gemeenskapsgesondheidsuitkomstes maak. Dit help om ’n duideliker begrip van beide, directe an indirekte bydraes te ontwikkel. Verder beklemtoon dit die rol van kontekstuele faktore—soos publiek-private vennootskappe, gemeenskapsvertroue en gesondheidsorgtoeganklikheid—in die vorming van sukses en volhoubaarheid van sociale initiatiewe. Beperkings wat ontstaan as gevolg van databeskikbaarheid en die Gesondheidnavorsingsetiekkomitee se beperkings word erken, en aanbevelings vir toekomstige navorsing sluit uitgebreide longitudinale studies in, sowel as die insluiting van addisionele maatstawwe om die raamwerk se toepasbaarheid oor verskeie sosiale sektore te valideer. Ten slotte dra hierdie tesis by tot die veld van monitoring en evaluering deur ’n gevalideerde, teoreties gefundeerde raamwerk te bied wat die gaping oorbrug tussen monitoring en evalueringteorie en praktiese toepassing op die monitoring en evaluering van sosiale inisiatiewe. Deur sisteemdenke en realistiese evaluering in te sluit, stel die raamwerk evaluators in staat om sosiale inisiatiewe omvattend te assesseer in komplekse, werklike omgewings, wat uiteindelik meer ingeligte besluitneming ondersteun en die langtermyn-doeltreffendheid van sosiale intervensies versterk. Die aanpasbaarheid en bruikbaarheid van die raamwerk maak dit ’n waardevolle hulpmiddel vir belanghebbendes wat betrokke is by die beplanning, implementering en evaluering van sosiale inisiatiewe, veral in kontekste waar tradisionele monitoring en evaliering-benaderings moontlik tekort skiet.Master