Current barriers to and facilities of undertaking rigorous impact evaluations of Social Impact Bonds (SIBs) - a stakeholder-informed systematic review and thematic synthesis

Background: Social Impact Bonds (SIBs) (an investor backed form of payment-by-results), have attracted considerable political attention and capital since 2010. In the May 2010 General Election, the opposition Conservative Party included SIBs in their manifesto “Big Society, Not Big Government”. They promoted the use of SIBs as a way to achieve value for money and innovation, announcing that they would establish a social investment bank (Big Society Capital) to invest in innovative products such as SIBs (Conservative Party 2010). Since then, 89 SIBs have been launched in 20 countries, with a total investment of over 322 million USD (Social Finance 2017). Despite the political popularity, evaluation practices have differed significantly. In an initial scoping study, the reviewer identified that few SIBs have been subject to detailed evaluation. Many SIBs have not monitored performance against a robust counterfactual and there is a lack of consistency in the processes through which outcome payments are decided. Objectives: This review aimed to identify the factors that act as barriers to or facilitators of conducting or being involved in rigorous impact evaluations of the efficacy of individual (SIB-funded) programmes. Evaluation of the efficacy of SIB commissioning versus other types of commissioning was outside the scope of this review. Methods: To identify empirical literature that met the inclusion criteria, the reviewer searched 13 electronic databases, 91 grey literature sources and hand searched 33 academic articles. 92 experts were contacted, to identify additional articles. To enhance the usefulness of this research and minimise biases, this review was informed by an Expert Advisory Panel (10 experts). Prior to establishing the Panel, the project was accepted for ethical approval (CUREC_1A). Included studies needed to be primary research (any research design) or systematic reviews that focus on the design and/or implementation of SIBs and investigate barriers to or facilitators of undertaking or being involved in rigorous impact evaluations of SIBs (as identified by practitioners). Included studies were subject to quality appraisal. To generate new knowledge, a thematic synthesis of qualitative research was conducted. Barriers and facilitators were categorised into descriptive and analytical themes, which were generated inductively. Results: 5,567 studies were retrieved for title and abstract review. 5,448 studies were excluded; 119 were retrieved for full text review. Of these studies, 25 were included and 94 were excluded. Included studies reported a range of factors which act as barriers to and/or facilitators of undertaking or being involved in rigorous impact evaluations of SIBs. The most frequently reported barriers included: 1) attribution challenges (e.g small sample sizes, no theory of change, overlap of services provided etc); 2) challenges in identifying appropriate outcome metrics; 3) lack of access to required data; 4) lack of financial resources; and 5) differing stakeholder incentives, views and needs. The most frequently reported facilitators included: 1) access to expertise in evaluation, design and data management; 2) access to required data; 3) availability of comparable counterfactual/control group; 4) early and ongoing stakeholder consultation to refine and test outcome metrics; and 5) regular monitoring and performance management

Social Impact Bonds (SIBs): analyzing efficiency and legitimacy motivation using systematic review methods and social media analysis

Policymakers frequently endorse evidence-informed policy design. But, in the context of public management reform, the most technically suitable innovations are often not the ones that become widely and deeply adopted. Legitimate (i.e. socially ‘acceptable’) reforms can spread greatly within and across public sectors, despite being inefficient; and efficient reforms may nonetheless remain unimplemented if they lack the legitimacy necessary to gain support. Inefficient reforms may even become “institutionalized”, which means they are regarded as having social value beyond their technical value. At best, the result is ineffectual public management reforms, with attendant opportunity costs; at worst, poorly-targeted interventions directly harm public service delivery and policy outcomes. In between these widespread and un-spread management initiatives, however, are those reforms that gain some traction among governments, yet never really “take-off” to become fully institutionalized. Social Impact Bonds (SIBs) are a recent example of this. A SIB is a type of payment by results (PbR) contracting, in which the commissioner pays for outcomes achieved rather than services delivered, using upfront capital from private investors who earn a financial return if the intervention is successful. As of 2023, more than twelve years after the first SIB was launched in Britain, there have been 276 SIBs in 38 countries, with approximately 745 million USD raised in upfront capital and 1.7 million beneficiaries reached (GO Lab, 2023a). This growth is far less than initially expected; in recent years (from 2019 onwards), the usage of SIBs has decreased markedly. In 2021, only 19 SIBs were launched (raising approximately 25 million USD), compared to 44 SIBs in 2018 (with 54 million USD raised). Thus, the spread of SIBs has been “wide” but “thin” – i.e., to many places and sectors, but only to a very limited number of organizations or programs in each. What explains this truncated reform diffusion? Why did SIBS rapidly appear in successive jurisdictions, but penetrate those public sectors barely at all? And what does this “in-betweener” category of innovation, neither widespread nor un-spread, tell us about the combined effect of instrumental and institutional, or efficiency and legitimacy, pressures for public management reform, and the role of evidence in decision-making? The empirical chapters of this thesis suggest that this reform has been truncated and only ‘thinly’ diffused because it has often been adopted for institutional reasons. The legitimacy benefits have caused decision makers to interrogate the reform idea with lower thresholds of rigour than they otherwise would. Yet, legitimacy benefits can only propel the innovation forward so far and for so long without concrete evidence of effectiveness. Thus SIBs usage has declined, in recent years, with Development Impact Bonds (DIBs) being introduced (and gaining popularity), likely in a deliberate attempt to re-package the idea as novel. Building on seminal works in organizational sociology, this thesis shows how efficiency and legitimacy exercise varying influence over reform processes as levels of uncertainty change over time regarding the effectiveness of an innovation. I use an effectiveness systematic review of SIBs to gauge the changing state of academic knowledge about the benefits of SIBs during 2010-22, as well as a qualitative systematic review to identify broader drivers for the adoption of the reform. I perform sentiment analysis and network analysis on social media data to understand changing practitioner understanding of the reform between 2010-2020 and use framework analysis to understand how rhetorical and discursive techniques have been applied, in this same period, to frame the SIB idea as a worthwhile reform to pursue. Overall, these analyses demonstrate that legitimation is central to the early-stage diffusion of management innovations that, ultimately, spread widely but only thinly

Social Outcomes Contracting (SOC) in Social Programmes and Public Services: A Mixed-Methods Systematic Review Protocol

Background: Across a range of policy areas and geographies, governments and philanthropists are increasingly looking to adopt a social outcomes contracting (SOC) approach. Under this model, an agreement is made that a provider of services must achieve specific, measurable social and/or environmental outcomes and payments are only made when these outcomes have been achieved. Despite this growing interest, there is currently a paucity of evidence in relation to the tangible improvement in outcomes associated with the implementation of these approaches. Although promising, evidence suggests that there are risks (especially around managing perverse incentives).[1] The growing interest in SOC has been accompanied by research of specific programmes, policy domains or geographies, but there has not been a systematic attempt to synthetise this emerging evidence. To address this gap, this systematic review aims to surface the best evidence on when and where effects have been associated with SOC.  Methods: This mixed-methods systematic review protocol has been prepared using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA-P) guidelines (Additional File 2) (Shamseer et al., 2010). The review aims to consult policymakers throughout the evidence synthesis process, by adopting a user-involved research process. This will include the establishment and involvement of a Policy Advisory Group (PAG). The PAG will consist of a large, diverse, international group of policy makers who are or have been actively involved in funding and shaping social outcomes contracts (Additional File 3). The following electronic databases will be searched: ABI/INFORM Global, Applied Social Sciences Index & Abstracts (ASSIA), Scopus, International Bibliography of the Social Sciences (IBSS), PAIS Index, PolicyFile Index, Proquest Dissertations and Theses, ProQuest Social Science, Social Services Abstracts, Web of Science, Worldwide Political Science Abstracts and PsycINFO. We will also conduct a comprehensive search of grey literature sources. Studies will be imported into Covidence and screened (after de-duplication) independently by two reviewers, using explicit inclusion/exclusion criteria. We will conduct risk of bias and quality assessment using recommended tools and we will extract data using a pre-piloted, standardised data extraction form. If meta-synthesis cannot be conducted for the effectiveness component, we will carry out a descriptive narrative synthesis of the quantitative evidence, categorised by type of intervention, type of outcome/s, population characteristics and/or policy sector. The qualitative studies will be synthesised using thematic content analysis (Thomas and Harden 2008). If possible, we will also analyse the available economic data to understand the costs and benefits associated with SOC. Finally, we will conduct a cross-study synthesis, which will involve bringing together the findings from the effectiveness review, economic review and qualitative review. We recognise that the proposed conventional effectiveness review method may lead to inconclusive or partial findings given the complexity of the intervention, the likely degree of heterogeneity and the under-developed evidence base. We see a traditional systematic review as an important foundation to describe the evidence landscape. We will use this formal review as a starting point and then explore more contextually rooted review work in future. Discussion: We will use the systematic review findings to produce accessible and reliable empirical insights on whether, when, and where (and if possible, how) SOC approaches deliver improved impact when compared to more conventional funding arrangements. The outputs will support policymakers to make informed decisions in relation to commissioning and funding approaches. Systematic   review   registration: This   systematic review was registered with the International Prospective Register of Systematic Reviews (PROSPERO), on 20th November 2020 and was last updated on 21 January 2021: (registration number PROSPERO CRD42020215207). [1] A perverse incentive in an outcomes-based contract is an incentive that has unintended and undesirable results. For instance, a poorly designed welfare-to-work scheme could create incentives for service providers to prioritise clients who are easier to help and to ‘park’ those who are harder to assist (NAO 2015)

Interventions to reduce gender-based violence among young people living with or affected by HIV/AIDS in low-income and middle-income countries

Objective(s): This study explored the effectiveness of gender-based violence (GBV) interventions on young people living with or affected by HIV in low- and middle-income countries (LMICs). Design: Systematic review and meta-analysis. Methods: We pre-registered a protocol, then searched thirteen databases and grey literature. We screened randomised and quasi-experimental studies (n = 2199) of young people (aged 10–24) living with or affected by HIV in LMICs. Outcomes were GBV and/or GBV-related attitudes. We appraised the data for risk of bias and quality of evidence. Narrative syntheses and multi-level random effects meta-analyses were conducted. Results: We included 18 studies evaluating 21 interventions. Intervention arms were categorised as: a) sexual health and social empowerment (SHSE) (n = 7); b) SHSE plus economic strengthening (n = 4); c) self-defence (n = 3); d) safer schools (n = 2); e) economic strengthening only (n = 2); f) GBV sensitisation (n = 2) and g) safer schools plus parenting (n = 1). Risk of bias was moderate/high and quality of evidence low. Narrative syntheses indicated promising effects on GBV exposure, but no or mixed effects on GBV perpetration and attitudes for self-defence and GBV sensitisation interventions. Safer schools interventions showed no effects. For SHSE interventions and SHSE plus economic strengthening, meta-analysis showed a small reduction in GBV exposure but not perpetration. Economic-only interventions had no overall effect. Conclusions: SHSE, SHSE plus and self-defence and gender sensitisation interventions may be effective for GBV exposure and GBV-related attitudes but not for GBV perpetration. However, the quality of evidence is poor. Future intervention research must include both boys and girls, adolescents living with HIV and key populations.</p

Inhibition of Adaptive Immune Responses Leads to a Fatal Clinical Outcome in SIV-Infected Pigtailed Macaques but Not Vervet African Green Monkeys

African green monkeys (AGM) and other natural hosts for simian immunodeficiency virus (SIV) do not develop an AIDS-like disease following SIV infection. To evaluate differences in the role of SIV-specific adaptive immune responses between natural and nonnatural hosts, we used SIVagmVer90 to infect vervet AGM and pigtailed macaques (PTM). This infection results in robust viral replication in both vervet AGM and pigtailed macaques (PTM) but only induces AIDS in the latter species. We delayed the development of adaptive immune responses through combined administration of anti-CD8 and anti-CD20 lymphocyte-depleting antibodies during primary infection of PTM (n = 4) and AGM (n = 4), and compared these animals to historical controls infected with the same virus. Lymphocyte depletion resulted in a 1-log increase in primary viremia and a 4-log increase in post-acute viremia in PTM. Three of the four PTM had to be euthanized within 6 weeks of inoculation due to massive CMV reactivation and disease. In contrast, all four lymphocyte-depleted AGM remained healthy. The lymphocyte-depleted AGM showed only a trend toward a prolongation in peak viremia but the groups were indistinguishable during chronic infection. These data show that adaptive immune responses are critical for controlling disease progression in pathogenic SIV infection in PTM. However, the maintenance of a disease-free course of SIV infection in AGM likely depends on a number of mechanisms including non-adaptive immune mechanisms

Redefining the MED13L syndrome

Congenital cardiac and neurodevelopmental deficits have been recently linked to the mediator complex subunit 13-like protein MED13L, a subunit of the CDK8-associated mediator complex that functions in transcriptional regulation through DNA-binding transcription factors and RNA polymerase II. Heterozygous MED13L variants cause transposition of the great arteries and intellectual disability (ID). Here, we report eight patients with predominantly novel MED13L variants who lack such complex congenital heart malformations. Rather, they depict a syndromic form of ID characterized by facial dysmorphism, ID, speech impairment, motor developmental delay with muscular hypotonia and behavioral difficulties. We thereby define a novel syndrome and significantly broaden the clinical spectrum associated with MED13L variants. A prominent feature of the MED13L neurocognitive presentation is profound language impairment, often in combination with articulatory deficits

Redefining the MED13L syndrome.

Congenital cardiac and neurodevelopmental deficits have been recently linked to the mediator complex subunit 13-like protein MED13L, a subunit of the CDK8-associated mediator complex that functions in transcriptional regulation through DNA-binding transcription factors and RNA polymerase II. Heterozygous MED13L variants cause transposition of the great arteries and intellectual disability (ID). Here, we report eight patients with predominantly novel MED13L variants who lack such complex congenital heart malformations. Rather, they depict a syndromic form of ID characterized by facial dysmorphism, ID, speech impairment, motor developmental delay with muscular hypotonia and behavioral difficulties. We thereby define a novel syndrome and significantly broaden the clinical spectrum associated with MED13L variants. A prominent feature of the MED13L neurocognitive presentation is profound language impairment, often in combination with articulatory deficits