Developing policy evaluation in an academic setting : assets and challenges

Published: 19 April 2023Based on a seminar organized by LIEPP and CIVICA which took place at Sciences Po in June 2022, this publication brings together ten academic researchers from seven different CIVICA universities (Bocconi, CEU, EUI, Hertie School, LSE, Sciences Po, SNSPA), who are involved in various forms of policy evaluation. These contributions from Austria, France, Germany, Hungary, Italy, Romania, and the United Kingdom, reflect on the assets and challenges of developing policy evaluation in an academic setting. The seminar was organized as part of CIVICA’s research focus on “Democracy in the 21st century”, but through the crosscutting nature of program evaluation, it is also of interest to CIVICA’s three others research streams (on societies in transition, data, and Europe revisited). The aim of this debate is thus to better understand the specificities, assets and challenges of developing evaluation from within an academic setting, in view of eventually reflecting on possible ways to collectively reinforce this practice within CIVICA, and use CIVICA as a leverage to reinforce this practice. This debate is organized around two topics, developing academic evaluative research, and the role of academic institutions in outreach and training in evaluation. Contributions are based on presentations of the experiences of each CIVICA partner

Identification of Genetic Factors that Modify Clinical Onset of Huntington’s Disease

SummaryAs a Mendelian neurodegenerative disorder, the genetic risk of Huntington’s disease (HD) is conferred entirely by an HTT CAG repeat expansion whose length is the primary determinant of the rate of pathogenesis leading to disease onset. To investigate the pathogenic process that precedes disease, we used genome-wide association (GWA) analysis to identify loci harboring genetic variations that alter the age at neurological onset of HD. A chromosome 15 locus displays two independent effects that accelerate or delay onset by 6.1 years and 1.4 years, respectively, whereas a chromosome 8 locus hastens onset by 1.6 years. Association at MLH1 and pathway analysis of the full GWA results support a role for DNA handling and repair mechanisms in altering the course of HD. Our findings demonstrate that HD disease modification in humans occurs in nature and offer a genetic route to identifying in-human validated therapeutic targets in this and other Mendelian disorders.PaperCli

CAG repeat not polyglutamine length determines timing of Huntington’s disease onset

Variable, glutamine-encoding, CAA interruptions indicate that a property of the uninterrupted HTT CAG repeat sequence, distinct from the length of huntingtin’s polyglutamine segment, dictates the rate at which Huntington’s disease (HD) develops. The timing of onset shows no significant association with HTT cis-eQTLs but is influenced, sometimes in a sex-specific manner, by polymorphic variation at multiple DNA maintenance genes, suggesting that the special onset-determining property of the uninterrupted CAG repeat is a propensity for length instability that leads to its somatic expansion. Additional naturally occurring genetic modifier loci, defined by GWAS, may influence HD pathogenesis through other mechanisms. These findings have profound implications for the pathogenesis of HD and other repeat diseases and question the fundamental premise that polyglutamine length determines the rate of pathogenesis in the “polyglutamine disorders.

Exome sequencing of individuals with Huntington’s disease implicates FAN1 nuclease activity in slowing CAG expansion and disease onset

The age at onset of motor symptoms in Huntington’s disease (HD) is driven by HTT CAG repeat length but modified by other genes. In this study, we used exome sequencing of 683 patients with HD with extremes of onset or phenotype relative to CAG length to identify rare variants associated with clinical effect. We discovered damaging coding variants in candidate modifier genes identified in previous genome-wide association studies associated with altered HD onset or severity. Variants in FAN1 clustered in its DNA-binding and nuclease domains and were associated predominantly with earlier-onset HD. Nuclease activities of purified variants in vitro correlated with residual age at motor onset of HD. Mutating endogenous FAN1 to a nuclease-inactive form in an induced pluripotent stem cell model of HD led to rates of CAG expansion similar to those observed with complete FAN1 knockout. Together, these data implicate FAN1 nuclease activity in slowing somatic repeat expansion and hence onset of HD

Genetic risk underlying psychiatric and cognitive symptoms in Huntington’s Disease

Background Huntington’s disease (HD) is an inherited neurodegenerative disorder caused by an expanded CAG repeat in the HTT gene. It is diagnosed following a standardized exam of motor control and often presents with cognitive decline and psychiatric symptoms. Recent studies have detected genetic loci modifying the age at onset of motor symptoms in HD, but genetic factors influencing cognitive and psychiatric presentations are unknown. Methods We tested the hypothesis that psychiatric and cognitive symptoms in HD are influenced by the same common genetic variation as in the general population by constructing polygenic risk scores from large genome-wide association studies of psychiatric and neurodegenerative disorders, and of intelligence, and testing for correlation with the presence of psychiatric and cognitive symptoms in a large sample (n=5160) of HD patients. Results Polygenic risk score for major depression was associated specifically with increased risk of depression in HD, as was schizophrenia risk score with psychosis and irritability. Cognitive impairment and apathy were associated with reduced polygenic risk score for intelligence. Conclusions Polygenic risk scores for psychiatric disorders, particularly depression and schizophrenia, are associated with increased risk of the corresponding psychiatric symptoms in HD, suggesting a common genetic liability. However, the genetic liability to cognitive impairment and apathy appears to be distinct from other psychiatric symptoms in HD. No associations were observed between HD symptoms and risk scores for other neurodegenerative disorders. These data provide a rationale for treatments effective in depression and schizophrenia to be used to treat depression and psychotic symptoms in HD

Basic science232. Certolizumab pegol prevents pro-inflammatory alterations in endothelial cell function

Background: Cardiovascular disease is a major comorbidity of rheumatoid arthritis (RA) and a leading cause of death. Chronic systemic inflammation involving tumour necrosis factor alpha (TNF) could contribute to endothelial activation and atherogenesis. A number of anti-TNF therapies are in current use for the treatment of RA, including certolizumab pegol (CZP), (Cimzia ®; UCB, Belgium). Anti-TNF therapy has been associated with reduced clinical cardiovascular disease risk and ameliorated vascular function in RA patients. However, the specific effects of TNF inhibitors on endothelial cell function are largely unknown. Our aim was to investigate the mechanisms underpinning CZP effects on TNF-activated human endothelial cells. Methods: Human aortic endothelial cells (HAoECs) were cultured in vitro and exposed to a) TNF alone, b) TNF plus CZP, or c) neither agent. Microarray analysis was used to examine the transcriptional profile of cells treated for 6 hrs and quantitative polymerase chain reaction (qPCR) analysed gene expression at 1, 3, 6 and 24 hrs. NF-κB localization and IκB degradation were investigated using immunocytochemistry, high content analysis and western blotting. Flow cytometry was conducted to detect microparticle release from HAoECs. Results: Transcriptional profiling revealed that while TNF alone had strong effects on endothelial gene expression, TNF and CZP in combination produced a global gene expression pattern similar to untreated control. The two most highly up-regulated genes in response to TNF treatment were adhesion molecules E-selectin and VCAM-1 (q 0.2 compared to control; p > 0.05 compared to TNF alone). The NF-κB pathway was confirmed as a downstream target of TNF-induced HAoEC activation, via nuclear translocation of NF-κB and degradation of IκB, effects which were abolished by treatment with CZP. In addition, flow cytometry detected an increased production of endothelial microparticles in TNF-activated HAoECs, which was prevented by treatment with CZP. Conclusions: We have found at a cellular level that a clinically available TNF inhibitor, CZP reduces the expression of adhesion molecule expression, and prevents TNF-induced activation of the NF-κB pathway. Furthermore, CZP prevents the production of microparticles by activated endothelial cells. This could be central to the prevention of inflammatory environments underlying these conditions and measurement of microparticles has potential as a novel prognostic marker for future cardiovascular events in this patient group. Disclosure statement: Y.A. received a research grant from UCB. I.B. received a research grant from UCB. S.H. received a research grant from UCB. All other authors have declared no conflicts of interes

Adjunctive rifampicin for Staphylococcus aureus bacteraemia (ARREST): a multicentre, randomised, double-blind, placebo-controlled trial.

BACKGROUND: Staphylococcus aureus bacteraemia is a common cause of severe community-acquired and hospital-acquired infection worldwide. We tested the hypothesis that adjunctive rifampicin would reduce bacteriologically confirmed treatment failure or disease recurrence, or death, by enhancing early S aureus killing, sterilising infected foci and blood faster, and reducing risks of dissemination and metastatic infection. METHODS: In this multicentre, randomised, double-blind, placebo-controlled trial, adults (≥18 years) with S aureus bacteraemia who had received ≤96 h of active antibiotic therapy were recruited from 29 UK hospitals. Patients were randomly assigned (1:1) via a computer-generated sequential randomisation list to receive 2 weeks of adjunctive rifampicin (600 mg or 900 mg per day according to weight, oral or intravenous) versus identical placebo, together with standard antibiotic therapy. Randomisation was stratified by centre. Patients, investigators, and those caring for the patients were masked to group allocation. The primary outcome was time to bacteriologically confirmed treatment failure or disease recurrence, or death (all-cause), from randomisation to 12 weeks, adjudicated by an independent review committee masked to the treatment. Analysis was intention to treat. This trial was registered, number ISRCTN37666216, and is closed to new participants. FINDINGS: Between Dec 10, 2012, and Oct 25, 2016, 758 eligible participants were randomly assigned: 370 to rifampicin and 388 to placebo. 485 (64%) participants had community-acquired S aureus infections, and 132 (17%) had nosocomial S aureus infections. 47 (6%) had meticillin-resistant infections. 301 (40%) participants had an initial deep infection focus. Standard antibiotics were given for 29 (IQR 18-45) days; 619 (82%) participants received flucloxacillin. By week 12, 62 (17%) of participants who received rifampicin versus 71 (18%) who received placebo experienced treatment failure or disease recurrence, or died (absolute risk difference -1·4%, 95% CI -7·0 to 4·3; hazard ratio 0·96, 0·68-1·35, p=0·81). From randomisation to 12 weeks, no evidence of differences in serious (p=0·17) or grade 3-4 (p=0·36) adverse events were observed; however, 63 (17%) participants in the rifampicin group versus 39 (10%) in the placebo group had antibiotic or trial drug-modifying adverse events (p=0·004), and 24 (6%) versus six (2%) had drug interactions (p=0·0005). INTERPRETATION: Adjunctive rifampicin provided no overall benefit over standard antibiotic therapy in adults with S aureus bacteraemia. FUNDING: UK National Institute for Health Research Health Technology Assessment

Albiglutide and cardiovascular outcomes in patients with type 2 diabetes and cardiovascular disease (Harmony Outcomes): a double-blind, randomised placebo-controlled trial

Background: Glucagon-like peptide 1 receptor agonists differ in chemical structure, duration of action, and in their effects on clinical outcomes. The cardiovascular effects of once-weekly albiglutide in type 2 diabetes are unknown. We aimed to determine the safety and efficacy of albiglutide in preventing cardiovascular death, myocardial infarction, or stroke. Methods: We did a double-blind, randomised, placebo-controlled trial in 610 sites across 28 countries. We randomly assigned patients aged 40 years and older with type 2 diabetes and cardiovascular disease (at a 1:1 ratio) to groups that either received a subcutaneous injection of albiglutide (30–50 mg, based on glycaemic response and tolerability) or of a matched volume of placebo once a week, in addition to their standard care. Investigators used an interactive voice or web response system to obtain treatment assignment, and patients and all study investigators were masked to their treatment allocation. We hypothesised that albiglutide would be non-inferior to placebo for the primary outcome of the first occurrence of cardiovascular death, myocardial infarction, or stroke, which was assessed in the intention-to-treat population. If non-inferiority was confirmed by an upper limit of the 95% CI for a hazard ratio of less than 1·30, closed testing for superiority was prespecified. This study is registered with ClinicalTrials.gov, number NCT02465515. Findings: Patients were screened between July 1, 2015, and Nov 24, 2016. 10 793 patients were screened and 9463 participants were enrolled and randomly assigned to groups: 4731 patients were assigned to receive albiglutide and 4732 patients to receive placebo. On Nov 8, 2017, it was determined that 611 primary endpoints and a median follow-up of at least 1·5 years had accrued, and participants returned for a final visit and discontinuation from study treatment; the last patient visit was on March 12, 2018. These 9463 patients, the intention-to-treat population, were evaluated for a median duration of 1·6 years and were assessed for the primary outcome. The primary composite outcome occurred in 338 (7%) of 4731 patients at an incidence rate of 4·6 events per 100 person-years in the albiglutide group and in 428 (9%) of 4732 patients at an incidence rate of 5·9 events per 100 person-years in the placebo group (hazard ratio 0·78, 95% CI 0·68–0·90), which indicated that albiglutide was superior to placebo (p<0·0001 for non-inferiority; p=0·0006 for superiority). The incidence of acute pancreatitis (ten patients in the albiglutide group and seven patients in the placebo group), pancreatic cancer (six patients in the albiglutide group and five patients in the placebo group), medullary thyroid carcinoma (zero patients in both groups), and other serious adverse events did not differ between the two groups. There were three (<1%) deaths in the placebo group that were assessed by investigators, who were masked to study drug assignment, to be treatment-related and two (<1%) deaths in the albiglutide group. Interpretation: In patients with type 2 diabetes and cardiovascular disease, albiglutide was superior to placebo with respect to major adverse cardiovascular events. Evidence-based glucagon-like peptide 1 receptor agonists should therefore be considered as part of a comprehensive strategy to reduce the risk of cardiovascular events in patients with type 2 diabetes. Funding: GlaxoSmithKline

31st Annual Meeting and Associated Programs of the Society for Immunotherapy of Cancer (SITC 2016) : part two

Background The immunological escape of tumors represents one of the main ob- stacles to the treatment of malignancies. The blockade of PD-1 or CTLA-4 receptors represented a milestone in the history of immunotherapy. However, immune checkpoint inhibitors seem to be effective in specific cohorts of patients. It has been proposed that their efficacy relies on the presence of an immunological response. Thus, we hypothesized that disruption of the PD-L1/PD-1 axis would synergize with our oncolytic vaccine platform PeptiCRAd. Methods We used murine B16OVA in vivo tumor models and flow cytometry analysis to investigate the immunological background. Results First, we found that high-burden B16OVA tumors were refractory to combination immunotherapy. However, with a more aggressive schedule, tumors with a lower burden were more susceptible to the combination of PeptiCRAd and PD-L1 blockade. The therapy signifi- cantly increased the median survival of mice (Fig. 7). Interestingly, the reduced growth of contralaterally injected B16F10 cells sug- gested the presence of a long lasting immunological memory also against non-targeted antigens. Concerning the functional state of tumor infiltrating lymphocytes (TILs), we found that all the immune therapies would enhance the percentage of activated (PD-1pos TIM- 3neg) T lymphocytes and reduce the amount of exhausted (PD-1pos TIM-3pos) cells compared to placebo. As expected, we found that PeptiCRAd monotherapy could increase the number of antigen spe- cific CD8+ T cells compared to other treatments. However, only the combination with PD-L1 blockade could significantly increase the ra- tio between activated and exhausted pentamer positive cells (p= 0.0058), suggesting that by disrupting the PD-1/PD-L1 axis we could decrease the amount of dysfunctional antigen specific T cells. We ob- served that the anatomical location deeply influenced the state of CD4+ and CD8+ T lymphocytes. In fact, TIM-3 expression was in- creased by 2 fold on TILs compared to splenic and lymphoid T cells. In the CD8+ compartment, the expression of PD-1 on the surface seemed to be restricted to the tumor micro-environment, while CD4 + T cells had a high expression of PD-1 also in lymphoid organs. Interestingly, we found that the levels of PD-1 were significantly higher on CD8+ T cells than on CD4+ T cells into the tumor micro- environment (p < 0.0001). Conclusions In conclusion, we demonstrated that the efficacy of immune check- point inhibitors might be strongly enhanced by their combination with cancer vaccines. PeptiCRAd was able to increase the number of antigen-specific T cells and PD-L1 blockade prevented their exhaus- tion, resulting in long-lasting immunological memory and increased median survival

Between policy failure and policy success : bricolage, experimentalism and translation in policy transfer

This chapter re-assesses some of the literature on policy transfer (see for example, Benson and Jordan, 2011; Dolowitz and Marsh, 2000; Evans and Davies, 1999) and policy diffusion (see for example, Dobbin et al, 2007; Meseguer, 2005; Shipan and Volden, 2012) in light of ideas as to what constitutes failure, partial failure or limited success. It is often presumed that when policy transfer occurs internationally, ‘best practices’ or superior standards are being transferred to recipient jurisdictions. Or, in other words, the policy success of one country is exported to another. In a more or less rational process of decision-making, importing governments recognise policy failures or shortcomings within their borders and through processes of evaluation and learning, as well as peer review, seek solutions and adopt reforms based on successful experience elsewhere. The discussion moves away from ‘orthodox’ policy transfer studies – where there is often assumed to be a motivated importer and a willing exporter country – abandoning the linear perspectives of country ‘A’ sending policy to ‘B’ and shifting to a stronger analytical focus on the messy processes of hybrid policies emerging from multiple exemplars, and the messy interpretative processes where importing countries translate and amend transferred policies. Rather than focusing on policy failure as a starting point for analysis and a prompt for reforms and policy transfer, this chapter first looks at imperfect or uninformed transfer processes as one locus of policy failure. Second, little discussed in the literature is the concept of ‘negative lesson-drawing’ which amounts to learning what not to do. Yet, widespread consensus on what to do, or not do, is often absent when stakeholders have different and divergent perspectives. The various informants intersecting with policy transfer processes at different stages also complicate a neat linear transmission of an intact policy approach or tool. Third, whether ‘transfer’ can be said to have been successfully accomplished is qualified by the extensiveness of hybridity, synthesis, tinkering with models, adaptation and ‘localisation’ (see for example, Mukhtarov, 2014; Stone, 2012).This can result in a transferred policy tool or institution from Country A looking completely different in Country B and operationalised in substantively different fashion than originally conceived. Something is either lost, or learnt, in translation