Program Evaluation of Population- and System-Level Policies: Evidence for Decision Making

BACKGROUND: Policy evaluations often focus on ex post estimation of causal effects on short-term surrogate outcomes. The value of such information is limited for decision making, as the failure to reflect policy-relevant outcomes and disregard for opportunity costs prohibits the assessment of value for money. Further, these evaluations do not always consider all relevant evidence, other courses of action, or decision uncertainty. METHODS: In this article, we explore how policy evaluation could better meet the needs of decision making. We begin by defining the evidence required to inform decision making. We then conduct a literature review of challenges in evaluating policies. Finally, we highlight potential methods available to help address these challenges. RESULTS: The evidence required to inform decision making includes the impacts on the policy-relevant outcomes, the costs and associated opportunity costs, and the consequences of uncertainty. Challenges in evaluating health policies are described using 8 categories: 1) valuation space; 2) comparators; 3) time of evaluation; 4) mechanisms of action; 5) effects; 6) resources, constraints, and opportunity costs; 7) fidelity, adaptation, and level of implementation; and 8) generalizability and external validity. Methods from a broad set of disciplines are available to improve policy evaluation, relating to causal inference, decision-analytic modeling, theory of change, realist evaluation, and structured expert elicitation. LIMITATIONS: The targeted review may not identify all possible challenges, and the methods covered are not exhaustive. CONCLUSIONS: Evaluations should provide appropriate evidence to inform decision making. There are challenges in evaluating policies, but methods from multiple disciplines are available to address these challenges. IMPLICATIONS: Evaluators need to carefully consider the decision being informed, the necessary evidence to inform it, and the appropriate methods.[Box: see text]

Linear and nonlinear coupling between transverse modes of a nanomechanical resonator

We measure both the linear and nonlinear coupling between transverse modes in a nanomechanical resonator. The nonlinear coupling is due to the displacement dependent tension of the resonator and leads to a frequency shift (“pulling”) of each mode proportional to the square of the orthogonal mode's displacement amplitude. The linear coupling is apparent as an avoided crossing of the resonant frequencies that occurs when one electrostatically tunes the modes into degeneracy via a nearby DC gate. We consider the possibility that the linear coupling results from an electrostatic interaction and find that this effect can only partially explain the magnitude of the observed coupling. By measuring the coupled amplitudes magnetomotively at various angles to the applied field, we find that as the modes are tuned through the degeneracy point, they remain linearly polarized, while their planes of vibration rotate by 90°

Spin relaxation in (110) and (001) InAs/GaSb superlattices

We report an enhancement of the electron spin relaxation time (T1) in a (110) InAs/GaSb superlattice by more than an order of magnitude (25 times) relative to the corresponding (001) structure. The spin dynamics were measured using polarization sensitive pump probe techniques and a mid-infrared, subpicosecond PPLN OPO. Longer T1 times in (110) superlattices are attributed to the suppression of the native interface asymmetry and bulk inversion asymmetry contributions to the precessional D'yakonov Perel spin relaxation process. Calculations using a nonperturbative 14-band nanostructure model give good agreement with experiment and indicate that possible structural inversion asymmetry contributions to T1 associated with compositional mixing at the superlattice interfaces may limit the observed spin lifetime in (110) superlattices. Our findings have implications for potential spintronics applications using InAs/GaSb heterostructures.Comment: 4 pages, 2 figure

Potential value of identifying type 2 diabetes subgroups for guiding intensive treatment: a comparison of novel data-driven clustering with risk-driven subgroups

OBJECTIVETo estimate the impact on lifetime health and economic outcomes of different methods of stratifying individuals with type 2 diabetes, followed by guideline-based treatment intensification targeting BMI and LDL in addition to HbA1c.RESEARCH DESIGN AND METHODSWe divided 2,935 newly diagnosed individuals from the Hoorn Diabetes Care System (DCS) cohort into five Risk Assessment and Progression of Diabetes (RHAPSODY) data-driven clustering subgroups (based on age, BMI, HbA1c, C-peptide, and HDL) and four risk-driven subgroups by using fixed cutoffs for HbA1c and risk of cardiovascular disease based on guidelines. The UK Prospective Diabetes Study Outcomes Model 2 estimated discounted expected lifetime complication costs and quality-adjusted life-years (QALYs) for each subgroup and across all individuals. Gains from treatment intensification were compared with care as usual as observed in DCS. A sensitivity analysis was conducted based on Ahlqvist subgroups.RESULTSUnder care as usual, prognosis in the RHAPSODY data-driven subgroups ranged from 7.9 to 12.6 QALYs. Prognosis in the risk-driven subgroups ranged from 6.8 to 12.0 QALYs. Compared with homogenous type 2 diabetes, treatment for individuals in the high-risk subgroups could cost 22.0% and 25.3% more and still be cost effective for data-driven and risk-driven subgroups, respectively. Targeting BMI and LDL in addition to HbA1c might deliver up to 10-fold increases in QALYs gained.CONCLUSIONSRisk-driven subgroups better discriminated prognosis. Both stratification methods supported stratified treatment intensification, with the risk-driven subgroups being somewhat better in identifying individuals with the most potential to benefit from intensive treatment. Irrespective of stratification approach, better cholesterol and weight control showed substantial potential for health gains.Molecular Epidemiolog

Automated virtual reality therapy to treat agoraphobic avoidance and distress in patients with psychosis (gameChange): a multicentre, parallel-group, single-blind, randomised, controlled trial in England with mediation and moderation analyses

Background: Automated delivery of psychological therapy using immersive technologies such as virtual reality (VR) might greatly increase the availability of effective help for patients. We aimed to evaluate the efficacy of an automated VR cognitive therapy (gameChange) to treat avoidance and distress in patients with psychosis, and to analyse how and in whom it might work. // Methods: We did a parallel-group, single-blind, randomised, controlled trial across nine National Health Service trusts in England. Eligible patients were aged 16 years or older, with a clinical diagnosis of a schizophrenia spectrum disorder or an affective diagnosis with psychotic symptoms, and had self-reported difficulties going outside due to anxiety. Patients were randomly assigned (1:1) to either gameChange VR therapy plus usual care or usual care alone, using a permuted blocks algorithm with randomly varying block size, stratified by study site and service type. gameChange VR therapy was provided in approximately six sessions over 6 weeks. Trial assessors were masked to group allocation. Outcomes were assessed at 0, 6 (primary endpoint), and 26 weeks after randomisation. The primary outcome was avoidance of, and distress in, everyday situations, assessed using the self-reported Oxford Agoraphobic Avoidance Scale (O-AS). Outcome analyses were done in the intention-to-treat population (ie, all participants who were assigned to a study group for whom data were available). We performed planned mediation and moderation analyses to test the effects of gameChange VR therapy when added to usual care. This trial is registered with the ISRCTN registry, 17308399. // Findings: Between July 25, 2019, and May 7, 2021 (with a pause in recruitment from March 16, 2020, to Sept 14, 2020, due to COVID-19 pandemic restrictions), 551 patients were assessed for eligibility and 346 were enrolled. 231 (67%) patients were men and 111 (32%) were women, 294 (85%) were White, and the mean age was 37·2 years (SD 12·5). 174 patients were randomly assigned to the gameChange VR therapy group and 172 to the usual care alone group. Compared with the usual care alone group, the gameChange VR therapy group had significant reductions in agoraphobic avoidance (O-AS adjusted mean difference –0·47, 95% CI –0·88 to –0·06; n=320; Cohen's d –0·18; p=0·026) and distress (–4·33, –7·78 to –0·87; n=322; –0·26; p=0·014) at 6 weeks. Reductions in threat cognitions and within-situation defence behaviours mediated treatment outcomes. The greater the severity of anxious fears and avoidance, the greater the treatment benefits. There was no significant difference in the occurrence of serious adverse events between the gameChange VR therapy group (12 events in nine patients) and the usual care alone group (eight events in seven patients; p=0·37). // Interpretation: Automated VR therapy led to significant reductions in anxious avoidance of, and distress in, everyday situations compared with usual care alone. The mediation analysis indicated that the VR therapy worked in accordance with the cognitive model by reducing anxious thoughts and associated protective behaviours. The moderation analysis indicated that the VR therapy particularly benefited patients with severe agoraphobic avoidance, such as not being able to leave the home unaccompanied. gameChange VR therapy has the potential to increase the provision of effective psychological therapy for psychosis, particularly for patients who find it difficult to leave their home, visit local amenities, or use public transport. // Funding: National Institute of Health Research Invention for Innovation programme, National Institute of Health Research Oxford Health Biomedical Research Centre

Appraisal of patient-level health economic models of severe mental illness: systematic review

Background Healthcare decision makers require accurate long-term economic models to evaluate the cost-effectiveness of new mental health interventions. Aims To assess the suitability of current patient-level economic models to estimate long-term economic outcomes in severe mental illness. Method We undertook pre-specified systematic searches in MEDLINE, Embase and PsycINFO to identify reviews and stand-alone publications of economic models of interventions for schizophrenia, bipolar disorder and major depressive disorder (PROSPERO: CRD42020158243). We screened paper titles and abstracts to identify unique patient-level economic models. We conducted a structured extraction of identified models, recording the presence of key predefined model features. Model quality and validation were appraised using the 2014 ISPOR and 2016 AdViSHE model checklists. Results We identified 15 unique patient-level models for psychosis and major depressive disorder from 1481 non-duplicate records. Models addressed schizophrenia (n = 6), bipolar disorder (n = 2) and major depressive disorder (n = 7). The predominant model type was discrete event simulation (n = 9). Model complexity and incorporation of patient heterogeneity varied considerably, and only five models extrapolated costs and outcomes over a lifetime horizon. Key model parameters were often based on low-quality evidence, and checklist quality assessment revealed weak model verification procedures. Conclusions Existing patient-level economic models of interventions for severe mental illness have considerable limitations. New modelling efforts must be supplemented by the generation of good-quality, contemporary evidence suitable for model building. Combined effort across the research community is required to build and validate economic extrapolation models suitable for accurately assessing the long-term value of new interventions from short-term clinical trial data