Use of progression criteria to support monitoring and commissioning decision making of public health services: : lessons from Better Start Bradford

BACKGROUND:Commissioning and monitoring of community-based interventions is a challenge due to the complex nature of the environment and the lack of any explicit cut-offs to guide decision making. At what point, for example, is participant enrolment to interventions, course completion or satisfaction deemed to be acceptable or sufficient for continued funding? We aimed to identify and quantify key progression criteria for fourteen early years interventions by (1) agreeing the top three criteria for monitoring of successful implementation and progress; and (2) agreeing boundaries to categorise interventions as 'meeting anticipated target' (green); 'falling short of targets' (amber) and 'targets not being met' (red). METHODS:We ran three workshops in partnership with the UK's Big Lottery Fund commissioned programme 'Better Start Bradford' (implementing more than 20 interventions to improve the health, wellbeing and development of children aged 0-3) to support decision making by agreeing progression criteria for the interventions being delivered. Workshops included 72 participants, representing a range of professional groups including intervention delivery teams, commissioners, intervention-monitoring teams, academics and community representatives. After discussion and activities, final decisions were submitted using electronic voting devices. All participants were invited to reconsider their responses via a post-workshop questionnaire. RESULTS:Three key progression criteria were assigned to each of the 14 interventions. Overall, criteria that participants most commonly voted for were recruitment, implementation and reach, but these differed according to each intervention. Cut-off values used to indicate when an intervention moved to 'red' varied by criteria; the lowest being for recruitment, where participants agreed that meeting less than 65% of the targeted recruitment would be deemed as 'red' (falling short of target). CONCLUSIONS:Our methodology for monitoring the progression of interventions has resulted in a clear pathway which will support commissioners and intervention teams in local decision making within the Better Start Bradford programme and beyond. This work can support others wishing to implement a formal system for monitoring the progression of public health interventions

Assessing and predicting adolescent and early adulthood common mental disorders using electronic primary care data:analysis of a prospective cohort study (ALSPAC) in Southwest England

OBJECTIVES: We aimed to examine agreement between common mental disorders (CMDs) from primary care records and repeated CMD questionnaire data from ALSPAC (the Avon Longitudinal Study of Parents and Children) over adolescence and young adulthood, explore factors affecting CMD identification in primary care records, and construct models predicting ALSPAC-derived CMDs using only primary care data. DESIGN AND SETTING: Prospective cohort study (ALSPAC) in Southwest England with linkage to electronic primary care records. PARTICIPANTS: Primary care records were extracted for 11 807 participants (80% of 14 731 eligible). Between 31% (3633; age 15/16) and 11% (1298; age 21/22) of participants had both primary care and ALSPAC CMD data. OUTCOME MEASURES: ALSPAC outcome measures were diagnoses of suspected depression and/or CMDs. Primary care outcome measure were Read codes for diagnosis, symptoms and treatment of depression/CMDs. For each time point, sensitivities and specificities for primary care CMD diagnoses were calculated for predicting ALSPAC-derived measures of CMDs, and the factors associated with identification of primary care-based CMDs in those with suspected ALSPAC-derived CMDs explored. Lasso (least absolute selection and shrinkage operator) models were used at each time point to predict ALSPAC-derived CMDs using only primary care data, with internal validation by randomly splitting data into 60% training and 40% validation samples. RESULTS: Sensitivities for primary care diagnoses were low for CMDs (range: 3.5%–19.1%) and depression (range: 1.6%–34.0%), while specificities were high (nearly all >95%). The strongest predictors of identification in the primary care data for those with ALSPAC-derived CMDs were symptom severity indices. The lasso models had relatively low prediction rates, especially in the validation sample (deviance ratio range: −1.3 to 12.6%), but improved with age. CONCLUSIONS: Primary care data underestimate CMDs compared to population-based studies. Improving general practitioner identification, and using free-text or secondary care data, is needed to improve the accuracy of models using clinical data

Implementation evaluation of multiple complex early years interventions: : an evaluation framework and study protocol

Introduction: Implementation evaluations are integral to understanding whether, how and why interventions work. However, unpicking the mechanisms of complex interventions is often challenging in usual service settings where multiple services are delivered concurrently. Furthermore, many locally developed and/or adapted interventions have not undergone any evaluation, thus limiting the evidence base available. Born in Bradford’s Better Start cohort is evaluating the impact of multiple early life interventions being delivered as part of the Big Lottery Fund’s ‘A Better Start’ programme to improve the health and well-being of children living in one of the most socially and ethnically diverse areas of the UK. In this paper, we outline our evaluation framework and protocol for embedding pragmatic implementation evaluation across multiple early years interventions and services. Methods and analysis: The evaluation framework is based on a modified version of The Conceptual Framework for Implementation Fidelity. Using qualitative and quantitative methods, our evaluation framework incorporates semistructured interviews, focus groups, routinely collected data and questionnaires. We will explore factors related to content, delivery and reach of interventions at both individual and wider community levels. Potential moderating factors impacting intervention success such as participants’ satisfaction, strategies to facilitate implementation, quality of delivery and context will also be examined. Interview and focus guides will be based on the Theoretical Domains Framework to further explore the barriers and facilitators of implementation. Descriptive statistics will be employed to analyse the routinely collected quantitative data and thematic analysis will be used to analyse qualitative data. Ethics and dissemination: The Health Research Authority (HRA) has confirmed our implementation evaluations do not require review by an NHS Research Ethics Committee (HRA decision 60/88/81). Findings will be shared widely to aid commissioning decisions and will also be disseminated through peer-reviewed journals, summary reports, conferences and community newsletters