Harron, Katie

Woodman, Jenny

English

UCL Discovery

Commentwww.thelancet.com/public-health   Vol 8   July 2023 e476See Articles page e504Using administrative data to assess early-life policiesIn The Lancet Public Health, Ronan McCabe and colleagues1 show how observational data that are routinely collected when individuals interact with services can be used to build evidence when a randomised control trial is not feasible, efficient, or acceptable. The authors report that the introduction of the universal Baby Box Scheme in Scotland, UK—a national programme offering a box of essential items that doubles as a baby bed free of charge to all pregnant women—was associated with a small reduction in infant and primary carer tobacco smoke exposure in the early postnatal period, with some evidence of an increase in breastfeeding for mothers younger than 25 years, but no evidence of an effect on sleeping position or hospital admissions of the child or mother.In interpreting these findings of small and no effect, it is important to consider the challenges of using a non-randomised design and relying on administrative data for outcome measures.The first challenge is to create a suitable comparison group. McCabe and colleagues used a quasi-experimental design to compare all babies born after the introduction of the Baby Box Scheme (intervention group) with babies born before the policy (control group). This intention-to-treat interrupted time series analysis makes sense given the high uptake of the scheme by parents (>85%) and absence of data on which families did not receive the box. The authors exploited longitudinal data to account for background trends in outcomes and tested for the effect of other external events using temporal falsification and negative controls, increasing the confidence that any changes in outcomes are indeed the result of the intervention. This approach also overcomes the limitation that exposed and unexposed populations can differ in characteristics related to the outcome; crude comparisons between intervention and control groups can result in underestimation of intervention effect or even spurious associations because families might be targeted for the intervention because they are at systematically higher risk of worse outcomes (indication bias). But even when intervention and control groups are matched on some characteristics, there might be residual confounding because the data available for matching do not capture all relevant differences.2A further challenge with administrative data is the availability and interpretation of outcome measures. All-cause hospital admission is a readily available and easy-to-use proxy for maternal and child health and wellbeing, but might not be affected by a low-dose universal intervention. Additionally, differences in hospital admissions might reflect differences in either underlying health needs or health-care-seeking behaviours, making interpretation difficult. Disaggregating hospital admissions into planned and unplanned, assessing reasons for admission, and investigating emergency department attendances could provide further insight into the effect of interventions on health-care use.Despite advances made in linking health data with education and social care data across the UK,3,4 benefits of early interventions that accrue across the wider system and social determinants of health remain hard to detect—eg, those effecting housing services, access to childcare, and drug and alcohol support services. If effects are detected, they are difficult to attribute to the policy or intervention. Administrative data do not routinely measure more proximal and perhaps more appropriate outcomes that are expected to be amenable to intervention and which effect long-term outcomes, such as child–parent interaction, secure attachment, regular bedtimes, or confident parenting. Supplemental data sources are needed but, as McCabe and colleagues allude to, additional outcome data can only be collected if the evaluation is built into the roll-out of the policy.Finally, expecting to detect effects of an early-life intervention on inequalities might be unrealistic in the context of the challenges that socially disadvantaged families currently face in the UK. Increasing child poverty and adversity, and retraction of preventive services across the board, might have a much larger impact on outcomes and inequalities than can be addressed by any single early-years intervention. Cattan and colleagues suggest that even three decades of early-years reform and policy might have only been enough to stabilise inequalities in child development, given the context of growing adversity.5 Strategies to address the root causes of social disadvantage are required if inequalities are to be reduced. But beyond that, policies and interventions such as Scotland’s Baby Box Scheme can play a role in Commente477 www.thelancet.com/public-health   Vol 8   July 2023signalling values such as a commitment to equality and fairness and should perhaps be judged on these grounds, too.6McCabe and colleagues’ study is an important example of the power of administrative data, combined with robust analytical approaches, for adding to the evidence base. The challenges of retrospectively evaluating early-years policy mean that even the small effects in only some outcomes should be viewed as a promising result for Scotland’s Baby Box Scheme.We declare no competing interests. Crown Copyright © 2023 Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license. *Katie Harron, Jenny Woodman k.harron@ucl.ac.ukPopulation, Policy and Practice, UCL Great Ormond Street Institute of Child Health, London WC1N 1EH, UK (KH); Thomas Corum Research Unit, Social Research Institute, UCL Institute of Education, London, UK (JW)1 McCabe R, Katikireddi SV, Dundas R, Craig P. The health impact of Scotland’s Baby Box Scheme: a natural experiment evaluation using national linked health data. Lancet Public Health 2023; 8: e504–10. 2 Cavallaro FL, Cannings-John R, Lugg-Widger F, et al. Lessons learned from using linked administrative data to evaluate the Family Nurse Partnership in England and Scotland. Int J Popul Data Sci 2023; 8: 10.3 Mc Grath-Lone L, Libuy N, Harron K, et al. Data resource profile: the Education and Child Health Insights from Linked Data (ECHILD) database. Int J Epidemiol 2022; 51: 17–17f.4 Fleming M, Salim EE, Mackay DF, et al. Neurodevelopmental multimorbidity and educational outcomes of Scottish schoolchildren: a population-based record linkage cohort study. PLoS Med 2020; 17: e1003290.5 Cattan S, Fitzsimons E, Goodman A, Phimister A, Ploubidis GB, Wertz J. Early childhood inequalities. IFS Deaton Review of Inequalities. 2018. https://ifs.org.uk/inequality/wp-content/uploads/2022/06/Early-childhood-inequalities-IFS-Deaton-Review-2.pdf (accessed June 13, 2023).6 Scottish Government. Scotland’s Baby Box: evaluability assessment. 2018. https://www.gov.scot/publications/evaluability-assessment-scotlands-baby-box-report-scottish-government/ (accessed June 13, 2023).

Using administrative data to assess early-life policies

https://discovery.ucl.ac.uk/10173249/1/1-s2.0-S2468266723001275-main.pdf

Using administrative data to assess early-life policies

Abstract

Similar works

Full text

Available Versions

UCL Discovery