Measuring facilitator competent adherence and examining its role in the outcomes of parenting programme beneficiaries: an investigation of the broader literature and the delivery of parenting for lifelong health for parents and adolescents (PLH-teens) at scale in Tanzania

Abstract

Background: Implementation fidelity is a critical component of intervention science research, which aims to understand how interventions unfold in practice to improve their outcomes. A key element of fidelity is facilitator competent adherence - the extent to which a programme is delivered as prescribed with the specified level of quality. The dissertation endeavoured to better understand how to measure facilitator competent adherence and the role facilitator competent adherence plays in achieving intended parent/caregiver (parent) and child outcomes in the parenting programme literature and, specifically, within Parenting for Lifelong Health for Parents and Adolescents (PLH-Teens). PLH-Teens is a parenting programme designed to reduce violence against children and child behavioural and emotional problems in low- and middle-income countries (LMICs). The dissertation is composed of three studies – one which synthesised data from the parenting programme literature and two which analysed data from the 2020-2021 scale-up of PLH-Teens in Tanzania to 75,061 participants by community facilitators (school teachers and community health workers; N=444). Objectives: The dissertation had three objectives with each corresponding to an individual paper. The first objective was to synthesise the evidence on the relationship between observational measures of facilitator competent adherence and parent and child outcomes in the parenting programme literature. The second objective was to examine whether the observational measure of facilitator competent adherence used in the large-scale implementation of PLH-Teens in Tanzania is reliable and valid for use in research and practice and to determine the level of competent adherence with which community facilitators delivered PLH-Teens in Tanzania. The third objective was to determine the predictive validity of the observational measure of competent adherence used in PLH-Teens by examining whether competence adherence is associated with parent and adolescent outcomes. Methods: Paper 1 synthesised the results of a systematic review of studies on parenting programmes aiming to reduce violence against children and child behavioural and emotional problems to examine the associations between observational measures of facilitator competent adherence and parent and child outcomes. Due to study heterogeneity and poor reporting, Synthesis Without Meta-Analysis (SWiM) guidelines were followed. Paper 2 used 95 facilitator assessments collected by implementing partners during the 2020-2021 delivery of PLH-Teens in Tanzania. The paper evaluated the reliability and validity of the measure used to assess facilitator competent adherence in PLH-Teens - the Facilitator Assessment Tool (PLH-FAT-T). Reliability was assessed by conducting intra-rater reliability, inter-rater reliability, and internal consistency analyses using percentage agreements, intra-class correlations, Cronbach’s alphas, and omegas. Validity was assessed via consultations with stakeholders (content validity) and exploratory factor analyses (construct validity). This paper also estimated the level of competent adherence with which community facilitators delivered PLH-Teens by calculating the average PLH-FAT-T score achieved by facilitators. Paper 3 investigated the relationship between facilitator competent adherence and the pre-post outcomes of PLH-Teens participants. Analyses used 24 PLH-FAT-T assessments that could be linked to the pre-post surveys of 3,057 families. This analysis was conducted using multi-level Poisson regressions with fixed and random effects. Results: Paper 1 found 18 studies reporting on the relationship between observational measures of facilitator competent adherence and parent and child outcomes. The review found that most studies (n=13) reported a statistically significant positive relationship with at least one of the parent or child outcomes reported. However, eight studies reported inconsistent findings across outcomes. Four studies found no significant association with outcomes. Paper 2 found that the PLH-FAT-T showed strong content validity, poor to moderate intra- and inter-rater reliability, strong internal consistency, and moderate construct validity. Iterative exploratory factor analyses produced a shortened PLH-FAT-T, the PLH-FAT-T Short Form, comprised of 19 fewer items which had stronger psychometric properties. Analyses of the PLH-FAT-T Short Form found that community facilitators delivered PLH-Teens at scale in Tanzania to a high-level of competent adherence (82.3% average). Using the PLH-FAT-T Short Form, Paper 3 found that the relationship between facilitator competent adherence and outcomes was mixed with some positive, some insignificant, and some negative associations. A positive association was found between competent adherence and the primary outcome of interest, child maltreatment, as reported by adolescents. The analysis found that increased competent adherence had a positive association with two of the 12 parent-reported outcomes and seven of the 10 adolescent-reported outcomes (including child maltreatment). Yet, increased competent adherence also had a negative association with five parent-reported outcomes, as well as insignificant associations with five parent-reported outcomes and three adolescent-reported outcomes. Discussion: Paper 1 suggests that better facilitator competent adherence is generally associated with positive parent and child outcomes. However, this finding is weakened by the methodological heterogeneity of included studies and due to the wide variety of ways in which studies conceptualised competent adherence-outcome relationships. As a result, the paper reveals that there is substantial methodological work to be done in the broader parenting programme community to improve the rigour of and reporting on investigations regarding this relationship. As the amount of literature on the measurement and role of facilitator competent adherence grows in the behavioural intervention literature, the recommendations made in Paper 1 have relevance for other implementation scientists conducting and sharing studies on competent adherence. Paper 2 reports on the first psychometric evaluation of the PLH-FAT-T and is the first study of its kind to report on the fidelity achieved by facilitators during routine parenting programme delivery at scale in a low-income country. Findings suggest that the PLH-FAT-T had poor to moderate reliability and sufficient validity and that the PLH-FAT-T Short Form had stronger psychometric properties. Although the tool was stronger following iterative exploratory factor analyses, the findings indicate that further work is needed to strengthen the reliability and validity of the PLH-FAT-T Short Form. Findings also suggest that community facilitators with minimal background in and training on parenting programmes delivered PLH-Teens to a high level of quality at scale in a low-income community setting despite significant barriers. Thus, the findings of Paper 2 suggest that it may be possible for community facilitators to deliver behavioural interventions to a high level of competent adherence in low-income routine delivery settings at scale. The findings of Paper 3 are similar to the findings of Paper 1 in that Paper 3 does not provide a clear answer as to whether, and to what extent, facilitator competent adherence impacts participant outcomes. Potential explanations of the findings include the PLH-FAT-T Short Form has poor predictive validity; the PLH-FAT-T Short Form assessments were not reliable; a variety of methodological challenges may have prevented an examination of the true relationship between competent adherence and outcomes; competent adherence does not relate to outcomes in the manner theorised; competent adherence plays a less important role in the achievement of outcomes than anticipated or, at some point, plays a negative role; and only certain programme components are achieving outcomes so the PLH-FAT-T Short Form is not capturing the important aspects of programme delivery. The alignment of the findings of Papers 1 and 3 with some other systematic reviews and meta-analyses in the broader implementation science literature suggests that the role facilitator competent adherence plays in participant outcomes is not fully understood. Thus, there is reason to further investigate the theorised relationship between facilitator competent adherence and outcomes outlined in seminal implementation science theories and models to fully illuminate the inner workings of the ‘black box’ of interventions. A fuller understanding of the role that facilitator competent adherence plays in participant outcomes is essential to maximise the benefits to be reaped from evidence-based behavioural interventions. Conclusion: The dissertation provides important evidence regarding the measurement and role of facilitator competent adherence in the parenting programme literature and in Parenting for Lifelong Health. As a result, the dissertation provides a series of recommendations for the future of competent adherence monitoring in research and practice that are relevant to both the parenting programme literature and the broader implementation science literature. As parenting programmes continue to be delivered and scaled worldwide, it is intended that the findings and recommendations herein will be used to benefit both Parenting for Lifelong Health and the broader parenting programme community in the quest to maximise opportunity for vulnerable children and families globally to benefit from evidence-based parenting programmes