Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: A systematic review and meta-analysis

Background: Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. “Risk of generalizability biases (RGB)” in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity. Methods: We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes. Results: A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of − 0.325 (95CI − 0.556 to − 0.094), − 0.346 (− 0.640 to − 0.052), − 0.342 (− 0.498 to − 0.187), and − 0.360 (− 0.631 to − 0.089), respectively. Conclusions Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation

Converting between estimates of moderate-to-vigorous physical activity derived from raw accelerations measured at the wrist and from ActiGraph counts measured at the hip: The Rosetta Stone

The ability to compare published group-level estimates of objectively measured moderate-to-vigorous physical activity (MVPA) across studies continues to increase in difficulty. The objective of this study was to develop conversion equations and demonstrate their utility to compare estimates of MVPA derived from the wrist and hip. Data from three studies of youth (N = 232, 9-12yrs, 50% boys) who concurrently wore a hip-worn ActiGraph and a wrist-worn GENEActiv for 7-days. ActiGraph hip count data were reduced using four established cutpoints: Evenson, Pate, Puyau, and Freedson 3MET. Wrist accelerations were reduced using the Hildebrand MVPA 200 mg threshold. Conversion equations were developed on a randomly selected subsample of 132 youth. Equations were cross-validated and absolute error, absolute percent error, and modified Bland-Altman plots were evaluated for conversion accuracy. Across equations R2adj was 0.51-0.56 with individual-level absolute error in minutes ranging from 7 (wrist-to-hip Puyau) to 14.5 minutes (wrist-to-hip Freedson 3MET) and absolute percent differences ranging from 13.9%-24.5%. Group-level cross-validation to convert hip-to-wrist MVPA resulted in average absolute percent errors ranging from 3.1%-4.9%. Conversion of wrist- to-hip MVPA resulted in average absolute percent errors ranging from 3.0%-10.0%. We recommend the use of these equations to compare published estimates of MVPA between the wear-site cut-point combinations presented

The theory of expanded, extended, and enhanced opportunities for youth physical activity promotion

Background Physical activity interventions targeting children and adolescents (≤18 years) often focus on complex intra- and inter-personal behavioral constructs, social-ecological frameworks, or some combination of both. Recently published meta-analytical reviews and large-scale randomized controlled trials have demonstrated that these intervention approaches have largely produced minimal or no improvements in young people\u27s physical activity levels. Discussion In this paper, we propose that the main reason for previous studies\u27 limited effects is that fundamental mechanisms that lead to change in youth physical activity have often been overlooked or misunderstood. Evidence from observational and experimental studies is presented to support the development of a new theory positing that the primary mechanisms of change in many youth physical activity interventions are approaches that fall into one of the following three categories: (a) the expansion of opportunities for youth to be active by the inclusion of a new occasion to be active, (b) the extension of an existing physical activity opportunity by increasing the amount of time allocated for that opportunity, and/or (c) the enhancement of existing physical activity opportunities through strategies designed to increase physical activity above routine practice. Their application and considerations for intervention design and interpretation are presented. Summary The utility of these mechanisms, referred to as the Theory of Expanded, Extended, and Enhanced Opportunities (TEO), is demonstrated in their parsimony, logical appeal, support with empirical evidence, and the direct and immediate application to numerous settings and contexts. The TEO offers a new way to understand youth physical activity behaviors and provides a common taxonomy by which interventionists can identify appropriate targets for interventions across different settings and contexts. We believe the formalization of the TEO concepts will propel them to the forefront in the design of future intervention studies and through their use, lead to a greater impact on youth activity behaviors than what has been demonstrated in previous studies

Effect and cost of an after-school dance programme on the physical activity of 11-12 year old girls: The Bristol Girls Dance Project, a school-based cluster randomised controlled trial

© 2015 Jago et al. Background: The aim of this study was to examine the effectiveness and cost of an after-school dance intervention at increasing the physical activity levels of Year 7 girls (age 11-12). Methods: A cluster randomised controlled trial was conducted in 18 secondary schools. Participants were Year 7 girls attending a study school. The Bristol Girls Dance Project (BGDP) intervention consisted of up to forty, 75-minute dance sessions delivered in the period immediately after school by experienced dance instructors over 20-weeks. The pre-specified primary outcome was accelerometer assessed mean minutes of weekday moderate to vigorous physical activity (MVPA) at time 2 (52weeks are T0 baseline assessments). Secondary outcomes included accelerometer assessed mean minutes of weekday MVPA at time 1 (while the intervention was still running) and psychosocial outcomes. Intervention costs were assessed. Results: 571 girls participated. Valid accelerometer data were collected from 549 girls at baseline with 508 girls providing valid accelerometer data at baseline and time 2. There were no differences between the intervention and control group for accelerometer assessed physical activity at either time 1 or time 2. Only one third of the girls in the intervention arm met the pre-set adherence criteria of attending two thirds of the dance sessions that were available to them. Instrumental variable regression analyses using complier average causal effects provided no evidence of a difference between girls who attended the sessions and the control group. The average cost of the intervention was £73 per girl, which was reduced to £63 when dance instructor travel expenses were excluded. Conclusion: This trial showed no evidence that an after-school dance programme can increase the physical activity of Year 7 girls. The trial highlighted the difficulty encountered in maintaining attendance in physical activity programmes delivered in secondary schools. There is a need to find new ways to help adolescent girls to be physically active via identifying ways to support and encourage sustained engagement in physical activity over the life course. Trial registration: ISRCTN5288252

Association of self-reported physical activity patterns and socio-demographic factors among normal-weight and overweight Japanese men

<p>Abstract</p> <p>Background</p> <p>It is still not known whether overweight men have different patterns and socio-demographic correlates of self-reported physical activity (PA) compared with normal-weight men. Thus, this study examined the perceived PA patterns and associated socio-demographic factors among normal-weight and overweight Japanese men.</p> <p>Methods</p> <p>Data were analyzed for 1,420 men (aged 44.48.3years) who responded to an Internet-based cross-sectional survey relating to socio-demographic variables, BMI status, and a short version of the International Physical Activity Questionnaire. Mann-Whitney, chi-square, and binary logistic regression analyses were employed.</p> <p>Results</p> <p>Normal-weight men were significantly more likely to attain 150 minutes per week of moderate-to-vigorous PA than overweight men (26.6% vs. 21.3%; <it>p</it>=0.035), whereas there were no significant proportional differences in total PA and walking between the two BMI subgroups. With PA, a significant interaction was observed between BMI status and household income (<it>p</it>=0.004 for total PA; <it>p</it>=0.02 for walking). In the subgroup analyses, having a lower household income (odds ratio, 0.63; 95% confidence interval, 0.41-0.96) was negatively associated with attaining 150 minutes of walking per week among normal-weight men. No significant associations between household income and attaining 150 minutes per week of total PA and walking were found among overweight men.</p> <p>Conclusions</p> <p>The results revealed that patterns and socio-demographic correlates of self-reported PA in overweight men are different from those in normal-weight men. This finding suggests the necessity of developing specific strategies for PA intervention among overweight men. Socio-demographic correlates of PA may be more important for normal-weight than overweight men.</p

Randomised feasibility trial of a teaching assistant led extracurricular physical activity intervention for 9 to 11 year olds: Action 3:30

Background: Extracurricular programmes could provide a mechanism to increase the physical activity (PA) of primary-school-aged children. The aim of this feasibility study was to examine whether the Action 3:30 intervention, which is delivered by teaching assistants, holds promise as a means of increasing the PA of Year 5 and 6 children. Methods: A cluster randomised feasibility trial was conducted in 20 primary schools. Ten schools received the Action 3:30 intervention and 10 schools were allocated to the control arm. The intervention was 40 one-hour sessions, delivered twice a week by teaching assistants. The proportion of participants recruited per school was calculated. Session delivery and session attendance was calculated for intervention schools. Weekday and after-school (3.30 to 8.30 pm) moderate to vigorous intensity physical (MVPA) was assessed by accelerometer at baseline (T0), during the last few weeks of the intervention (T1) and four months after the intervention had ended (T2). The costs of delivering the intervention were estimated. Results: Five intervention schools ran all 40 of the intended sessions. Of the remaining five, three ran 39, one ran 38 and one ran 29 sessions. Mean attendance was 53%. The adjusted difference in weekday MVPA at T1 was 4.3 minutes (95% CI −2.6 to 11.3). Sex-stratified analyses indicated that boys obtained 8.6 more minutes of weekday MVPA than the control group (95% CI 2.8 to 14.5) at T1 with no effect for girls (0.15 minutes, 95% CI −9.7 to 10.0). There was no evidence that participation in the programme increased MVPA once the club sessions ceased (T2). The indicative average cost of this intervention was £2,425 per school or £81 per participating child during its first year and £1,461 per school or £49 per participating child thereafter. Conclusions: The effect of the Action 3:30 intervention was comparable to previous physical activity interventions but further analysis indicated that there was a marked sex difference with a positive impact on boys and no evidence of an effect on girls. The Action 3:30 intervention holds considerable promise but more work is needed to enhance the effectiveness of the intervention, particularly for girls