Multidimensional signals and analytic flexibility: Estimating degrees of freedom in human speech analyses

Recent empirical studies have highlighted the large degree of analytic flexibility in data analysis which can lead to substantially different conclusions based on the same data set. Thus, researchers have expressed their concerns that these researcher degrees of freedom might facilitate bias and can lead to claims that do not stand the test of time. Even greater flexibility is to be expected in fields in which the primary data lend themselves to a variety of possible operationalizations. The multidimensional, temporally extended nature of speech constitutes an ideal testing ground for assessing the variability in analytic approaches, which derives not only from aspects of statistical modeling, but also from decisions regarding the quantification of the measured behavior. In the present study, we gave the same speech production data set to 46 teams of researchers and asked them to answer the same research question, resulting insubstantial variability in reported effect sizes and their interpretation. Using Bayesian meta-analytic tools, we further find little to no evidence that the observed variability can be explained by analysts’ prior beliefs, expertise or the perceived quality of their analyses. In light of this idiosyncratic variability, we recommend that researchers more transparently share details of their analysis, strengthen the link between theoretical construct and quantitative system and calibrate their (un)certainty in their conclusions

Optimising the implementation of digital-supported interventions for the secondary prevention of heart disease: a systematic review using the RE-AIM planning and evaluation framework

Abstract Background mHealth technologies are now widely utilised to support the delivery of secondary prevention programs in heart disease. Interventions with mHealth included have shown a similar efficacy and safety to conventional programs with improvements in access and adherence. However, questions remain regarding the successful wider implementation of digital-supported programs. By applying the Reach-Effectiveness-Adoption-Implementation-Maintenance (RE-AIM) framework to a systematic review and meta-analysis, this review aims to evaluate the extent to which these programs report on RE-AIM dimensions and associated indicators. Methods This review extends our previous systematic review and meta-analysis that investigated the effectiveness of digital-supported programs for patients with coronary artery disease. Citation searches were performed on the 27 studies of the systematic review to identify linked publications that reported data for RE-AIM dimensions. All included studies and, where relevant, any additional publications, were coded using an adapted RE-AIM extraction tool. Discrepant codes were discussed amongst reviewers to gain consensus. Data were analysed to assess reporting on indicators related to each of the RE-AIM dimensions, and average overall reporting rates for each dimension were calculated. Results Searches found an additional nine publications. Across 36 publications that were linked to the 27 studies, 24 (89%) of the studies were interventions solely delivered at home. The average reporting rates for RE-AIM dimensions were highest for effectiveness (75%) and reach (67%), followed by adoption (54%), implementation (36%) and maintenance (11%). Eleven (46%) studies did not describe relevant characteristics of their participants or of staff involved in the intervention; most studies did not describe unanticipated consequences of the intervention; the ongoing cost of intervention implementation and maintenance; information on intervention fidelity; long-term follow-up outcomes, or program adaptation in other settings. Conclusions Through the application of the RE-AIM framework to a systematic review we found most studies failed to report on key indicators. Failing to report these indicators inhibits the ability to address the enablers and barriers required to achieve optimal intervention implementation in wider settings and populations. Future studies should consider alternative hybrid trial designs to enable reporting of implementation indicators to improve the translation of research evidence into routine practice, with special consideration given to the long-term sustainability of program effects as well as corresponding ongoing costs. Registration PROSPERO—CRD42022343030