Multidimensional Signals and Analytic Flexibility: Estimating Degrees of Freedom in Human-Speech Analyses

Recent empirical studies have highlighted the large degree of analytic flexibility in data analysis that can lead to substantially different conclusions based on the same data set. Thus, researchers have expressed their concerns that these researcher degrees of freedom might facilitate bias and can lead to claims that do not stand the test of time. Even greater flexibility is to be expected in fields in which the primary data lend themselves to a variety of possible operationalizations. The multidimensional, temporally extended nature of speech constitutes an ideal testing ground for assessing the variability in analytic approaches, which derives not only from aspects of statistical modeling but also from decisions regarding the quantification of the measured behavior. In this study, we gave the same speech-production data set to 46 teams of researchers and asked them to answer the same research question, resulting in substantial variability in reported effect sizes and their interpretation. Using Bayesian meta-analytic tools, we further found little to no evidence that the observed variability can be explained by analysts’ prior beliefs, expertise, or the perceived quality of their analyses. In light of this idiosyncratic variability, we recommend that researchers more transparently share details of their analysis, strengthen the link between theoretical construct and quantitative system, and calibrate their (un)certainty in their conclusions

Pragmatic Abilities in Children with Congenital Visual Impairment: An Exploration of Non-literal Language and Advanced Theory of Mind Understanding

Contains fulltext : 102796.pdf (publisher's version ) (Closed access)Children with congenital visual impairment have been reported to be delayed in theory of mind development. So far, research focused on first-order theory of mind, and included mainly blind children, whereas the majority of visually impaired children is not totally blind. The present study set out to explore whether children with a broader range of congenital visual impairments have a delay in more advanced theory of mind understanding, in particular second-order theory of mind (i.e. awareness that other people have beliefs about beliefs) and non-literal language (e.g. irony or figure of speech). Twenty-four children with congenital visual impairment and 24 typically developing sighted children aged between 6 and 13 were included. All children were presented with a series of stories involving understanding of theory of mind and non-literal language. When compared with sighted children of similar age and verbal intelligence, performance of children with congenital visual impairment on advanced theory of mind and non-literal stories was alike. The ability to understand the motivations behind non-literal language was associated with age, verbal intelligence and theory of mind skills, but was not associated with visual ability