Multidimensional signals and analytic flexibility: Estimating degrees of freedom in human speech analyses

Recent empirical studies have highlighted the large degree of analytic flexibility in data analysis which can lead to substantially different conclusions based on the same data set. Thus, researchers have expressed their concerns that these researcher degrees of freedom might facilitate bias and can lead to claims that do not stand the test of time. Even greater flexibility is to be expected in fields in which the primary data lend themselves to a variety of possible operationalizations. The multidimensional, temporally extended nature of speech constitutes an ideal testing ground for assessing the variability in analytic approaches, which derives not only from aspects of statistical modeling, but also from decisions regarding the quantification of the measured behavior. In the present study, we gave the same speech production data set to 46 teams of researchers and asked them to answer the same research question, resulting insubstantial variability in reported effect sizes and their interpretation. Using Bayesian meta-analytic tools, we further find little to no evidence that the observed variability can be explained by analysts’ prior beliefs, expertise or the perceived quality of their analyses. In light of this idiosyncratic variability, we recommend that researchers more transparently share details of their analysis, strengthen the link between theoretical construct and quantitative system and calibrate their (un)certainty in their conclusions

A Meta-analysis of the Executive Function components Inhibition, Shifting, and Attention in Intellectual Disabilities

This is a preprint of a manuscript that is currently under review. Analysis code and data can be found in the accompanying OSF project https://osf.io/njsd3/. Background Executive function is a concept for higher-order cognitive functions which have the role of controller and modulator of cognitive abilities. The consensus in the literature is that people with an intellectual disability perform significantly lower on executive function tasks than groups matched on chronological age. The comparison with groups matched on mental age is less clear. Therefore, the objective of this meta-analysis was to investigate to what extent executive function is impaired in people with intellectual disability compared to a typically developing control group matched on mental age. It was also investigated if the executive function component and intellectual disability aetiology moderated the effect. Methods Eligibility criteria were participants with intellectual disability (IQ < 75) without a dual diagnosis; a comparison group matched on mental age; executive function outcome reported in a group comparison study design with n > 10. Working memory tasks and ratings of executive function were not included. The literature search yielded 6637 potentially interesting articles. 26 studies (with 99 effect sizes) including 1395 participants were included in the quantitative synthesis. Results A multilevel random-effects meta-analysis found that people with intellectual disability performed statistically significantly lower than the mental age matched group on the executive function tasks, g = -0.34, CI 95%= [-0.53,- 0.16]. However, the heterogeneity between effect sizes was large. The intellectual disability aetiology moderator was significant, but it only reduced the heterogeneity marginally. Conclusion The overall conclusion is that individuals with an intellectual disability have more problems with executive function tasks than mental age-matched controls. Limitations are the large unexplained variance and the remarkably high number (69) of different tests that were used, which make more detailed conclusions problematic. This meta-analysis implies that future studies need to be of better quality, have higher power, and to a higher degree use the same executive function tests

Interactions between autistic adults offer a new perspective on social gaze

Face-to-face communication is highly complex, with information being transmitted via multiple channels simultaneously. Social gaze can regulate conversation, express emotions, and signal interest or disinterest, and eye contact, or a lack thereof, is a powerful visual cue that influences the dynamics of communication. While previous research has shed light on gaze in autism in general, there remains a lack of 1) evidence on interactions in dialogue between autistic adults (rather than mixed dialogues) and 2) investigations on the influence of gaze on conversational dynamics and interpersonal rapport. We have developed a novel setup with mobile dual eye-tracking glasses that allows for the automatic detection of mutual eye contact. Our exploratory analyses of conversations in homogeneous autistic dyads provide new insights into autistic gaze dynamics and their interrelation with rapport, ultimately helping to advance the current understanding of cognitive diversity and of the fundamental elements of social interaction

The relation between gaze and turn-taking in dyadic avatar-mediated conversations

Human communication comprises a complex and dynamic interplay of verbal and nonverbal communication channels. The investigation of these channels therefore represents a major methodological challenge. Technical developments in interaction platforms using virtual characters provide tools for these investigations. Paradigms in which participants interact with algorithmically controlled agents have already enabled the investigation of individual nonverbal communication channels with the necessary experimental control. However, it is unclear how these results relate to human-human communication. Here, we present a study with a new system for human-human interactions mediated by avatars. As a proof-of-concept, we tested the generalisability of gaze patterns during turn-taking in avatar-mediated conversations. Results show that given our Bayesian mixed effects model, priors and data, there is compelling evidence that gaze patterns are comparable to natural interactions. Exploratory analyses show that our system is suitable to shed light on variability in individual-specific gaze behaviour, which we plan to investigate further

Meta-Analysis and Open Science summer school 2017

Meta-analysis and Open science summer school arranged by Henrik Danielsson, The Swedish Institute for Disability Research. Location: Mjölby Stadshotell, Mjölby, Sweden. Dates: 12-16th of August 201