CNN Based Touch Interaction Detection for Infant Speech Development

In this paper, we investigate the detection of interaction in videos between two people, namely, a caregiver and an infant. We are interested in a particular type of human interaction known as touch, as touch is a key social and emotional signal used by caregivers when interacting with their children. We propose an automatic touch event recognition method to determine the potential time interval when the caregiver touches the infant. In addition to label the touch events, we also classify them into six touch types based on which body part of infant has been touched. CNN based human pose estimation and person segmentation are used to analyze the spatial relationship between the caregivers hands and the infants. We demonstrate promising results for touch detection and show great potential of reducing human effort in manually generating precise touch annotations

Vocal Rhythm Coordination and Preterm Infants: Rhythms of Dialogue in a High-Risk NICU Sample

The contemporary bio-psycho-social view of mother-infant relationships holds that early interactions form the foundation of the growing infant’s sense of himself and the world. Prior to the development of linguistically-based communication, nonverbal communication patterns foster the infant’s socio-emotional growth, cognitive capacity and the development of optimal regulatory patterns. Preterm birth significantly alters the typical developmental trajectory on multiple levels and disrupts normal neurobiological and socio-emotional maturational processes, including those that build on early interpersonal experiences with caregivers. The current study of vocal rhythm coordination in preterm mother-infant dyads is the first of its kind. Aspects of infant prematurity (degree of prematurity, infant autonomic maturity, neurobehavioral regulatory capacity) and aspects of maternal influence (including the quality of maternal caregiving and maternal depression and anxiety) were examined in relation to vocal rhythm coordination outcomes at infant age 4 months (CA). Multi-level time-series models were used to generate infant and mother vocal rhythm self-contingency (self-predictability, a form of self-regulation) and vocal rhythm interactive contingency (the degree to which each individual predictably adjusted to the vocal rhythms of the partner). For interactive contingency, results demonstrated that mothers and preterm infants coordinated the duration of pauses and switching pauses (at the turn exchange), indicating that the basic temporal and organizational mechanisms required for interpersonal vocal coordination were in place for this group. Bidirectional coordination was found for mothers’ and infants’ switching pause; unidirectional coordination was present for mothers’ pause. For self-contingency, results demonstrated that both preterm infants and their mothers showed significant self-contingency, indicating that both preterm infants and their mothers were firmly self-rooted, that is, predictable from their own prior behavior. As hypothesized, both infant influences and mother influences contributed to vocal rhythm coordination at 4 months (CA). Infant sex, birthweight, neonatal neurobehavioral regulatory capacity and concurrent vagal tone predicted mother-infant vocal coordination at 4 months (CA). Mothers’ age, ethnicity, and depression and anxiety symptoms at hospital discharge also contributed to vocal coordination at 4 months (CA). Viewed in conjunction with prior vocal rhythm research on term infants, these new findings may be able to aid in the assessment and early intervention of preterm infant dyads that may be at risk for less optimal cognitive and relational outcomes

Child-Directed Speech and the Developing Brain: An Investigation of Adult Verbal Warmth and Negative Affect

This dissertation examines the association between the quality of children’s language experiences – as operationalized by adult verbal warmth – and their cognitive developmental outcomes. A socioeconomically diverse sample of 43 parents and their 5-to-9-year-old children participated in this study. A digital audio recording of the home environment was obtained, and children completed a high-resolution, structural MRI scan as well as direct assessments of their language and reading skills. The audio recordings were transcribed and coded using a coding scheme newly developed by the candidate in consultation with leading experts, in order to identify and quantify psycholinguistic elements of adult-child communication. Primary hypotheses included that adult verbal warmth is associated with (1) language and reading outcomes (2) the neural regions associated with each. To date, no studies have combined a transcription-based, fine-grained analysis of naturalistic home recordings with neuroimaging data. As such, this study represents a new line of inquiry at the nexus of developmental psychology, neuroscience, and education. The findings shed light on the impact of psychosocial language experiences on child development and on which forms of adult-child communication are most conducive to learning. Such information can inform programs that aim to teach parents ways to nurture their children’s development through high-quality child-directed speech. Social, educational, and clinical implications for mitigating risk factors and bolstering protective factors in order to, ultimately, foster healthy development for all children, are discussed

Dissociation and interpersonal autonomic physiology in psychotherapy research: an integrative view encompassing psychodynamic and neuroscience theoretical frameworks

Interpersonal autonomic physiology is an interdisciplinary research field, assessing the relational interdependence of two (or more) interacting individual both at the behavioral and psychophysiological levels. Despite its quite long tradition, only eight studies since 1955 have focused on the interaction of psychotherapy dyads, and none of them have focused on the shared processual level, assessing dynamic phenomena such as dissociation. We longitudinally observed two brief psychodynamic psychotherapies, entirely audio and video-recorded (16 sessions, weekly frequency, 45 min.). Autonomic nervous system measures were continuously collected during each session. Personality, empathy, dissociative features and clinical progress measures were collected prior and post therapy, and after each clinical session. Two-independent judges, trained psychotherapist, codified the interactions\u2019 micro-processes. Time-series based analyses were performed to assess interpersonal synchronization and de-synchronization in patient\u2019s and therapist\u2019s physiological activity. Psychophysiological synchrony revealed a clear association with empathic attunement, while desynchronization phases (range of length 30-150 sec.) showed a linkage with dissociative processes, usually associated to the patient\u2019s narrative core relational trauma. Our findings are discussed under the perspective of psychodynamic models of Stern (\u201cpresent moment\u201d), Sander, Beebe and Lachmann (dyad system model of interaction), Lanius (Trauma model), and the neuroscientific frameworks proposed by Thayer (neurovisceral integration model), and Porges (polyvagal theory). The collected data allows to attempt an integration of these theoretical approaches under the light of Complex Dynamic Systems. The rich theoretical work and the encouraging clinical results might represents a new fascinating frontier of research in psychotherapy

Interactive tracking and action retrieval to support human behavior analysis

The goal of this thesis is to develop a set of tools for continuous tracking of behavioral phenomena in videos to support human behavior study. Current standard practices for extracting useful behavioral information from a video are typically difficult to replicate and require a lot of human time. For example, extensive training is typically required for a human coder to reliably code a particular behavior/interaction. Also, manual coding typically takes a lot more time than the actual length of the video (e.g. , it can take up to 6 times the actual length of the video to do human-assisted single object tracking. The time intensive nature of this process (due to the need to train expert and manual coding) puts a strong burden on the research process. In fact, it is not uncommon for an institution that heavily uses videos for behavioral research to have a massive backlog of unprocessed video data. To address this issue, I have developed an efficient behavior retrieval and interactive tracking system. These tools allow behavioral researchers/clinicians to more easily extract relevant behavioral information, and more objectively analyze behavioral data from videos. I have demonstrated that my behavior retrieval system achieves state-of-the-art performance for retrieving stereotypical behaviors of individuals with autism in a real-world video data captured in a classroom setting. I have also demonstrated that my interactive tracking system is able to produce high-precision tracking results with less human effort compared to the state-of-the-art. I further show that by leveraging the tracking results, we can extract an objective measure based on proximity between people that is useful for analyzing certain social interactions. I validated this new measure by showing that we can use it to predict qualitative expert ratings in the Strange Situation (a procedure for studying infant attachment security), a quantity that is difficult to obtain due to the difficulty in training the human expert.Ph.D

Paralinguistic event detection in children's speech

Paralinguistic events are useful indicators of the affective state of a speaker. These cues, in children's speech, are used to form social bonds with their caregivers. They have also been found to be useful in the very early detection of developmental disorders such as autism spectrum disorder (ASD) in children's speech. Prior work on children's speech has focused on the use of a limited number of subjects which don't have sufficient diversity in the type of vocalizations that are produced. Also, the features that are necessary to understand the production of paralinguistic events is not fully understood. To account for the lack of an off-the-shelf solution to detect instances of laughter and crying in children's speech, the focus of the thesis is to investigate and develop signal processing algorithms to extract acoustic features and use machine learning algorithms on various corpora. Results obtained using baseline spectral and prosodic features indicate the ability of the combination of spectral, prosodic, and dysphonation-related features that are needed to detect laughter and whining in toddlers' speech with different age groups and recording environments. The use of long-term features were found to be useful to capture the periodic properties of laughter in adults' and children's speech and detected instances of laughter to a high degree of accuracy. Finally, the thesis focuses on the use of multi-modal information using acoustic features and computer vision-based smile-related features to detect instances of laughter and to reduce the instances of false positives in adults' and children's speech. The fusion of the features resulted in an improvement of the accuracy and recall rates than when using either of the two modalities on their own.Ph.D

Vocal-Affective Weaning and its role in musical enculturation: children’s interest in music beyond the Infant-Directed Register

Well-established theoretical approaches to the Infant-directed Register (IDReg) draw a more or less tacit causal relation between it and all future musical engagement: the former establishes a basis for the latter. However, a mechanistic, observable chain of events linking these two phenomena has only been assumed, remaining for the most part unproblematized and undescribed. This thesis represents a first systematic attempt to fill this theoretical and empirical gap by concretely linking early musicality and the first manifestations of the most characteristic and widespread Western form of musical engagement: listening to recorded music. It does so by introducing a new construct as a mediating motivational factor— Vocal- Affective Weaning (VAW) —which mainly concerns the variation of caregivers’ use of the IDReg across developmental time. The thesis is grounded and tested in three literature reviews, two theoretical chapters, and three empirical ones. In terms of main findings, little evidence is found to support the existence of VAW as depicted in the main thesis. Consequently, any direct relationship between VAW and toddlers' attention to recorded music seems doubtful. Instead, data evidences a progressive use of Infant-Directed Speech (IDSp) as an ostensive cue used in the context of Natural Pedagogy, which allows for a better understanding of the relative importance of affectivity and cognition as parallel, coexisting governing principles that exert an influence on developmental changes concerning parental use of IDSp. If anything, data highlights interaction as a much more promising element in an explanatory chain linking the IDReg and Western forms of music engagement. The mentioned mismatch between the main thesis and results is interpreted in terms of the former’s unjustified stress on ‘centripetal’ over ‘centrifugal’ attachment dynamics. Resulting data also allows to delineate more nuanced factorial and developmental accounts of toddlers’ sustained attention to musical stimuli than has been previously advanced.Becas-Chile Scholarship, CONICYT Chile CONICYT- Chile Cambridge Scholarship, Cambridge Commonwealth, European & International Trus

Endogenous and social factors influencing infant vocalizations as fitness signals

Endogenous and social factors influencing infant vocalizations as fitness signal