Emerging Linguistic Functions in Early Infancy

This paper presents results from experimental studies on early language acquisition in infants and attempts to interpret the experimental results within the framework of the Ecological Theory of Language Acquisition (ETLA) recently proposed by (Lacerda et al., 2004a). From this perspective, the infant’s first steps in the acquisition of the ambient language are seen as a consequence of the infant’s general capacity to represent sensory input and the infant’s interaction with other actors in its immediate ecological environment. On the basis of available experimental evidence, it will be argued that ETLA offers a productive alternative to traditional descriptive views of the language acquisition process by presenting an operative model of how early linguistic function may emerge through interaction

Cognitive, neural, and social mechanisms of rhythmic interpersonal coordination

Humans possess the exceptional capacity to temporally coordinate their movements with one another with a high degree of accuracy, precision, and flexibility. Musical ensemble performance is a refined example of this, where a range of cognitive and sensory-motor processes work together to support rhythmic interpersonal coordination. However, the influence of social factors on the underlying cognitive-motor and neural mechanisms that facilitate rhythmic interpersonal coordination is yet to be established. This thesis draws on theoretical perspectives related to joint action, including co-representation, self-other integration and segregation, and theoretical models of sensorimotor synchronisation to consider this topic. Three experiments were conducted to investigate how social factors influence rhythmic interpersonal coordination. This broad empirical question was broken down by considering both extrinsic factors—such as the social context and perceived characteristics of an interaction partner (e.g. the degree of partner intentionality and responsiveness)—as well as intrinsic social factors, such as individual differences in attitudes and social preferences. This thesis concludes that extrinsic and intrinsic social factors affect rhythmic interpersonal coordination at multiple levels. A key aspect of this influence relates to how people regulate the integration and segregation of their representations of self and others. However, importantly, these effects are mediated by individual differences in intrinsic social factors such as personal preferences and biases. Top-down processes related to beliefs thus influence bottom-up sensorimotor processes during joint action, but the nature of this influence appears to be different for different people. This outcome highlights the necessity of taking individual differences into account, particularly when investigating the nuances of social processing during dynamic social interactions. Furthermore, the current findings suggest that beliefs about a partner during social interaction may be just as, or even more so, influential on performance than the actual characteristics of the partner. Recognising the potency of social beliefs has implications not only for research into basic psychological mechanisms underpinning rhythmic interpersonal coordination, but also for understanding the broader social dynamics of real-life situations involving cooperative joint action understanding the broader social dynamics of real-life situations involving cooperative joint action

Visually Indicated Sounds

Objects make distinctive sounds when they are hit or scratched. These sounds reveal aspects of an object's material properties, as well as the actions that produced them. In this paper, we propose the task of predicting what sound an object makes when struck as a way of studying physical interactions within a visual scene. We present an algorithm that synthesizes sound from silent videos of people hitting and scratching objects with a drumstick. This algorithm uses a recurrent neural network to predict sound features from videos and then produces a waveform from these features with an example-based synthesis procedure. We show that the sounds predicted by our model are realistic enough to fool participants in a "real or fake" psychophysical experiment, and that they convey significant information about material properties and physical interactions

A Trip to the Moon: Personalized Animated Movies for Self-reflection

Self-tracking physiological and psychological data poses the challenge of presentation and interpretation. Insightful narratives for self-tracking data can motivate the user towards constructive self-reflection. One powerful form of narrative that engages audience across various culture and age groups is animated movies. We collected a week of self-reported mood and behavior data from each user and created in Unity a personalized animation based on their data. We evaluated the impact of their video in a randomized control trial with a non-personalized animated video as control. We found that personalized videos tend to be more emotionally engaging, encouraging greater and lengthier writing that indicated self-reflection about moods and behaviors, compared to non-personalized control videos