Emotion and imitation in early infant-parent interaction: a longitudinal and cross-cultural study

Following a brief introduction to the diverse views on the motives for imitation, a review of the literature is presented covering the following topics: early theories and observations concerning the origin and development of human imitation in infancy; recent theoretical models that have emerged from experimental studies of infant imitation and from naturalistic studies of imitation in infant -mother communication; and traditional and recent theoretical and empirical approaches to imitative phenomena in infant -father interaction. This review leads to the following conclusions:a) The failure of attempts to confirm certain ideas, hypotheses and suggestions built into the theories and strategies of earlier studies does not detract from their great contribution, which set the foundations upon which recent research is carried forward.b) Despite the different theoretical frameworks and the lack of a consensus as to the best method for investigating early imitative phenomena in experimental settings, neonatal imitation is now accepted as a fact.c) Imitative phenomena found in empirical studies focusing on infant -father interaction, as well as the relevant theoretical interpretations, are characterised by a contradiction; theory predicts bidirectional regulations, but studies employ an empirical approach that favours the view that regulation is only on the parental side.In this investigation, observations were made of thirty infants, fifteen from Greece and fifteen from Scotland. All were seen every 15 days interacting with their mothers and with their fathers at home, from the 8th to the 24th week of life. A total of 540 home recordings were made. Units of interaction that contained imitative episodes were subjected to microanalysis with the aid of specialized software, in a multi -media system that provides the capability for detection, recording, timing and signal analysis of the variables under consideration to an accuracy of 1 /25th of a second.The main findings may be summarised as follows: a) Imitation was evident, as early as the 8th week, irrespective of the country, the parent or the infant's sex. b) Cultural differences, reflecting the predominance of non -vocal and vocal imitative expressive behaviour in the two countries, were found. c) The developmental course of early imitative expressive behaviours was typically non -linear. d) Turn-taking imitative exchanges predominated over co-actions. e) Parents were found to imitate their infants more than vice versa. f) Regulation of emotion, either in the sense of emotional matching or of emotional attunement, proved to be the underlying motivating principle for both parental and infant imitations.The implications of these findings for understanding universal intersubjective nature of early imitation in infant -father and infant-mother interactions are discussed

Computational and Psycho-Physiological Investigations of Musical Emotions

The ability of music to stir human emotions is a well known fact (Gabrielsson & Lindstrom. 2001). However, the manner in which music contributes to those experiences remains obscured. One of the main reasons is the large number of syndromes that characterise emotional experiences. Another is their subjective nature: musical emotions can be affected by memories, individual preferences and attitudes, among other factors (Scherer & Zentner, 2001). But can the same music induce similar affective experiences in all listeners, somehow independently of acculturation or personal bias? A considerable corpus of literature has consistently reported that listeners agree rather strongly about what type of emotion is expressed in a particular piece or even in particular moments or sections (Juslin & Sloboda, 2001). Those studies suggest that music features encode important characteristics of affective experiences, by suggesting the influence of various structural factors of music on emotional expression. Unfortunately, the nature of these relationships is complex, and it is common to find rather vague and contradictory descriptions. This thesis presents a novel methodology to analyse the dynamics of emotional responses to music. It consists of a computational investigation, based on spatiotemporal neural networks sensitive to structural aspects of music, which "mimic" human affective responses to music and permit to predict new ones. The dynamics of emotional responses to music are investigated as computational representations of perceptual processes (psychoacoustic features) and self-perception of physiological activation (peripheral feedback). Modelling and experimental results provide evidence suggesting that spatiotemporal patterns of sound resonate with affective features underlying judgements of subjective feelings. A significant part of the listener's affective response is predicted from the a set of six psychoacoustic features of sound - tempo, loudness, multiplicity (texture), power spectrum centroid (mean pitch), sharpness (timbre) and mean STFT flux (pitch variation) - and one physiological variable - heart rate. This work contributes to new evidence and insights to the study of musical emotions, with particular relevance to the music perception and emotion research communities

USING DEEP LEARNING-BASED FRAMEWORK FOR CHILD SPEECH EMOTION RECOGNITION

Biological languages of the body through which human emotion can be detected abound including heart rate, facial expressions, movement of the eyelids and dilation of the eyes, body postures, skin conductance, and even the speech we make. Speech emotion recognition research started some three decades ago, and the popular Interspeech Emotion Challenge has helped to propagate this research area. However, most speech recognition research is focused on adults and there is very little research on child speech. This dissertation is a description of the development and evaluation of a child speech emotion recognition framework. The higher-level components of the framework are designed to sort and separate speech based on the speaker’s age, ensuring that focus is only on speeches made by children. The framework uses Baddeley’s Theory of Working Memory to model a Working Memory Recurrent Network that can process and recognize emotions from speech. Baddeley’s Theory of Working Memory offers one of the best explanations on how the human brain holds and manipulates temporary information which is very crucial in the development of neural networks that learns effectively. Experiments were designed and performed to provide answers to the research questions, evaluate the proposed framework, and benchmark the performance of the framework with other methods. Satisfactory results were obtained from the experiments and in many cases, our framework was able to outperform other popular approaches. This study has implications for various applications of child speech emotion recognition such as child abuse detection and child learning robots

Emotion Recognition with Asymmetry Features of EEG Signals

Currently the study of affective computing (AC) includes a focus on researching emotion regulation and recognition. Recent studies in this field have utilized deep learning architectures to enhance emotion recognition from EEG signals. An alternative approach to deep learning is to use feature engineering to extract relevant features to train supervised machine learning models. Current theories in the neuroscience field can guide this feature engineering process. Neuroscientists have suggested various models to clarify how emotions are processed. One of these models suggests that positive emotions are processed in the left hemisphere, while negative emotions are processed in the right hemisphere. This emotional processing model has inspired previous studies to propose asymmetrical features to predict emotions. However, none of these studies have statistically evaluated whether the inclusion of asymmetrical features could yield benefits such as increased accuracy or reduced training time. To address that direction, this research presents both statistical evaluations for emotion regulation and a comparable model for emotion recognition. The outcomes show that brain hemispheres and frequency bands participate differently in processing emotions and observed the presence of the two asymmetry emotion processing models but in different frequency ranges. Also, the results from this study imply that by using asymmetry EEG, emotion recognition approaches can use fewer features without significantly compromising performance.Master of Science in Applied Computer Scienc

ENHANCING THE MOTIVATION AND LEARNING PERFORMANCE IN AN ONLINE CLASSROOM WITH THE USE OF NEUROMARKETING

In recent years, the newly emerging discipline of neuromarketing, which employs brain (emotions and behaviour) research in an organisational context, has grown in prominence in academic and practice literature. With the increasing growth of online teaching, COVID-19 left no option for higher education institutions to go online. As a result, students who attend an online course are more prone to lose focus and attention, resulting in poor academic performance. Therefore, the primary purpose of this study is to observe the learner's behaviour while making use of an online learning platform. This study presents neuromarketing to enhance students' learning performance and motivation in an online classroom. Using a web camera, we used facial coding and eye-tracking techniques to study students' attention, motivation, and interest in an online classroom. In collaboration with Oxford Business College's marketing team, the Institute for Neuromarketing distributed video links via email, a student representative from Oxford Business College, the WhatsApp group, and a newsletter developed explicitly for that purpose to 297 students over the course of five days. To ensure the research was both realistic and feasible, the instructors in the videos were different, and students were randomly allocated to one video link lasting 90 seconds (n=142) and a second one lasting 10 minutes (n=155). An online platform for self-service called Tobii Sticky was used to measure facial coding and eye-tracking. During the 90-second online lecture, participants' gaze behaviour was tracked overtime to gather data on their attention distribution, and emotions were evaluated using facial coding. In contrast, the 10-minute film looked at emotional involvement. The findings show that students lose their listening focus when no supporting visual material or virtual board is used, even during a brief presentation. Furthermore, when they are exposed to a single shareable piece of content for longer than 5.24 minutes, their motivation and mood decline; however, when new shareable material or a class activity is introduced, their motivation and mood rise. JEL: I20; I21 Article visualizations

Grounding language in events

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2008.Includes bibliographical references (p. 137-142).Broadcast video and virtual environments are just two of the growing number of domains in which language is embedded in multiple modalities of rich non-linguistic information. Applications for such multimodal domains are often based on traditional natural language processing techniques that ignore the connection between words and the non-linguistic context in which they are used. This thesis describes a methodology for representing these connections in models which ground the meaning of words in representations of events. Incorporating these grounded language models with text-based techniques significantly improves the performance of three multimodal applications: natural language understanding in videogames, sports video search and automatic speech recognition. Two approaches to representing the structure of events are presented and used to model the meaning of words. In the domain of virtual game worlds, a hand-designed hierarchical behavior grammar is used to explicitly represent all the various actions that an agent can take in a virtual world. This grammar is used to interpret events by parsing sequences of observed actions in order to generate hierarchical event structures. In the noisier and more open -ended domain of broadcast sports video, hierarchical temporal patterns are automatically mined from large corpora of unlabeled video data. The structure of events in video is represented by vectors of these hierarchical patterns.(cont.) Grounded language models are encoded using Hierarchical Bayesian models to represent the probability of words given elements of these event structures. These grounded language models are used to incorporate non-linguistic information into text-based approaches to multimodal applications. In the virtual game domain, this non-linguistic information improves natural language understanding for a virtual agent by nearly 10% and cuts in half the negative effects of noise caused by automatic speech recognition. For broadcast video of baseball and American football, video search systems that incorporate grounded language models are shown to perform up to 33% better than text-based systems. Further, systems for recognizing speech in baseball video that use grounded language models show 25% greater word accuracy than traditional systems.by Michael Ben Fleischman.Ph.D