Being in-sync: A multimodal framework on the emotional and cognitive synchronization of collaborative learners

Collaborative learners share an experience when focusing on a task together and coevally influence each other’s emotions and motivations. Continuous emotional synchronization relates to how learners co-regulate their cognitive resources, especially regarding their joint attention and transactive discourse. “Being in-sync” then refers to multiple emotional and cognitive group states and processes, raising the question: to what extent and when is being in-sync beneficial and when is it not? In this article, we propose a framework of multi-modal learning analytics addressing synchronization of collaborative learners across emotional and cognitive dimensions and different modalities. To exemplify this framework and approach the question of how emotions and cognitions intertwine in collaborative learning, we present contrasting cases of learners in a tabletop environment that have or have not been instructed to coordinate their gaze. Qualitative analysis of multimodal data incorporating eye-tracking and electrodermal sensors shows that gaze instruction facilitated being emotionally, cognitively, and behaviorally “in-sync” during the peer collaboration. Identifying and analyzing moments of shared emotional shifts shows how learners are establishing shared understanding regarding both the learning task as well as the relationship among them when they are emotionally “in-sync.

Implementing process methods in learning research:targeting emotional responses in collaborative learning

Abstract. Context and aim: While interacting with peers and teachers in a collaborative learning task, students experience socio-emotional challenges and display emotional responses. These responses have two major components: arousal and valence, which influence the learning process and its outcomes. The aim of the study was twofold: first, to explore how group members’ arousal levels vary across different phases of a collaborative learning task; and second, to investigate how case students’ emotional responses are distributed in the arousal-valence space across the phases of the collaborative task. Methods: Twelve 6th graders from a school of Finland participated in a collaborative task, in groups of three students. The task was to build an energy efficient house in three distinct phases: brainstorming, planning, and building. While performing the activity, students wore Empatica E4 wristbands to measure their electrodermal activity (EDA) and were video-recorded with 360° cameras. Arousal levels were calculated in peaks per min (ppm) and classified as low, middle, and high. Emotional valence was classified from video analysis into positive, neutral, and negative. Results: The ranges for arousal levels were established between 26 and 88 ppm. Only two students displayed the same arousal level across the three phases of the experiment. Three students displayed higher arousal at first and then fell in to lower levels. Four students had the opposite experience and three students did not display a pattern. As for the case students, the student leading a poorly collaborating group experienced oscillating levels of arousal, from middle to high, and displayed a mix of negative and positive valence most of the time. The student loafing around experienced all arousal levels and positive valence most of the time. Overall conclusions and relevance: The study allowed to establish measurement thresholds for arousal as a starting point for future studies in collaborative learning and the arousal-valence space provided a quantifiable picture to help teachers understand the importance of emotional responses in classroom during collaborative learning

Biosignals reflect pair-dynamics in collaborative work : EDA and ECG study of pair-programming in a classroom environment

Collaboration is a complex phenomenon, where intersubjective dynamics can greatly affect the productive outcome. Evaluation of collaboration is thus of great interest, and can potentially help achieve better outcomes and performance. However, quantitative measurement of collaboration is difficult, because much of the interaction occurs in the intersubjective space between collaborators. Manual observation and/or self-reports are subjective, laborious, and have a poor temporal resolution. The problem is compounded in natural settings where task-activity and response-compliance cannot be controlled. Physiological signals provide an objective mean to quantify intersubjective rapport (as synchrony), but require novel methods to support broad deployment outside the lab. We studied 28 student dyads during a self-directed classroom pair-programming exercise. Sympathetic and parasympathetic nervous system activation was measured during task performance using electrodermal activity and electrocardiography. Results suggest that (a) we can isolate cognitive processes (mental workload) from confounding environmental effects, and (b) electrodermal signals show role-specific but correlated affective response profiles. We demonstrate the potential for social physiological compliance to quantify pair-work in natural settings, with no experimental manipulation of participants required. Our objective approach has a high temporal resolution, is scalable, non-intrusive, and robust.Peer reviewe

Psychophysiology

Abstract Joint performance can lead to the synchronization of physiological processes among group members during a shared task. Recently, it has been shown that synchronization is indicative of subjective ratings of group processes and task performance. However, different methods have been used to quantify synchronization, and little is known about the effects of the choice of method and level of analysis (individuals, dyads, or triads) on the results. In this study, participants performed a decision-making task in groups of three while physiological signals (heart rate and electrodermal activity), positive affective behavior, and personality traits were measured. First, we investigated the effects of different levels of analysis of physiological synchrony on affective behavior. We computed synchrony measures as (a) individual contributions to group synchrony, (b) the average dyadic synchrony within a group, and (c) group-level synchrony. Second, we assessed the association between physiological synchrony and positive affective behavior. Third, we investigated the moderating effects of trait anxiety and social phobia on behavior. We discovered that the effects of physiological synchrony on positive affective behavior were particularly strong at the group level but nonsignificant at the individual and dyadic levels. Moreover, we found that heart rate and electrodermal synchronization showed opposite effects on group members' display of affective behavior. Finally, trait anxiety moderated the relationship between physiological synchrony and affective behavior, perhaps due to social uncertainty, while social phobia did not have a moderating effect. We discuss these results regarding the role of different physiological signals and task demands during joint action

Empathy and Inter-brain Synchrony During Online Collaboration

Tavoitteet. Tutkimustietoa heikentyneistä empatiataidoista tietokonevälitteisen vuorovaikutuksen aikana on runsaasti. Myös empatiakykyjen neuraalisesta perustasta tiedetään jo melko paljon. Tietoa siitä, miten puutteet empatiataidoissa näkyvät aivotoiminnan tasolla ihmisten välisen tietokonevälitteisen vuorovaikutuksen aikana, on kuitenkin huomattavasti vähemmän. Aivojen sähköisen toiminnan on löydetty synkronoituvan kasvokkain yhteistyötä tekevien henkilöiden välillä, ja tämä synkronia näyttäisi olevan yhteydessä vuorovaikutuksen laatuun. Tämän tutkimuksen tarkoituksena onkin selvittää 1) löytyykö samanlaista sähköisen toiminnan synkronoitumista myös tietokonevälitteisen yhteistyön aikana, kun henkilöt eivät ole samassa fyysisessä tilassa 2) onko tietokonevälitteisen yhteistyön aikainen aivojen sähköisen toiminnan synkronoituminen yhteydessä vuorovaikutuksessa olevien henkilöiden empatiakykyihin. Menetelmät. Tutkimuksessa oli mukana 21 paria, joista jokainen muodostui kahdesta koehenkilöstä, jotka tunsivat toisensa, ja päättivät osallistua tutkimukseen yhdessä. Parit suorittivat ensin yksilöllisiä empatiatestejä, jonka jälkeen he saivat tehtäväkseen pelata tietokoneen välityksellä yhdessä autopeliä, jossa toisen tuli säädellä auton ajonopeutta ja toisen auton ajosuuntaa. Tehtävän aikana koehenkilöiden aivojen sähköistä toimintaa mitattiin EEG:n avulla. Henkilöiden välistä aivojen toiminnan synkroniaa tutkittiin tarkastelemalla theta-, alfa-, beta- ja gamma-aktivaation voimakkuuden eroja koehenkilöiden välillä frontaali-, frontosentraali-, sentraali-, parietaali-, temporoparietaali- ja oksipitaalialueilta mitattuna. Tulokset ja johtopäätökset. Koehenkilöiden väliseen yhteistyöhön liittyvää synkroniaa löytyi theta-taajuuden osalta frontaali-, frontosentraali-, sentraali-, parietaali- ja temporoparietaalialueilta; alfa-taajuuden osalta frontosentraalialueilta; beta-taajuuden osalta frontosentraali-, sentraali-, parietaali-, ja oksipitaalialueilta sekä gamma-taajuuden osalta frontosentraali- ja sentraalialueilta mitattuna. Synkronoitunut aktviaatio näillä taajuuskaistoilla, näiltä alueilta mitattuna näyttäisi olevan tietokonevälitteisen yhteistyön kannalta merkityksellistä. Tilastollisesti merkitseviä yhteyksiä löydetyn synkronian ja empatiataitojen välillä ei löytynyt.Objective. In cognitive neuroscience empathy is defined as a set of skills and tendencies that enables us to interpret and predict the mental states and actions of others and share emotional states and the experience of others. These skills and tendencies are important for successful interaction and in most situations rely heavily on natural social cues. In addition to verbal cues, these natural cues consist of for example facial expressions, bodily gestures, and prosody of speech. Also, a shared environment that enables for example eye contact and joint attention have previously been found beneficial for empathy. However, a growing percentage of our social interaction takes place in online environments where many of these features found important during face-to-face interaction are absent. A great body of evidence exists on the decrease in empathy skills during online compared to face-to-face interaction. A fair amount of research also exists on the neural foundation underlying empathy. Research on how this decrease in empathy processes during online interaction can be observed on the neural level is however limited. One phenomenon found to occur during face-to-face interaction is the synchronization of the brain's electric activity between collaborating individuals. Associations between this neural synchrony and the quality of interaction have also been found. The purpose of this study is to investigate 1) whether inter-brain synchrony occurs during online collaboration in the absence of natural social cues and 2) whether this synchrony is associated with the empathy skills of the collaborating individuals. Methods. The subjects of the study consisted of 21 pairs, each in which the two subjects knew each other in advance and decided to participate in the study together. The subjects first completed individual empathy tests, after which their task was to play a collaborative online car game together in separate physical locations during which one of the subjects was to control the speed while the other was to control the direction of the car. During this task, the neural activity of each subject was measured with EEG. The inter-brain synchrony between the collaborating individuals was studied by investigating the associations of power in the theta, alpha, beta, and gamma frequency bands measured over the frontal, frontocentral, central, parietal, temporoparietal, and occipital regions between the two individuals. Results and Conclusions. Inter-brain synchrony specific to collaboration was found in the theta frequency band over the frontal, frontocentral, central, parietal, and temporoparietal regions; in the alpha frequency band over the frontocentral region; in the beta frequency band over the frontocentral, central, parietal, and occipital regions; and in the gamma frequency band over the frontocentral and central regions. This suggests that the synchrony in these frequency bands measured over these regions is related to computer-mediated collaboration. No significant associations were found between the inter-brain synchrony and empathy skills

Amygdala Neurofeedback Training in Borderline Personality Disorder: Capturing Improvements in Emotion Regulation

The way we regulate emotions is a powerful determinant of behavior and directly impacts affect and physiology. Many mental disorders, such as borderline personality disorder, are in large part disorders of emotion dysregulation. Because of its important role in mental health, research has endeavored to understand the mechanisms and biological underpinnings of emotion regulation and to create trainings and specific clinical programs that aim to augment the ability to regulate emotions. The assessment of psychophysiological responses represents an important complementary method to quantify emotion regulation in both studies on healthy individuals and studies assessing clinical emotion regulation trainings. However, psychophysiological effects have been inconsistent across literature, which impedes informed decisions about suitable psychophysiological variables of emotion regulation experiments and clinical trainings. A new technique assumed to improve emotion regulation is amygdala neurofeedback training. Because patients with borderline personality disorder show hyperreactivity of the amygdala likely underlying the severe emotion regulation problems they suffer from, amygdala neurofeedback training may be a candidate training to improve emotion regulation in these patients. Until now, it has been unclear which aspects of psychopathology and emotion regulation may change with neurofeedback-aided amygdala downregulation in borderline personality disorder, which is urgently needed to determine a primary outcome measure for future randomized controlled trails. To fill these gaps, the present doctoral thesis identified the effects of psychophysiological responses of emotion regulation as well as important moderators and identified primary outcome measures of emotion dysregulation after neurofeedback training in patients with borderline personality disorder. In total, three studies were conducted. In Study I, a total of 1353 studies on psychophysiological responses of emotion regulation were screened through a systematic search of articles and meta-analyses were used to evaluate effect sizes of instructed downregulation strategies on common autonomic and electromyographic measures. Following this, Study II systematically tested effects of the startle probe timing on startle responses during emotion regulation in 47 healthy individuals. Study II aimed at optimizing emotion regulation assessment with the emotion-modulated startle that was then used in Study III. In Study III, a four-session amygdala neurofeedback training was tested in 24 female patients with borderline personality disorder. Before and after the neurofeedback training, as well as at a 6-week follow-up assessment, measures of emotion dysregulation and borderline personality disorder psychopathology were tested at diverse levels of analysis. Results from Study I demonstrate that effects of emotion regulation on autonomic measures, even if significant, were small and heterogeneous across studies, while electromyographic measures were more homogeneous and revealed medium effect sizes. Important study characteristics such as the study design, control instruction and trial duration moderated some autonomic effects of suppression and reappraisal. Study II demonstrated a significant inhibition of the startle response with emotion downregulation. Startle probes delivered at >7 seconds into the regulation phase were useful to quantify reappraisal effects, although earlier probes did not yield significantly smaller effects. Finally, Study III demonstrated that the inhibition of the startle with emotion downregulation increased after the training, suggesting improved emotion regulation abilities. In addition, we could demonstrate that general BPD psychopathology as well as affective instability and negative affect in daily life improved after training. However, after correction for multiple comparisons, observed effect sizes did not surpass the significance level and some effects (e.g., startle) faded to the 6-week follow-up assessment. In sum, the present thesis provides the groundwork for future randomized controlled trials of amygdala neurofeedback training and enables future laboratory and clinical studies to gain more stable effects in psychophysiological measurements of emotion regulation. In particular, the findings implicate that with regard to emotion regulation research, autonomic measures appear to be highly variable and thus should be selected carefully. In addition, we need more comparable psychophysiological set-ups in the empirical study of emotion regulation. The emotion-modulated startle not only proved to be a robust measure to quantify emotion regulation effects in general, but also appeared to be suitable to track improvements in emotion regulation in the context of a neurofeedback training targeting emotion dysregulation. With respect to emotion regulation outcome measures for future amygdala neurofeedback studies, further improvement of the specific paradigms is needed. In addition, the neurofeedback training itself should be optimized in terms of e.g. training time and booster sessions. Future placebo-controlled trials must then confirm that the treatment is effective in improving emotion regulation in those with borderline personality disorder