Intersubjectivity and cooperation in synchronous computer-mediated interaction

Interpersonal communication depends on a variety of signals, which humans have evolved to observe and understand in face-to-face interaction. Direct social perception and primary intersubjectivity refer to the ability to process non-verbal signals about the mental states of other people quickly and automatically, as a way to sort of co-experience them. Mediated communication is often necessary and even preferred due to different reasons, such as distance, time, or the use of technological tools, but currently there is no match for physical presence when it comes to engaging the mechanisms of primary intersubjectivity for understanding emotion and intent. This thesis is about the development of digital tools for improving collaboration in synchronous computer-mediated communication. While exploring multimodal technologies for communication, a lack of quantifiable measures for assessing the effects of prototypes on cooperation was identified. To this end, collaborative performance tasks were developed for a series of studies and used in order to investigate cooperation in computer-mediated interaction. In Study I, a joint coordination task in the form of a collaborative car racing game was used to study inter-brain synchronization without physical presence. The study revealed that EEG synchonization happens in the online gaming context. EEG synchronization was connected to task performance both momentarily, with higher gamma synchrony occurring during better performance, and pairwise, with higher overall alpha synchrony among high-performing pairs. Study II details the development and testing of a collaborative block design task, which was created to assess pair performance in social virtual reality (VR). The task can also be replicated in face-to-face interaction, making it possible to compare effects in these two domains. Individual visuospatial intelligence was identified as a factor that needs to be controlled when using the collaborative block design task. In Study III, a mouse controlled visual guidance task was developed to study the application of heart rate sharing in online chat box customer service. Combining quantitative and qualitative data, the study revealed that biosignal sharing might not be an optimal strategy for supporting cooperation and presence in customer service text chat. Taken together, the results suggest that both cooperative task performance and inter-brain synchronization can be meaningful measures in the development of synchronous computer-mediated communication. A final task, SynchroMouse is presented as a possible activity for increasing inter-brain and interpersonal synchronization online.Tämä väitöskirja koskee läsnäolon ja yhteistyön tukemista samanaikaisessa verkon yli tapahtuvassa vuorovaikutuksessa, ja menetelmiä yhteistyön mittaamiseksi. Yhteistyötehtäviä kehitettiin tutkimussarjassa, joka tutkii yhteistoimintaa tietokonevälitteisessä kommunikaatiossa. Verkossa tapahtuva ihmistenvälinen vuorovaikutus on hyvin suosittua, ja sen käyttö on lisääntynyt viime vuosina. Sitä käytetään yhteistyöhön ja sosiaalisten suhteiden ylläpitoon. Meidän sosiaaliset kykymme ovat kuitenkin kehittyneet kasvokkain tapahtuvassa vuorovaikutuksessa, ja verkossa pienet eleet, ilmeet ja ihmisten välinen tahdistuminen, joiden avulla ymmärrämme toistemme tunteita ja aikeita, jäävät usein vähemmälle huomiolle. Ensimmäinen osatutkimus osoitti, että aivojen välistä tahdistumista tapahtuu kun ihmiset pelaavat verkkopeliä ilman fyysistä läsnäoloa, ja että tahdistuminen on yhteydessä yhteistyössä suoriutumiseen. Toisessa osatutkimuksessa kehitettiin menetelmää, jolla voidaan arvioida yhteistyötä virtuaalitodellisuudessa, ja vertailla sitä kasvokkain tapahtuvaan yhteistyöhön. Kolmas osatutkimus tutki sykeinformaation käyttämistä kommunikaatiossa, ja osoitti sen olevan haitallista yhteistyötehtävässä suoriutumisen suhteen. Kokonaisuudessaan väitöstutkimus osoittaa, että verkkovuorovaikutusta voidaan kehittää yhteistyötehtävillä. Aivojen välisen tahdistumisen lisääminen voi myös olla keino parantaa läsnäolon tunnetta ja empatiaa verkossa. Lopuksi esitellään moninpelitehtävä, SynchroMouse, jonka on tarkoitus lisätä tahdistumista etänä

Emotion transfer protocol

A problem exists in computer-mediated communication (CMC). A distinct lack of presence and emotional nuance causes the quality of CMC to be shallower than face-to-face communication, causing misunderstandings and a lack of empathy. This thesis proposes a solution by widening the emotional bandwidth, with the help of augmenting the digital communication channel by utilizing new technologies and principles derived from scientific theory and practice in design. The goal of this thesis is to draft a proposal for a new internet protocol: the Emotion Transfer Protocol. Several questions need to be answered: How can emotions be described in an accurate and meaningful way? How can emotions be measured, transmitted, and represented? This thesis approaches these questions from an inclusive point of view, by considering different and even op-posing answers, leaving space for future work to expand and reduce the scope of the protocol. The protocol itself is divided into three components: input, transmission, and output. Each of the components is presented as a collection of approaches that are currently used in daily life, and in research to represent, map, and read emotions. An interesting finding that is present on all levels of emotion science and technology is a divide between unconscious and conscious representations, and this is also considered in the protocol by dividing it into an explicit and an implicit version A novel idea of unlabeled emotions is presented, meaning emotional representations that are left to be interpreted by the receiver. Unlabeled emotions and emotion transmission are explored in three different practical art, design, and research projects

Collaborative block design task for assessing pair performance in virtual reality and reality

Collaborative problem solving is more important than ever as the problems we try to solve become increasingly complex. Meanwhile, personal and professional communication has moved from face-to-face to computer-mediated environments, but there is little understanding on how the characteristics of these environments affect the quality of interaction and joint problem solving. To develop this understanding, methods are needed for measuring success of collaboration. For this purpose, we created a collaborative block design task intended to evaluate and quantify pair performance. In this task, participants need to share information to complete visuospatial puzzles. Two versions of the task are described: a physical version and one that can be completed in virtual reality. A preliminary study was conducted with the physical version (N = 18 pairs) and the results were used to develop the task for a second study in virtual reality (N = 31 pairs). Performance measures were developed for the task, and we found that pair performance was normally distributed and positively associated with visuospatial skills, but not with other participant-specific background factors. The task specifications are released for the research community to apply and adapt in the study of computer-mediated social interaction.Peer reviewe

Perceptions of Doctors’ Empathy and Patients’ Subjective Health Status at an Online Clinic : Development of an Empathic Anamnesis Questionnaire

Objective Medical doctors' empathy is known to support patients' health status after face-to-face clinical visits. However, the role of doctors' empathy in chat-based encounters is not yet well understood. This study investigates whether patients' evaluations of doctors' empathy are associated with their health perceptions after a meeting at an online clinic and whether experiences of empathy could be enhanced by augmenting an automated anamnesis questionnaire completed before the visit. Methods A total of 209 adult patients agreed to participate in the study. First 103 patients filled out the regular version of the questionnaire (June-August 2019) and then 106 filled out the augmented version of the online anamnesis questionnaire (August-November 2019). Patients' perceptions of doctors' empathy were measured with the Consultation and Relational Empathy questionnaire. Patients' self-perceived health status, potential confounders, and demographic background information were measured via questionnaires. Results Patients' perceptions of doctors' empathy during a chat-based encounter were associated with patients being less concerned about their symptoms (estimated odds ratios varied between 0.45 and 0.55 depending on the model, p values < .003) and considering their symptoms as less severe (estimated odds ratios = 0.54-0.61, p values < .007), as well as a higher probability of alleviation of symptoms as rated by the patients (estimated odds ratios = 2.16-2.24, p values < .001). Augmenting the anamnesis questionnaire did not affect patient reports on doctors' overall empathy, but it did have positive effects on specific areas of doctors' empathy covered by the questionnaire. Conclusions These results show that patients' experience of doctors' empathy not only is important during face-to-face encounters but also supports patients' perceptions of health when the interaction is text based. The results also encourage further development of means to support patients' experiences of empathy during online interaction with medical doctors.Peer reviewe

Neural processing of changes in phonetic and emotional speech sounds and tones in preterm infants at term age

Objective: Auditory change-detection responses provide information on sound discrimination and memory skills in infants. We examined both the automatic change-detection process and the processing of emotional information content in speech in preterm infants in comparison to full-term infants at term age. Methods: Preterm (n = 21) and full-term infants' (n = 20) event-related potentials (ERP) were recorded at term age. A challenging multi-feature mismatch negativity (MMN) paradigm with phonetic deviants and rare emotional speech sounds (happy, sad, angry), and a simple one-deviant oddball paradigm with pure tones were used. Results: Positive mismatch responses (MMR) were found to the emotional sounds and some of the phonetic deviants in preterm and full-term infants in the multi-feature MMN paradigm. Additionally, late positive MMRs to the phonetic deviants were elicited in the preterm group. However, no group differences to speech-sound changes were discovered. In the oddball paradigm, preterm infants had positive MMRs to the deviant change in all latency windows. Responses to non-speech sounds were larger in preterm infants in the second latency window, as well as in the first latency window at the left hemisphere electrodes (F3, C3). Conclusions: No significant group-level differences were discovered in the neural processing of speech sounds between preterm and full-term infants at term age. Change-detection of non-speech sounds, however, may be enhanced in preterm infants at term age. Significance: Auditory processing of speech sounds in healthy preterm infants showed similarities to full-term infants at term age. Large individual variations within the groups may reflect some underlying differences that call for further studies.Peer reviewe

Are you there? : Presence in collaborative distance work

doi linkki ei toimi 31.1.2022, ilmoitettu lehteen/USAlready before the pandemic, digitally mediated collaborative work and communication were perceived as challenging. We investigate the attitudes towards emerging technologies and for transforming practises in workplaces. The focus lies on understanding the readiness for appropriating emotional tracking on presence and support for collaboration. The research-based design framework allowed to combine the various perspectives of the transdisciplinary team. Methods included participatory design, design thinking, contextual inquiry and prototype testing for enhancing presence while working with shared objects in video conferencing to explore the appropriation of tools. The findings revealed four indications: 1) awareness of interlocutors’ presence during synchronous communication is crucial. 2) Emotion and behaviour tracking raises concerns about privacy and personal control over what is displayed to others, and technology could be simpler non-distracting the work at hand. 3) The prototype was found to enhance the feeling of presence without disturbing work at hand, and 4) appropriation requires a step-by-step approach.Peer reviewe

Transcriptome analysis in primary neural stem cells using a tag cDNA amplification method

BACKGROUND: Neural stem cells (NSCs) can be isolated from the adult mammalian brain and expanded in culture, in the form of cellular aggregates called neurospheres. Neurospheres provide an in vitro model for studying NSC behaviour and give information on the factors and mechanisms that govern their proliferation and differentiation. They are also a promising source for cell replacement therapies of the central nervous system. Neurospheres are complex structures consisting of several cell types of varying degrees of differentiation. One way of characterising neurospheres is to analyse their gene expression profiles. The value of such studies is however uncertain since they are heterogeneous structures and different populations of neurospheres may vary significantly in their gene expression. RESULTS: To address this issue, we have used cDNA microarrays and a recently reported tag cDNA amplification method to analyse the gene expression profiles of neurospheres originating from separate isolations of the lateral ventricle wall of adult mice and passaged to varying degrees. Separate isolations as well as consecutive passages yield a high variability in gene expression while parallel cultures yield the lowest variability. CONCLUSIONS: We demonstrate a low technical amplification variability using the employed amplification strategy and conclude that neurospheres from the same isolation and passage are sufficiently similar to be used for comparative gene expression analysis

Inter-brain synchronization occurs without physical co-presence during cooperative online gaming

Inter-brain synchronization during social interaction has been linked with several positive phenomena, including closeness, cooperation, prosociality, and team performance. However, the temporal dynamics of inter-brain synchronization during collaboration are not yet fully understood. Furthermore, with collaboration increasingly happening online, the dependence of inter-brain phase synchronization of oscillatory activity on physical presence is an important but understudied question. In this study, physically isolated participants performed a collaborative coordination task in the form of a cooperative multiplayer game. We measured EEG from 42 subjects working together as pairs in the task. During the measurement, the only interaction between the participants happened through on-screen movement of a racing car, controlled by button presses of both participants working with distinct roles, either controlling the speed or the direction of the car. Pairs working together in the task were found to have elevated neural coupling in the alpha, beta, and gamma frequency bands, compared to performance matched false pairs. Higher gamma synchrony was associated with better momentary performance within dyads and higher alpha synchrony was associated with better mean performance across dyads. These results are in line with previous findings of increased inter-brain synchrony during interaction, and show that phase synchronization of oscillatory activity occurs during online real-time joint coordination without any physical co-presence or video and audio connection. Synchrony decreased during a playing session, but was found to be higher during the second session compared to the first. The novel paradigm, developed for the measurement of real-time collaborative performance, demonstrates that changes in inter-brain EEG phase synchrony can be observed continuously during interaction.Peer reviewe

Maternal sleep quality during pregnancy is associated with neonatal auditory ERPs

Poor maternal sleep quality during pregnancy may act as a prenatal stress factor for the fetus and associate with neonate neurocognition, for example via fetal programming. The impacts of worsened maternal sleep on neonatal development and, more specifically on neonatal auditory brain responses, have not been studied. A total of 155 mother-neonate dyads drawn from the FinnBrain Birth Cohort Study participated in our study including maternal self-report questionnaires on sleep at gestational week 24 and an event-related potential (ERP) measurement among 1-2-day-old neonates. For sleep quality assessment, the Basic Nordic Sleep Questionnaire (BNSQ) was used and calculated scores for (1) insomnia, (2) subjective sleep loss and (3) sleepiness were formed and applied in the analyses. In the auditory ERP protocol, three emotionally uttered pseudo words (in happy, angry and sad valence) were presented among neutrally uttered pseudo words. To study the relations between prenatal maternal sleep quality and auditory emotion-related ERP responses, mixed-effects regression models were computed for early (100-200 ms) and late (300-500 ms) ERP response time-windows. All of the selected BNSQ scores were associated with neonatal ERP responses for happy and angry emotion stimuli (sleep loss and sleepiness in the early, and insomnia, sleep loss and sleepiness in the late time-window). For sad stimuli, only maternal sleep loss predicted the neonatal ERP response in the late time-window, likely because the overall ERP was weakest in the sad condition. We conclude that maternal sleep quality during pregnancy is associated with changes in neonatal auditory ERP responses.Peer reviewe