Tensions of Data-Driven Reflection: A Case Study of Real-Time Emotional Biosensing

Biosensing displays, increasingly enrolled in emotional reflection, promise authoritative insight by presenting users’ emotions as discrete categories. Rather than machines interpreting emotions, we sought to explore an alternative with emotional biosensing displays in which users formed their own interpretations and felt comfortable critiquing the display. So, we designed, implemented, and deployed, as a technology probe, an emotional biosensory display: Ripple is a shirt whose pattern changes color responding to the wearer’s skin conductance, which is associated with excitement. 17 participants wore Ripple over 2 days of daily life. While some participants appreciated the ‘physical connection’ Ripple provided between body and emotion, for others Ripple fostered insecurities about ‘how much’ feeling they had. Despite our design intentions, we found participants rarely questioned the display’s relation to their feelings. Using biopolitics to speculate on Ripple’s surprising authority, we highlight ethical stakes of biosensory representations for sense of self and ways of feeling

Emotional Biosensing: Exploring Critical Alternatives

Emotional biosensing is rising in daily life: Data and categories claim to know how people feel and suggest what they should do about it, while CSCW explores new biosensing possibilities. Prevalent approaches to emotional biosensing are too limited, focusing on the individual, optimization, and normative categorization. Conceptual shifts can help explore alternatives: toward materiality, from representation toward performativity, inter-action to intra-action, shifting biopolitics, and shifting affect/desire. We contribute (1) synthesizing wide-ranging conceptual lenses, providing analysis connecting them to emotional biosensing design, (2) analyzing selected design exemplars to apply these lenses to design research, and (3) offering our own recommendations for designers and design researchers. In particular we suggest humility in knowledge claims with emotional biosensing, prioritizing care and affirmation over self- improvement, and exploring alternative desires. We call for critically questioning and generatively re- imagining the role of data in configuring sensing, feeling, ‘the good life,’ and everyday experience

Multi‐speaker experimental designs: Methodological considerations

Research on language use has become increasingly interested in the multimodal and interactional aspects of language – theoretical models of dialogue, such as the Communication Accommodation Theory and the Interactive Alignment Model are examples of this. In addition, researchers have started to give more consideration to the relationship between physiological processes and language use. This article aims to contribute to the advancement in studies of physiological and/or multimodal language use in naturalistic settings. It does so by providing methodological recommendations for such multi-speaker experimental designs. It covers the topics of (a) speaker preparation and logistics, (b) experimental tasks and (c) data synchronisation and post-processing. The types of data that will be considered in further detail include audio and video, electroencephalography, respiratory data and electromagnetic articulography. This overview with recommendations is based on the answers to a questionnaire that was sent amongst the members of the Horizon 2020 research network ‘Conversational Brains’, several researchers in the field and interviews with three additional experts.H2020 Marie Skłodowska‐Curie Actions http://dx.doi.org/10.13039/100010665Peer Reviewe

Systemic physiology augmented functional near-infrared spectroscopy hyperscanning: a first evaluation investigating entrainment of spontaneous activity of brain and body physiology between subjects.

Significance: Functional near-infrared spectroscopy (fNIRS) enables measuring the brain activity of two subjects while they interact, i.e., the hyperscanning approach. Aim: In our exploratory study, we extended classical fNIRS hyperscanning by adding systemic physiological measures to obtain systemic physiology augmented fNIRS (SPA-fNIRS) hyperscanning while blocking and not blocking the visual communication between the subjects. This approach enables access brain-to-brain, brain-to-body, and body-to-body coupling between the subjects simultaneously. Approach: Twenty-four pairs of subjects participated in the experiment. The paradigm consisted of two subjects that sat in front of each other and had their eyes closed for 10 min, followed by a phase of 10 min where they made eye contact. Brain and body activity was measured continuously by SPA-fNIRS. Results: Our study shows that making eye contact for a prolonged time causes significant changes in brain-to-brain, brain-to-body, and body-to-body coupling, indicating that eye contact is followed by entrainment of the physiology between subjects. Subjects that knew each other generally showed a larger trend to change between the two conditions. Conclusions: The main point of this study is to introduce a new framework to investigate brain-to-brain, body-to-body, and brain-to-body coupling through a simple social experimental paradigm. The study revealed that eye contact leads to significant synchronization of spontaneous activity of the brain and body physiology. Our study is the first that employed the SPA-fNIRS approach and showed its usefulness to investigate complex interpersonal physiological changes

A Framework of Distinct Musical Chills: Theoretical, Causal, and Conceptual Evidence

The phenomenon of musical chills has attracted extensive attention in previous music and emotion research, correlating the experience with musical structure, psychoacoustics parameters, individual differences in listeners, and the listening situation. However, there are three crucial limitations in the literature: 1) The emotional characteristics of musical chills have not been explored, and are poorly understood; 2) musical chills have never been causally manipulated, and no theories have been tested; and 3) it is unclear whether chills are a unified psychological construct, or a set of distinct experiences, distinguished at the levels of subjective feeling, psychophysiological response, individual differences, and underlying psychological induction mechanisms. Across five studies, ranging from qualitative surveys to experimental manipulations of musical chills, these limitations were addressed in the current thesis, with results suggesting firstly that musical chills are often mixed emotional experiences, described as moving, bittersweet and intense; secondly, that musical chills can be manipulated, and corresponding theories tested, with a novel experimental paradigm, by removing key sections in a piece or changing psychoacoustic parameters such as loudness and brightness; finally, that there are likely distinct types of chills experiences, which across multimedia are linked to both the affective dimension of valence and individual differences such as trait empathy, and with music through mechanisms of fear and vigilance on the one hand, and social bonding on the other. The studies and results are discussed in terms of two categories of musical chills experiences, culminating in a preliminary Distinct Musical Chills Framework, producing a series of testable hypotheses for future empirical work, and a comprehensive research agenda for the field moving forward

‘What would happen if I said “Yes”?’ : Measuring the immediate and long-term impact of improvisation training on student teachers’ subjective, neuroendocrine and psychophysiological responses

Improvisation is commonly understood as a performance or creating something without preparation. As an art form, improvised theatrical plays are created spontaneously on stage without a script. As an applied form of theatre, improvisation has been utilised in fields requiring collaboration and a tolerance for uncertainty, such as in the business and education sectors. This dissertation contributes to the literature in educational research by investigating applied improvisation as a tool to promote student teachers’ interpersonal competence. Applied improvisation enables individuals to explore and practise teaching-related encounters in a fictional and psychologically safe context. Psychological safety is particularly important when practising challenging interactions. Despite the fictionality of the context, bodily experiences during improvisations may promote experiential learning. The research summarised in this dissertation was guided by two primary research questions. First, I asked whether improvisation training influenced student teachers’ interpersonal competence and social stress. Student teachers (n = 19) participated in a 7-week (17.5-h) improvisation intervention, comprising the fundamentals of theatre improvisation and status expression (verbal and nonverbal behaviours indicating the social dominance of a person). The impact of the intervention was measured using subjective self-reports (interpersonal confidence, i.e., belief regarding one’s capability related to effective social interactions, self-esteem and experienced stress) and a large array of physiological measurements (heart rate, heart rate variability, skin conductance, facial muscle activity, frontal electroencephalogram (EEG) alpha asymmetry and stress hormone cortisol). Self-reports, physiological measurements and Trier Social Stress Tests (TSST; including public speaking) were performed before and after the improvisation intervention. An improvisation course was arranged for the control group (n = 20) following the intervention study. One year later, the long-term effects of improvisation training on self-reported interpersonal confidence were measured in a follow-up study. Second, I asked how real versus fictional social rejections impact experienced stress and psychophysiological responses. Student teachers (n = 39) participated in an experiment including both real (interview) and fictional (improvisation exercises) dyadic interactions. In the real condition, student teachers were unaware that the interviewer was an actor trained to include subtle social rejections during the interview by using three types of social rejections: devaluing, interrupting and nonverbal rejections. In the fictional condition, student teachers were informed in advance which social rejection type would be used during a later improvisation exercise. Experienced stress and psychophysiological reactivity during social rejections were measured under both experimental conditions. Following an improvisation intervention, interpersonal confidence and its components of performance confidence and a tolerance for failure increased relative to controls, whilst one year later the improved performance confidence persisted. Furthermore, a heterogeneous treatment effect was found. Those with the lowest pretest interpersonal confidence score benefited most from the improvisation intervention. No between-group differences in self-esteem were observed. Psychological and physiological indications of relief from performance-related stress were also observed following improvisation training. In addition, interpersonal confidence moderated self-reported and cardiovascular stress responses. Thus, interpersonal confidence may be worth controlling for in future research which examines the effects of interventions aimed at relieving social stress. The results also support the notion that repetition may also diminish performance-related stress, since the control group exhibited decreases in cardiovascular stress during some of the test conditions. The primary finding regarding the second research question emerged through the absence of any systematic attenuation of the psychophysiological reactivity to fictional versus real-world social rejections. In other words, although student teachers knew that improvised social rejections were fictional, their psychophysiological responses during improvisation remained relatively similar and associated with those of real-world rejections. It appears as though personal relevance and engagement during improvisation explain the relatively similar bodily responses. This result suggests that interpersonal encounters can be realistically modelled through applied improvisation. In this dissertation research, I also produced a validated self-report measure, the Interpersonal Confidence Questionnaire (ICQ), to evaluate the impact of social interaction training relying on applied improvisation. Using an additional dataset (n = 208), I validated the questionnaire and examined the impact of improvisation training on a larger sample. A confirmatory factor analysis identified six factors—performance confidence, flexibility, listening skills, a tolerance for failure, collaboration motivation and presence—that contribute to interpersonal confidence. Thus, the ICQ appeared valid and reliable as a self-report measure of interpersonal confidence. In summary, the findings from this research indicate that a relatively brief improvisation intervention promotes interpersonal confidence, specifically amongst those with low interpersonal confidence. Furthermore, improvisation training serves as an intervention against performance anxiety and generates long-term improvements to performance confidence. This dissertation provides a theoretical framework and empirical support for the application of improvisation as a tool to develop interpersonal competence skills, particularly within professions requiring face-to-face interactions. Regardless of the fictionality of the improvisational context, genuine emotions and experiences may emerge, serving as experiential learning experiences. The significance of these findings may extend to theatre-based practices and drama education in general, which rely on holistic action and personal engagement in fictional contexts. The findings agree with previous research, suggesting that including the improvisation method in teacher education curricula can enhance student teachers’ interpersonal competence as well as their skills related to sensitive and responsive teaching. Finally, this dissertation contributes to social neuroscience by recommending an ecologically valid experimental design wherein naturally unfolding social interactions can be achieved using improvisation techniques. Keywords: experiential learning, fictionality, improvisation, interpersonal confidence, intervention, psychophysiology, social interaction, social rejection, social stress, teacher education, theatre-based practicesImprovisaatiolla tarkoitetaan yleisesti esiintymistä tai toimintaa, jota ei suunnitella etukäteen. Teatterissa improvisoidut näytelmät syntyvät näyttämöllä spontaanisti ilman käsikirjoitusta. Soveltava teatteri hyödyntää improvisaatiota aloilla, joissa tarvitaan yhteistyötä ja epävarmuuden sietokykyä, kuten liike-elämässä ja koulutuksessa. Tämä väitöstyö tarkastelee teatterilähtöistä improvisaatiomenetelmää opettajakoulutuksen kontekstissa. Käyttämällä improvisaatiota soveltavan teatterin muotona on mahdollista tutkia ja harjoitella vuorovaikutukseen liittyviä kohtaamisia kuvitteellisessa ja psykologisesti turvallisessa ympäristössä. Psykologinen turvallisuus on erityisen tärkeää, kun harjoitellaan haastavissa vuorovaikutustilanteissa toimimista. Ympäristön fiktiivisyydestä, eli kuvitteellisuudesta, huolimatta improvisaatioharjoittelu tarjoaa kehollisia kokemuksia, jotka voivat edistää kokemuksellista oppimista. Väitöstutkimuksen ensimmäisenä tavoitteena oli selvittää improvisaatioharjoittelun vaikutuksia opettajaopiskelijoiden vuorovaikutusosaamiseen ja sosiaaliseen stressiin. Opettajaopiskelijoille (N = 19) järjestettiin 7 viikon (17,5 h) interventio, jossa harjoiteltiin improvisaation perusteita ja statusilmaisua, eli sanallista ja sanatonta ilmaisua liittyen valta-asemaan vuorovaikutuksessa. Intervention vaikutuksia kartoitettiin itsearviointien (vuorovaikutusvarmuus, itsetunto ja koettu stressi) kautta ja kehollisia vaikutuksia mittaamalla opettajaopiskelijoiden kehollisia vasteita (syke, sykevälivaihtelu, ihon sähkönjohtavuus, kasvolihasten aktivaatio, aivosähkökäyrä ja stressihormoni kortisoli). Fysiologiset mittaukset, Trierin sosiaalisen stressin testit (mm. julkinen puhe) sekä itsearvioinnit suoritettiin ennen improvisaatiointerventiota ja sen jälkeen. Kontrolliryhmälle (N = 20) järjestettiin improvisaatiokurssi loppumittausten jälkeen. Vuorovaikutusvarmuutta mitattiin viivästetyllä kyselyllä vielä vuoden kuluttua interventiosta. Väitöstutkimuksen toinen tavoite oli syventää ymmärrystä siitä, miten tietoisuus tilanteen fiktiivisyydestä vaikuttaa kokemukseen vertaamalla aidon ja fiktiivisen vuorovaikutustilanteen kehollisia vasteita. Opettajaopiskelijat (N = 39) osallistuivat kokeeseen, joka sisälsi sekä aidon (haastattelu) että fiktiivisen (improvisaatioharjoittelu) vuorovaikutustilanteen. Aidossa tilanteessa opettajaopiskelijat eivät tienneet, että haastattelijan tehtävänä oli epäsuorasti torjua opiskelijoiden vastauksia vähättelemällä, keskeyttämällä ja sanattomalla torjunnalla. Fiktiivisessä tilanteessa opettajaopiskelijoille kerrottiin etukäteen mitä em. torjuntatapaa improvisaatioharjoituksessa käytetään. Molemmissa koetilanteissa mitattiin koettua stressiä ja kehollisia vasteita epäsuorien sosiaalisten torjuntojen aikana. Improvisaatioharjoittelu lisäsi osallistujien vuorovaikutusvarmuutta sekä sen osatekijöitä esiintymisvarmuutta ja epäonnistumisen sietokykyä, kun tuloksia verrattiin kontrolliryhmään. Vuoden kuluttua interventiosta esiintymisvarmuus säilyi korkeammalla tasolla kontrolliryhmään verrattuna. Tulosten mukaan improvisaatioharjoittelusta oli hyötyä erityisesti epävarmimmille opettajaopiskelijoille, joilla vuorovaikutus- ja esiintymisvarmuus kasvoi eniten. Itsetunnossa ei havaittu eroa interventio- ja kontrolliryhmien välillä. Sykevälivaihtelu ja kasvolihasten aktivaatio osoittivat, että improvisaatiokurssilaiset olivat rentoutuneempia esiintymisvuoroa odottaessa kuin kontrolliryhmä. Sykevälivaihtelun ja koetun stressin mukaan epävarmimpien kurssilaisten stressitaso laski enemmän kuin varmempien. Tulokset tukevat myös käsitystä, että toistoharjoittelu voi lieventää esiintymiseen liittyvää stressiä, koska myös kontrolliryhmässä stressivasteet laskivat osassa esiintymistilanteita. Väitöstutkimuksen keskeinen löydös vertailtaessa aitoja ja fiktiivisiä vuorovaikutustilanteita oli se, että improvisoitujen ja aitojen sosiaalisten torjuntojen keholliset vasteet eivät systemaattisesti eronneet toisistaan. Toisin sanoen, vaikka opettajaopiskelijat tiesivät, että improvisoidut sosiaaliset torjunnat olivat fiktiivisiä, keholliset vasteet olivat samankaltaisia kuin vastaavassa aidossa tilanteessa. Tulokset viittaavat siihen, että sosiaalisia vuorovaikutustilanteita voidaan mallintaa realistisesti teatterilähtöisen improvisaation keinoin. Väitöstyö tuotti myös validoidun vuorovaikutusvarmuuden itsearviointimittarin improvisaatiopohjaisten vuorovaikutuskoulutusten vaikutusten arviointiin. Itsearviointimittarin validoinnissa käytettiin konfirmatorista faktorianalyysia ja laajempaa kyselyaineistoa (N = 208). Faktorianalyysi osoitti, että vuorovaikutusvarmuuteen sisältyy kuusi osa-aluetta: esiintymisvarmuus, joustavuus, kuunteleminen, epäonnistumisen sietokyky, yhteistyömotivaatio ja läsnäolo. Tulosten mukaan mittari on myös luotettava vuorovaikutusvarmuuden itsearvioinnin väline. Tutkimustulosten mukaan jo suhteellisen lyhyt improvisaatioharjoittelu voi lisätä erityisesti epävarmojen henkilöiden vuorovaikutusvarmuutta sekä lieventää esiintymisjännitystä. Esiintymisvarmuuden osalta positiiviset vaikutukset olivat pitkäaikaisia. Väitöstyö tuottaa myös uutta, empiiristä tietoa improvisaatioharjoittelun kehollisista vaikutuksista. Tulosten merkitystä voidaan ulottaa myös muille soveltavan teatterin alueille, joissa teatterilähtöisillä menetelmillä pyritään tuottamaan elämyksellisiä oppimiskokemuksia ja kokemuksellista oppimista. Väitöstutkimus luo teoreettista pohjaa sovelletun improvisaation käyttämiselle laajemminkin työelämän ja arkipäivän vuorovaikutustilanteiden mallintamiseen. Löydökset vahvistavat aikaisempia kansainvälisiä tutkimustuloksia, joiden mukaan improvisaatio opettajankoulutuksen osana lisäisi opettajan vuorovaikutuskompetenssia sekä opetuksen sensitiivisyyttä ja vuorovaikutteisuutta. Poikkitieteellinen väitöstyö osoitti myös käytännön suuntaviivoja improvisaation ja neurotieteen yhdistämiselle vuorovaikutuksen kokeellisessa tutkimuksessa. Avainsanat: fiktiivisyys, improvisaatio, interventio, kokemuksellinen oppiminen, opettajakoulutus, psykofysiologia, sosiaalinen stressi, sosiaalinen torjunta, teatterilähtöiset menetelmät, vuorovaikutus, vuorovaikutusvarmuu