Audiovisual integration of emotional signals from others' social interactions

Audiovisual perception of emotions has been typically examined using displays of a solitary character (e.g., the face-voice and/or body-sound of one actor). However, in real life humans often face more complex multisensory social situations, involving more than one person. Here we ask if the audiovisual facilitation in emotion recognition previously found in simpler social situations extends to more complex and ecological situations. Stimuli consisting of the biological motion and voice of two interacting agents were used in two experiments. In Experiment 1, participants were presented with visual, auditory, auditory filtered/noisy, and audiovisual congruent and incongruent clips. We asked participants to judge whether the two agents were interacting happily or angrily. In Experiment 2, another group of participants repeated the same task, as in Experiment 1, while trying to ignore either the visual or the auditory information. The findings from both experiments indicate that when the reliability of the auditory cue was decreased participants weighted more the visual cue in their emotional judgments. This in turn translated in increased emotion recognition accuracy for the multisensory condition. Our findings thus point to a common mechanism of multisensory integration of emotional signals irrespective of social stimulus complexity

Perception of emotion in social interactions from body movement and voice

The central theme of this thesis was to examine different aspects related to the observation and judgement of emotions from the body movement and voice of two actors engaged in social interaction. There were four major goals related to this theme. The first goal was to create a novel stimulus set for the study of emotional social interactions. The second was to validate the created stimulus set by examining emotion perception in ways similar to that done with single actor displays. The third goal was to examine the effect of degrading visual and auditory information on the perception of emotional social interactions. The final goal was focused on the multimodal integration of emotional signals from body movement and voice. Initially, a stimulus set was created that incorporated body movement and dialogue between two actors in brief, natural interactions that were happy, angry or neutral at different levels of intensity. The stimulus set was captured using a Vicon motion and voice capture system and included a group of nine professional and non-professional actors. This resulted in a corpus of 756 dyadic, multimodal, emotional interactions. A series of experiments were conducted presenting participants with visual point-light displays, auditory voice dialogues or combinations of both visual and auditory displays. Observers could accurately identify happy and angry interactions from dyadic displays and voice. The intensity of expressions influenced the accuracy of the emotional identification but only for angry rather than happy displays. After validation of the stimulus set, a subset was selected for further studies. Various methods of auditory and visual distortion were tested separately for each modality to examine the effect of those distortions on recognition of emotions from body movement and voice. Results for dyadic point-light displays followed similar findings from single actor displays that inversion and scrambling decreased the overall accuracy of emotion judgements. An effect of viewpoint was also found, indicating that observation of interaction from a side viewpoint was easier for emotion detection than observation of interaction from an oblique viewpoint. In the case of voice, methods of brown noise and low-pass filtering were shown to degrade emotion identification. However, with both visual and auditory methods of distortion, participants were still able to identify emotions above the level of chance, suggesting high sensitivity to emotional cues in a social context. In the final set of studies, the stimulus set was used in a multimodal context to examine the perception of emotion from movement and voice in dyadic social interactions. It was repeatedly found that voice dominated body movement as a cue to emotions when observing social interactions. Participants were less accurate and slower in emotion discrimination when they were making judgements from body movement only, compared to conditions when movement was combined with dialogue or when dialogue was presented on its own. Even when participants watched emotionally mismatched displays with combined movement and voice, they predominantly oriented their responses towards the voice rather than movement. This auditory dominance persisted even when the reliability of the auditory signal was degraded with brown noise or low-pass filtering, although visual information had some effect on judgements of emotion when it was combined with a degraded auditory signal. These results suggest that when judging emotions from observed social interactions, we rely primarily on vocal cues from conversation rather than visual cues from body movement

Regulating wellbeing in the brave new quantified workplace

Purpose: The purpose of the paper is to lay out the conceptual issues arising alongside the rise of sensory technologies in workplaces designed to improve wellness and productivity. Design/methodology/approach: This is a text based conceptual paper. Our approach is to throw light on some of the emerging issues with the introduction of wearable self-tracking technologies in workplaces. Findings: The paper indicates that scholars will need to put ethical issues at the heart of research on sensory tracking technologies in workplaces that aim to regulate employee behaviour via wellness initiatives. Originality/value: This is an original article. Since there is very little scholarly research in this area, it is important to begin to consider the implications of sensory technology in workplaces linked to wellness initiatives, given the probable impact it will have on work design and appraisal systems

A dyadic stimulus set of audiovisual affective displays for the study of multisensory, emotional, social interactions

We describe the creation of the first multisensory stimulus set that consists of dyadic, emotional, point-light interactions combined with voice dialogues. Our set includes 238 unique clips, which present happy, angry and neutral emotional interactions at low, medium and high levels of emotional intensity between nine different actor dyads. The set was evaluated in a between-design experiment, and was found to be suitable for a broad potential application in the cognitive and neuroscientific study of biological motion and voice, perception of social interactions and multisensory integration. We also detail in this paper a number of supplementary materials, comprising AVI movie files for each interaction, along with text files specifying the three dimensional coordinates of each point-light in each frame of the movie, as well as unprocessed AIFF audio files for each dialogue captured. The full set of stimuli is available to download from: http://​motioninsocial.​com/​stimuli_​set/​

Changing emotional engagement with running through communal self-tracking: The implications of ‘teleoaffective shaping’ for public health,

Emerging research explores the role of self-tracking in supporting healthy behaviour. Self-tracking comprises a number of interrelated practices; some individual some communal. In this paper we focus on practices that enable interaction between self-trackers through data sharing and communication around personal data. For public health, communal self-tracking has been explored for the additional benefits it provides in addition to self-knowledge. However, under-explored is the emotional entanglement of self-tracking and tracked activities, or the role of practitioners in the dynamic evolution of tracked practices. Qualitative, mixed methods data was collected from leisure-time runners in the SW England who self-track using social fitness app ‘Strava’, and was interpreted through the lens of practice theory. We find that communal self-tracking affords the active shaping of the emotion and purpose of running. This ‘teleoaffective shaping’ allows practitioners to negotiate and reconstitute appealing meanings associated with running to protect their practice loyalty. We identify three mechanisms for teleoaffective shaping afforded by Strava: labelling, reward and materialising effort. Findings advance our understanding of how social fitness apps work to retain practitioners of physically active leisure practices. Future research should further explore the multiple ways that associations with tracked physical activity evolve through entanglement with self-tracking practices

Failing to encourage physical activity with wearable technology:what next?

Wearable devices and associated systems that provide real-time feedback aim to encourage healthy behaviours. However, while the research base has grown considerably, results continue to paint a mixed picture when demonstrating wearables’ ability to increase levels of physical activity. Given these recent developments, this commentary explores the key reasons why wearable devices and other mobile technologies often fail to change behaviour. We also provide several suggestions that could feed into future research designs and maximise the success of subsequent interventions. These recommendations aim to stimulate interdisciplinary discussions by encouraging clinicians and researchers to consider how these technological advances can be effectively leveraged, and become a core component of preventative medicine in the 21st century

A simple location-tracking app for psychological research

Location data gathered from a variety of sources is particularly valuable when it comes to understanding individuals and groups. However, much of this work relies on participants’ active engagement to regularly report their location. More recently, smartphones have been used to assist with this process, but while commercial smartphone applications are available, these are often expensive and not designed with researchers in mind. In order to overcome these and other related issues, we have developed a freely available Android application that logs location accurately, stores data securely, and ensures participants can provide consent or withdraw from a study at any time. Further recommendations and R code are provided to assist with subsequent data analysis

Look at those two!:The precuneus role in unattended third-person perspective of social interactions

Human beings often observe other people's social interactions without being a part of them. Whereas the implications of some brain regions (e.g. amygdala) have been extensively examined, the implication of the precuneus remains yet to be determined. Here we examined the implication of the precuneus in third-person perspective of social interaction using functional magnetic resonance imaging (fMRI). Participants performed a socially irrelevant task while watching the biological motion of two agents acting in either typical (congruent to social conventions) or atypical (incongruent to social conventions) ways. When compared to typical displays, the atypical displays elicited greater activation in the central and posterior bilateral precuneus, and in frontoparietal and occipital regions. Whereas the right precuneus responded with greater activation also to upside down than upright displays, the left precuneus did not. Correlations and effective connectivity analysis added consistent evidence of an interhemispheric asymmetry between the right and left precuneus. These findings suggest that the precuneus reacts to violations of social expectations, and plays a crucial role in third-person perspective of others' interaction even when the social context is unattended

The Bumps and BaBies Longitudinal Study (BaBBLeS): a multi-site cohort study of first-time mothers to evaluate the effectiveness of the Baby Buddy app

Background: Health mobile applications (apps) have become very popular, including apps specifically designed to support women during the ante- and postnatal periods. However, there is currently limited evidence for the effectiveness of such apps at improving pregnancy and parenting outcomes. Aim: to assess the effectiveness of a pregnancy and perinatal app, Baby Buddy, in improving maternal self-efficacy at three months post-delivery. Methods: Participants were 16-years and over, first-time pregnant women, 12-16 weeks gestation, recruited from five English study sites. The Tool of Parenting Self-efficacy (TOPSE) (primary outcome) was used to compare mothers at three months post-delivery who had downloaded the Baby Buddy app compared to those who had not downloaded the app, controlling for confounding factors. Results: 488 participants provided valid data at baseline (12-16 weeks gestation), 296 participants provided valid data at 3 months post-birth, 114 (38.5%) of whom reported that they had used the Baby Buddy app. Baby Buddy app users were more likely to use pregnancy or parenting apps (80.7% vs 69.6%, p=.035), more likely to have been introduced to the app by a healthcare professional (p=.005) and have a lower median score for perceived social support (81 vs 83, p=.034) than non-app users. The Baby Buddy app did not illicit a statistically significant change in TOPSE scores from baseline to 3 months post-birth (adjusted OR 1.12, 95%CI 0.59 to 2.13, p=.730). Finding out about the Baby Buddy app from a healthcare professional appeared to grant no additional benefit to app users compared to all other participants in terms of self-efficacy at three months post-birth (adjusted OR 1.16, 95%CI 0.60 to 2.23, p=.666). There were no statistically significant differences in the TOPSE scores for the in-app data between either the type of user who was engaged with the app and non-app users (adjusted OR 0.69, 95%CI 0.22 to 2.16, p=.519) or those who were highly engaged and non-app users (adjusted OR 0.48, 95%CI 0.14t o 1.68, p=.251). Conclusion: This study is one of few, to date, that has investigated the effectiveness of a pregnancy and early parenthood app. No evidence for the effectiveness of the Baby Buddy app was found. New technologies can enhance traditional healthcare services and empower users to take more control over their healthcare but app effectiveness needs to be assessed. Further work is needed to consider, a) how we can best use this new technology to deliver better health outcomes for health service users and, b) methodological issues of evaluating digital health interventions