Integrating art into bodily interactions : exploring digital art in HCI design to foster somaesthetic experiences

PhD ThesisMy interdisciplinary doctoral research of this thesis explored how interaction design – with a combination of digital art, body-centred practice and biophysical sensing technology – cultivates self-awareness and self-reflection to foster somaesthetic experiences in everyday walking. My research followed a Research through Design (RtD) approach to provide design artefacts as examples of research in the expanded territory of Somaesthetic Design, technology-enhanced body-centred practices and digital art applied in interaction design. Background research included a critical review of Affective Computing, the concept of somaesthetic experience, existing body-centred practices (e.g. mindfulness and deep listening), HCI designs for somaesthetic experiences, and interactive digital art applications (using biophysical data as input) to express bodily activities. In methodological terms the research could be summarized as a process of ‘making design theories’ (Redström, 2017) that draws upon a Research through Design (RtD) approach. The whole research process could be described with a ‘bucket’ model in making design theories (Redström, 2017): identified initial design space as the initial ‘bucket’; derived the first design artefact ‘Ambient Walk’ as a ‘fact’ to represent the initial design space and the cause of transitioning, re-accenting process from mindfulness to ‘adding a sixth-sense’ (i.e. to extend the initial ‘bucket’); the making of second design artefact ‘Hearing the Hidden’ as a ‘fact’ to represent the re-accented research rationale in designing for somaesthetic experience by ‘adding a sixth sense’. I followed a qualitative approach to evaluate individual user feedbacks on enhancing somaesthetic experiences, the aspects to be considered in designing for experiences, and how my design process contributed to refining design for experiences. At the end of this thesis, I discuss the findings from the two practical projects regarding the somaesthetic experiences that have been provoked during users’ engagement with ‘Ambient Walk’ and ‘Hearing the Hidden’; the inclusion of bodily interactions with surroundings in somaesthetic design; the use of ‘provotypes’ in experience-centred design practices; and the benefit of integrating digital art into technology for body-centred practices

Your Body of Water: A Somaesthetic Display for Embodied Reflection

In response to the Quantified Self movement, which uses body data for self-tracking and self-improvement, this thesis explores how aestheticized heart rate data can be used to get us more in touch with our bodies and how we are feeling. Utilizing somaesthetics, an interdisciplinary field with roots in philosophy that combines the soma (the living body) with aesthetics (our sensory perception and appreciation), this thesis explores how we can design interactions that help us to reflect on our embodied experience. Through a research through design process using somaesthetic appreciation design characteristics, I designed an interactive display that retrieved heart rate wirelessly with computer vision and then visualized one’s heart rate as water. The display was evaluated for somaesthetic characteristics using system critiques, and this evaluation method was found to be a timely and resource-effective way of evaluating a device for self reflection and embodiment

Mental Health Resources: Reflection on AffecTech Platform

This positon paper presents an overview of the mental health resources developed within the AffecTech, an interdisciplinary European Commission-funded project supporting the wok of 15 PhD students in the space of digital interventions for affective health. It describes different types of targeted resources highlighting both traditional and innovative ones, our approach to developing them, and the challenges and opportunities that our work has uncovered

Mental Wellbeing:Future Agenda Drawing from Design, HCI and Big Data

Most HCI work on the exploration and support of mental wellbeing involves mobiles, sensors, and various on-line systems which focus on tracking users. However, adoption of, and adherence to such systems is not ideal. Are there innovative ways to better design for mental wellbeing? A promising novel approach is to encourage changes to behavior through the use of tailored feedback informed by machine learning algorithms applied to large sets of use data. This one day workshop aims to explore novel ways to actively engage participants through interactive systems, with an overall aim to shape the research agenda of future HCI work on mental wellbeing. The workshop is designed in an innovative format offering a mixture of traditional presentation, hands-on design and future-thinking activities. The workshop brings together both practitioners and HCI researchers from across a range areas addressing mental wellbeing

Affective Umbrella – A Wearable System to Visualize Heart and Electrodermal Activity, towards Emotion Regulation through Somaesthetic Appreciation

In this paper, we introduce Affective Umbrella, a novel system to record, analyze and visualize physiological data in real time via an umbrella handle. We implement a biofeedback loop design in the system that triggers visualization changes to reflect and regulate emotions through somaesthetic appreciation. We report the methodology, processes, and results of data reliability and visual feedback impact on emotions. We evaluated the system using a real-life user study (n=21) in rainy weather at night. The statistical results demonstrate the potential of applying the visualization of biofeedback to regulate emotional arousal with a significantly higher (p=.0022) score, a lower (p=.0277) dominance than baseline from self-reported SAM Scale, and physiological arousal, which was shown to be significantly increased (p<.0001) with biofeedback in terms of pNN50 and a significant difference in terms of RMSSD. There was no significant difference in terms of emotional valence changes from SAM scale. Furthermore, we compared the difference between two biofeedback patterns (mirror and inversion). The mirror effect was with a significantly higher emotional arousal than the inversion effect (p=.0277) from SAM results and was with a significantly lower RMSSD performance than the inversion effect (p<.0001). This work demonstrates the potential for capturing physiological data using an umbrella handle and using this data to influence a user’s emotional state via lighting effects

Subtle Sensing:Detecting Differences in the Flexibility of Virtually Simulated Molecular Objects

During VR demos we have performed over last few years, many participants (in the absence of any haptic feedback) have commented on their perceived ability to 'feel' differences between simulated molecular objects. The mechanisms for such 'feeling' are not entirely clear: observing from outside VR, one can see that there is nothing physical for participants to 'feel'. Here we outline exploratory user studies designed to evaluate the extent to which participants can distinguish quantitative differences in the flexibility of VR-simulated molecular objects. The results suggest that an individual's capacity to detect differences in molecular flexibility is enhanced when they can interact with and manipulate the molecules, as opposed to merely observing the same interaction. Building on these results, we intend to carry out further studies investigating humans' ability to sense quantitative properties of VR simulations without haptic technology

Ming Shan Digital Experience

The Ming Shan Digital Experience is an immersive installation designed to support meditation in the context of a new Taoist center. Its creation confronted current academic literature on digital technology for meditation with the practical and cultural requirements of Taoist practice. Quantitative and qualitative learnings show the effectiveness of multimodal biofeedback on individual and collective meditative experience. Now instated in the Taoist center, the installation opens new perspectives for combining digital technology with ancient practice

Resonating Experiences of Self and Others enabled by a Tangible Somaesthetic Design

Digitalization is penetrating every aspect of everyday life including a human's heart beating, which can easily be sensed by wearable sensors and displayed for others to see, feel, and potentially "bodily resonate" with. Previous work in studying human interactions and interaction designs with physiological data, such as a heart's pulse rate, have argued that feeding it back to the users may, for example support users' mindfulness and self-awareness during various everyday activities and ultimately support their wellbeing. Inspired by Somaesthetics as a discipline, which focuses on an appreciation of the living body's role in all our experiences, we designed and explored mobile tangible heart beat displays, which enable rich forms of bodily experiencing oneself and others in social proximity. In this paper, we first report on the design process of tangible heart displays and then present results of a field study with 30 pairs of participants. Participants were asked to use the tangible heart displays during watching movies together and report their experience in three different heart display conditions (i.e., displaying their own heart beat, their partner's heart beat, and watching a movie without a heart display). We found, for example that participants reported significant effects in experiencing sensory immersion when they felt their own heart beats compared to the condition without any heart beat display, and that feeling their partner's heart beats resulted in significant effects on social experience. We refer to resonance theory to discuss the results, highlighting the potential of how ubiquitous technology could utilize physiological data to provide resonance in a modern society facing social acceleration.Comment: 18 page

Fiction and Physicality: a designerly approach towards complexities of emerging technologies

Rapid technological progression results in exciting new ways of interacting with our world whilst simultaneously limiting our experiences. Due to the pervasiveness of emerging technologies, designers are constantly faced with complexities and challenges, which necessitate the use of various tools and methodologies. This paper combines inspiration from the fields of aesthetics of interaction (Overbeek 1999), somaesthetics (Shusterman, 1999), design ethnography (Salvador, 1999), design fiction (Bleecker, 2009) and speculative design (Auger, 2013), to explore a designerly way of overcoming the complexity of implementing technologies into our daily life. We propose a holistic design approach to envision possibilities for emerging technology, integrating the physicality of human bodies with technological materiality. Further, we present a plausible narrative, containing visionary aspects and the investigated methodologies, alongside a series of design concepts that drive the storyline and form the basis for examining social implications, design and future contexts, and improving the way in which designers handle the limitations of a technology driven design approach