Biosensing and Actuation—Platforms Coupling Body Input-Output Modalities for Affective Technologies

Research in the use of ubiquitous technologies, tracking systems and wearables within mental health domains is on the rise. In recent years, affective technologies have gained traction and garnered the interest of interdisciplinary fields as the research on such technologies matured. However, while the role of movement and bodily experience to affective experience is well-established, how to best address movement and engagement beyond measuring cues and signals in technology-driven interactions has been unclear. In a joint industry-academia effort, we aim to remodel how affective technologies can help address body and emotional self-awareness. We present an overview of biosignals that have become standard in low-cost physiological monitoring and show how these can be matched with methods and engagements used by interaction designers skilled in designing for bodily engagement and aesthetic experiences. Taking both strands of work together offers unprecedented design opportunities that inspire further research. Through first-person soma design, an approach that draws upon the designer’s felt experience and puts the sentient body at the forefront, we outline a comprehensive work for the creation of novel interactions in the form of couplings that combine biosensing and body feedback modalities of relevance to affective health. These couplings lie within the creation of design toolkits that have the potential to render rich embodied interactions to the designer/user. As a result we introduce the concept of “orchestration”. By orchestration, we refer to the design of the overall interaction: coupling sensors to actuation of relevance to the affective experience; initiating and closing the interaction; habituating; helping improve on the users’ body awareness and engagement with emotional experiences; soothing, calming, or energising, depending on the affective health condition and the intentions of the designer. Through the creation of a range of prototypes and couplings we elicited requirements on broader orchestration mechanisms. First-person soma design lets researchers look afresh at biosignals that, when experienced through the body, are called to reshape affective technologies with novel ways to interpret biodata, feel it, understand it and reflect upon our bodies

Designing Monstrous Experiences Through Soma Design

There is currently a wave of research and development of novel on-body technologies and materials, including shape-changing technologies to be worn on or used close to the body. Traditional interaction design methods and interface models are not always a good fit for designing meaningful interactions with these technologies as they primarily interact with our somatic selves—not our language-oriented, symbol-processing ways of being in the world. Design researchers are therefore borrowing ideas and methods from the arts and humanities to design with and help to ”humanize” these technologies. One such approach, originating in pragmatist philosophy, is a design program named soma design. Soma design grounds design processes in our first-person sensuous, subjective experience of the world, aiming to deepen our aesthetic appreciation of ourselves and the world that surrounds us. Soma design has previously mostly addressed gentle, soft experiences. The work,I came to engage in, shifted away from the solely soft and comfortable, and instead addressed borderline uncomfortable, at times uncanny, yet evocative experiences. To shed light on and better articulate those experiences, I turned to monster theories, in particular, using the lens of the monster from the humanities, alongside the impurity concept from social anthropology. In my work, I define these monstrous experiences as ambiguous experiences, which transgress the seemingly opposing experiential categories of comfort versus discomfort. I did so through designing with technologies that apply restriction, pressure, shape-changing, weight-shifting onto our bodies that introduced ways of alienating us from our own bodily rhythms such as our breathing rhythm. As I argue, monstrous experiences constitute one way of addressing some of the dividing dualistic rifts in interaction design. Through the analysis of my design studies, I show how monstrous experiences help to dissolve the distinction between the inside and the outside of the human body—especially when they come into contact with each other through wearable technology, questioning and bridging the boundaries of where one ends and the other begins. In my design experiments, monster experiences are bordering between the human-like and organic versus the alien and unnatural. Monstrous experiences thus expand the repertoire of soma design, adding more ”radical” design exemplars. They help with ”humanizing” the domain of uncomfortable experiences through providing a different articulation, rooted in literature and art. My work on monstrous experiences makes two main contributions to design for somatic experiences. First, the monster can serve as an analytical lens that bridges dichotomies such as comfort/discomfort, inside/outside, or leader/follower. It provides another perspective from which such opposing categories, despite their seemingly built-in contradictions, can be united into a coherent whole. Second, monstrous experiences can serve as an estrangement method, a design tool for discovering new ways of connecting with our somatic selves, the world that surrounds us or other people, as well as new ways of using sensors and actuators to address, change, affect or alienate bodily processes. Monstrous experiences can help to explore the space between soft/hard and pleasant/uncomfortable experiences, thus, offering an alternative perspective on ”naturalness” in interaction design.Vi är mitt i en våg av forskning och utveckling av nya kroppsnära bärbara teknologier och material, inklusive formförändrande teknologier som appliceras på eller nära kroppen. Traditionella metoder för interaktionsdesign och gränssnittsmodeller är inte alltid lämpliga för att utforma meningsfulla interaktioner med dessa teknologier eftersom de primärt interagerar med vårt somatiska jag---inte våra språkorienterade, symbolbearbetande sätt att vara i världen. Designforskare lånar därför idéer och metoder från konst och humaniora för att designa och hjälpa till att ”humanisera” dessa teknologier. Ett sådant tillvägagångssätt, med grund i pragmatistisk filosofi, är ett designprogram som heter soma design. Soma design utgår från designprocesser grundade i individens personliga,  sensuella, subjektiva upplevelser av världen, med syfte att fördjupa vår estetiska uppskattning av oss själva och världen som omger oss. Soma design har tidigare mestadels fokuserat på lugna, stillsamma upplevelser. I mitt arbete valde jag att inte arbeta med mjuka och bekväma upplevelser utan fokuserade istället på upplevelser som gränsar till det obehagliga, ibland märkliga, men samtidigt suggestiva upplevelser. För att förstå och bättre artikulera dessa upplevelser valde jag att använda monster teorier, mer specifikt monster teorier från humaniora, samt det som kallats orena eller förorenade upplevelse i social-antropologin. I mitt arbete har jag kommit att definiera monstruösa upplevelser som ambiguösa upplevelser som överskrider eller upplöser synbart motsatta kategorier såsom behagliga versus obehagliga upplevelser. Det gör jag genom att designa med teknik på kroppen som begränsar, trycker, ändrar sin form, skapar viktförändringar, eller alienerar oss från våra egna kroppsliga rytmer, såsom vår andningsrytm. Dessa monstruösa erfarenheter utgör ett sätt att hantera några av de problem som uppstår när vi gör alltför skarpa dualistiska åtskillnader i interaktionsdesign. Genom analysen av mina designstudier visar jag hur monstruösa upplevelser hjälper till att upplösa idéer om skillnaden mellan insidan och utsidan av människokroppen -- speciellt när insidan och utsidan kommer i kontakt med varandra genom bärbar teknologi där de i någon mening ifrågasätter och överbryggar gränserna för var insidan slutar och utsidan börjar. I mina designexperiment upplöser monsterupplevelserna  gränserna mellan det mänskliga och organiska versus det främmande och onaturliga. Monstruösa erfarenheter utökar därmed soma design-repertoaren genom att addera fler, mer "radikala", design exempel. De hjälper till att "humanisera" de obekväma upplevelserna genom att tillhandahålla en annan artikulation, rotad i litteratur och konst. Mitt arbete med monstruösa upplevelser ger två huvudsakliga bidrag till design för somatiska upplevelser. För det första kan monstret fungera som en analytisk lins som överbryggar dikotomier såsom komfort/obehag, insida/utsida, eller ledare/följare. Det ger ett annat perspektiv från vilken sådana motsatta kategorier, trots deras till synes inbyggda motsägelser, kan förenas till en sammanhängande helhet. För det andra kan monstruösa upplevelser fungera som en förfrämlingsmetod, ett designverktyg för att upptäcka nya sätt att få kontakt med  våra somatiska jag, den värld som omger oss eller andra människor, såväl som  nya sätt att använda sensorer och aktuatorer för att adressera, förändra, påverka eller främmandegöra kroppsliga processer. Monstruösa upplevelser kan därmed hjälpa till att utforska utrymmet mellan mjuka/hårda och trevliga/obekväma upplevelser, och därmed erbjuda ett alternativt perspektiv på ”naturlighet" i interaktionsdesign.The room for the public defence have been altered, from room D3 to D2.QC 20230516</p

Designing Monstrous Experiences Through Soma Design

There is currently a wave of research and development of novel on-body technologies and materials, including shape-changing technologies to be worn on or used close to the body. Traditional interaction design methods and interface models are not always a good fit for designing meaningful interactions with these technologies as they primarily interact with our somatic selves—not our language-oriented, symbol-processing ways of being in the world. Design researchers are therefore borrowing ideas and methods from the arts and humanities to design with and help to ”humanize” these technologies. One such approach, originating in pragmatist philosophy, is a design program named soma design. Soma design grounds design processes in our first-person sensuous, subjective experience of the world, aiming to deepen our aesthetic appreciation of ourselves and the world that surrounds us. Soma design has previously mostly addressed gentle, soft experiences. The work,I came to engage in, shifted away from the solely soft and comfortable, and instead addressed borderline uncomfortable, at times uncanny, yet evocative experiences. To shed light on and better articulate those experiences, I turned to monster theories, in particular, using the lens of the monster from the humanities, alongside the impurity concept from social anthropology. In my work, I define these monstrous experiences as ambiguous experiences, which transgress the seemingly opposing experiential categories of comfort versus discomfort. I did so through designing with technologies that apply restriction, pressure, shape-changing, weight-shifting onto our bodies that introduced ways of alienating us from our own bodily rhythms such as our breathing rhythm. As I argue, monstrous experiences constitute one way of addressing some of the dividing dualistic rifts in interaction design. Through the analysis of my design studies, I show how monstrous experiences help to dissolve the distinction between the inside and the outside of the human body—especially when they come into contact with each other through wearable technology, questioning and bridging the boundaries of where one ends and the other begins. In my design experiments, monster experiences are bordering between the human-like and organic versus the alien and unnatural. Monstrous experiences thus expand the repertoire of soma design, adding more ”radical” design exemplars. They help with ”humanizing” the domain of uncomfortable experiences through providing a different articulation, rooted in literature and art. My work on monstrous experiences makes two main contributions to design for somatic experiences. First, the monster can serve as an analytical lens that bridges dichotomies such as comfort/discomfort, inside/outside, or leader/follower. It provides another perspective from which such opposing categories, despite their seemingly built-in contradictions, can be united into a coherent whole. Second, monstrous experiences can serve as an estrangement method, a design tool for discovering new ways of connecting with our somatic selves, the world that surrounds us or other people, as well as new ways of using sensors and actuators to address, change, affect or alienate bodily processes. Monstrous experiences can help to explore the space between soft/hard and pleasant/uncomfortable experiences, thus, offering an alternative perspective on ”naturalness” in interaction design.Vi är mitt i en våg av forskning och utveckling av nya kroppsnära bärbara teknologier och material, inklusive formförändrande teknologier som appliceras på eller nära kroppen. Traditionella metoder för interaktionsdesign och gränssnittsmodeller är inte alltid lämpliga för att utforma meningsfulla interaktioner med dessa teknologier eftersom de primärt interagerar med vårt somatiska jag---inte våra språkorienterade, symbolbearbetande sätt att vara i världen. Designforskare lånar därför idéer och metoder från konst och humaniora för att designa och hjälpa till att ”humanisera” dessa teknologier. Ett sådant tillvägagångssätt, med grund i pragmatistisk filosofi, är ett designprogram som heter soma design. Soma design utgår från designprocesser grundade i individens personliga,  sensuella, subjektiva upplevelser av världen, med syfte att fördjupa vår estetiska uppskattning av oss själva och världen som omger oss. Soma design har tidigare mestadels fokuserat på lugna, stillsamma upplevelser. I mitt arbete valde jag att inte arbeta med mjuka och bekväma upplevelser utan fokuserade istället på upplevelser som gränsar till det obehagliga, ibland märkliga, men samtidigt suggestiva upplevelser. För att förstå och bättre artikulera dessa upplevelser valde jag att använda monster teorier, mer specifikt monster teorier från humaniora, samt det som kallats orena eller förorenade upplevelse i social-antropologin. I mitt arbete har jag kommit att definiera monstruösa upplevelser som ambiguösa upplevelser som överskrider eller upplöser synbart motsatta kategorier såsom behagliga versus obehagliga upplevelser. Det gör jag genom att designa med teknik på kroppen som begränsar, trycker, ändrar sin form, skapar viktförändringar, eller alienerar oss från våra egna kroppsliga rytmer, såsom vår andningsrytm. Dessa monstruösa erfarenheter utgör ett sätt att hantera några av de problem som uppstår när vi gör alltför skarpa dualistiska åtskillnader i interaktionsdesign. Genom analysen av mina designstudier visar jag hur monstruösa upplevelser hjälper till att upplösa idéer om skillnaden mellan insidan och utsidan av människokroppen -- speciellt när insidan och utsidan kommer i kontakt med varandra genom bärbar teknologi där de i någon mening ifrågasätter och överbryggar gränserna för var insidan slutar och utsidan börjar. I mina designexperiment upplöser monsterupplevelserna  gränserna mellan det mänskliga och organiska versus det främmande och onaturliga. Monstruösa erfarenheter utökar därmed soma design-repertoaren genom att addera fler, mer "radikala", design exempel. De hjälper till att "humanisera" de obekväma upplevelserna genom att tillhandahålla en annan artikulation, rotad i litteratur och konst. Mitt arbete med monstruösa upplevelser ger två huvudsakliga bidrag till design för somatiska upplevelser. För det första kan monstret fungera som en analytisk lins som överbryggar dikotomier såsom komfort/obehag, insida/utsida, eller ledare/följare. Det ger ett annat perspektiv från vilken sådana motsatta kategorier, trots deras till synes inbyggda motsägelser, kan förenas till en sammanhängande helhet. För det andra kan monstruösa upplevelser fungera som en förfrämlingsmetod, ett designverktyg för att upptäcka nya sätt att få kontakt med  våra somatiska jag, den värld som omger oss eller andra människor, såväl som  nya sätt att använda sensorer och aktuatorer för att adressera, förändra, påverka eller främmandegöra kroppsliga processer. Monstruösa upplevelser kan därmed hjälpa till att utforska utrymmet mellan mjuka/hårda och trevliga/obekväma upplevelser, och därmed erbjuda ett alternativt perspektiv på ”naturlighet" i interaktionsdesign.QC 20230516</p

Boards Hit Back : Reflecting on Martial Arts Practices Through Soma Design

There is an increasing interest in the HCI community in designing for bodily practices. We report on a soma design process for martial arts and the resulting artifact – an interactive wooden dummy. Through a detailed account of the design process, we show how it enriched and revamped the bodily practice, but also how it changed the martial arts expert in the design team. Based on a phenomenological account of his experiences, we argue that the estrangement methods in soma design may allow practitioners engaging as soma designers, to cultivate and create new artistic habits fused with thought and feeling, changing themselves and their practice in directions they seek.QC 20230608</p

The Body Electric: A NIME designed through and with the somatic experience of singing

This paper presents the soma design process of creating Body Electric: a novel interface for the capture and use of biofeedback signals and physiological changes generated in the body by breathing, during singing. This NIME design is grounded in the performer's experience of, and relationship to, their body and their voice. We show that NIME design using principles from soma design can offer creative opportunities in developing novel sensing mechanisms, which can in turn inform composition and further elicit curious engagements between performer and artefact, disrupting notions of performer-led control. As contributions, this work 1) offers an example of NIME design for situated living, feeling, performing bodies, and 2) presents the rich potential of soma design as a path for designing in this context.Comment: International Conference on New Interfaces for Musical Expression (NIME 2021

Exploring Awareness of Breathing through Deep Touch Pressure

Deep Pressure Therapy relies on exerting firm touch to help individuals with sensory sensitivity. We performed first-person explorations of deep pressure enabled by shape-changing actuation driven by breathing sensing. This revealed a novel design space with rich, evocative, aesthetically interesting interactions that can help increase breathing awareness and appreciation through: (1) applying symmetrical as well as asymmetrical pressure on the torso; (2) using pressure to direct attention to muscles or bone structure involved in different breathing patterns; (3) apply synchronous as well as asynchronous feedback following or opposing the user’s breathing rhythm through applying rhythmic pressure. Taken together these explorations led us to design (4) breathing correspondence interactions – a balance point right between leading and following users’ breathing patterns by first applying deep pressure – almost to the point of being unpleasant – and then releasing in rhythmic flow.QC 20210521</p

Tangible interaction for supporting well-being

Our workshop aims to bring together researchers and practitioners across disciplines in HCI who share an interest in promoting well-being through tangible interaction. The workshop forms an impassioned response to the worldwide push towards more digital and remote interaction in nearly all domains of our lives in the context of the COVID-19 pandemic. One question we raise is: to what extent will measures like remote interaction remain in place post-pandemic, and to what extent these changes may influence future agendas for the design of interactive products and services to support living well? We aim to ensure that the workshop serves as a space for diverse participants to share ideas and engage in cooperative discussions through hands-on activities resulting in the co-creation of a Manifesto to demonstrate the importance of embodied and sensory interaction for supporting well-being in a post-pandemic context. All the workshop materials will be published online on the workshop website and disseminated through ongoing collaboration

HCI and Affective Health : Taking stock of a decade of studies and charting future research directions

QC 20190904AffecTec

Soma Bits - Mediating Technology to Orchestrate Bodily Experiences

The Soma Bits are a prototyping toolkit that facilitates Soma Design. Acting as an accessible ‘sociodigital material’ Soma Bits allow designers to pair digital technologies, with their whole body and senses, as part of an iterative soma design process.The Soma Bits addresses the difficulty we experienced in past Soma Design processes — that articulating ofsensations we want to evoke to others, and thenmaintaining these experiences in memory throughout a design process. Thus, the Soma Bits enable designers to know and experience what a designmight ‘feel like’ and to share that with others. The Soma Bits relate to three experiential qualities:‘feeling connected’, ‘feeling embraced’, and ‘being in correspondence’ with the interactive materials. The Soma Bits have a form factor and materiality thatallow actuators (heat, vibration, and shape-changing) to be placed on and around the body; they are easily configurable to enable quick and controllable creations of soma experiences which can be both part of a first-person approach as well as shared withothers. The Soma Bits are a living, growing library ofshapes and actuators. We use them in our own designpractices, as well as when engaging others in soma design processes.QC 20190619Affective Health, Innova- tive Training Network funded by the H2020 People Programme under Marie Skłodowska-Curie grant agreement No 72202