16 research outputs found
Human bodies as interaction materials for somatic, social, and multisensory Virtual Reality experiences
Recently, the field of Human-Computer Interaction has begun to embrace the crucial role of our bodies across our cognitive and
social processes. The movement of embodied interaction has been followed by a somatic and material turn that strives to design
technology for richer multi-sensory somatic experiences. Despite the booming developments in tangible and wearable technologies,
there is strikingly limited research exploring the potential of our own physical and virtual bodies to become the material for the design
of interaction, in the same ways that tangible materials are. In this position paper, we propose a series of approaches for designing
technology for social interaction, that integrate human bodies as interaction materials to elicit a multisensory embodied experience.
We illustrate this approach with examples of several virtual reality experiences that we have designed for supporting social connection
Design and Somatic Sensibilities
Cultivating somatic sensibilities involves heightening our sensory appreciation as a path to devise meaningful multisensory experiences in interaction design. Immersed in an increasingly digitalized and data-centric world, research projects centering on the sensory, embodied, and material reality of our experience, might start losing momentum. On the other hand, the pandemic has also transferred our labs from the public nature of our workshop room to the intimacy of our homes, bringing unexpected benefits and challenges. The articles included in this issue center on the sensory and material experiences of designers, who work with design methods that foreground somatic modes of inquiry and show how these changes could open a door for new opportunities to emphasize the importance of embodied and somatic practices in design and HCI
Biosensing and Actuation—Platforms Coupling Body Input-Output Modalities for Affective Technologies
Research in the use of ubiquitous technologies, tracking systems and wearables within
mental health domains is on the rise. In recent years, affective technologies have gained
traction and garnered the interest of interdisciplinary fields as the research on such technologies
matured. However, while the role of movement and bodily experience to affective experience is
well-established, how to best address movement and engagement beyond measuring cues and signals
in technology-driven interactions has been unclear. In a joint industry-academia effort, we aim to
remodel how affective technologies can help address body and emotional self-awareness. We present
an overview of biosignals that have become standard in low-cost physiological monitoring and show
how these can be matched with methods and engagements used by interaction designers skilled in
designing for bodily engagement and aesthetic experiences. Taking both strands of work together offers
unprecedented design opportunities that inspire further research. Through first-person soma design,
an approach that draws upon the designer’s felt experience and puts the sentient body at the forefront,
we outline a comprehensive work for the creation of novel interactions in the form of couplings that
combine biosensing and body feedback modalities of relevance to affective health. These couplings lie
within the creation of design toolkits that have the potential to render rich embodied interactions to
the designer/user. As a result we introduce the concept of “orchestration”. By orchestration, we refer
to the design of the overall interaction: coupling sensors to actuation of relevance to the affective
experience; initiating and closing the interaction; habituating; helping improve on the users’ body
awareness and engagement with emotional experiences; soothing, calming, or energising, depending
on the affective health condition and the intentions of the designer. Through the creation of a
range of prototypes and couplings we elicited requirements on broader orchestration mechanisms.
First-person soma design lets researchers look afresh at biosignals that, when experienced through
the body, are called to reshape affective technologies with novel ways to interpret biodata, feel it,
understand it and reflect upon our bodies
Unpacking Non-Dualistic Design: The Soma Design Case
We report on a somaesthetic design workshop and the subsequent analytical work aiming to demystify what is entailed in a non-dualistic design stance on embodied interaction and why a first-person engagement is crucial to its unfoldings. However, as we will uncover through a detailed account of our process, these first-person engagements are deeply entangled with second- and third-person perspectives, sometimes even overlapping. The analysis furthermore reveals some strategies for bridging the body-mind divide by attending to our inner universe and dissolving or traversing dichotomies between inside and outside; individual and social; body and technology. By detailing the creative process, we show how soma design becomes a process of designing with and through kinesthetic experience, in turn letting us confront several dualisms that run like fault lines through HCI's engagement with embodied interaction
Interoceptive Interaction:An Embodied Metaphor Inspired Approach to Designing for Meditation
Meditation is a mind-body practice with considerable wellbeing benefits which can take different forms. Novices usually start with focused attention meditation that supports regulation of attention towards an inward focus or internal bodily sensations and away from external stimuli or distractors. Most meditation technologies employ metaphorical mappings of meditative states to visual or soundscape representations to support awareness of mind wandering and attention regulation, although the rationale for such mappings is seldom articulated. Moreover, such external modalities also take the focus attention away from the body. We advance the concept of interoceptive interaction and employed the embodied metaphor theory to explore the design of mappings to the interoceptive sense of thermoception. We illustrate this concept with WarmMind, an on-body interface integrating heat actuators for mapping meditation states. We report on an exploratory study with 10 participants comparing our novel thermal metaphors for mapping meditation states with comparable ones, albeit in aural modality, as provided by Muse meditation app. Findings indicate a tension between the highly discoverable soundscape’s metaphors which however hinder attention regulation, and the ambiguous thermal metaphors experienced as coming from the body, and supported attention regulation. We discuss the qualities of embodied metaphors underpinning this tension and propose an initial framework to inform the design of metaphorical mappings for meditation technologies
Interdisciplinary Insights for Digital Touch Communication
Communication is increasingly moving beyond ‘ways of seeing’ to ‘ways of feeling’. This Open Access book provides social design insights and implications for HCI research and design exploring digitally mediated touch communication. It offers a socially orientated map to help navigate the complex social landscape of digitally mediated touch for communication: from everyday touch-screens, tangibles, wearables, haptics for virtual reality, to the tactile internet of skin. Drawing on literature reviews, new case-study vignettes, and exemplars of digital touch, the book examines the major social debates provoked by digital touch, and investigates social themes central to the communicative potential and societal consequences of digital touch: · Communication environments, capacities and practices · Norms associations and expectations · Presence, absence and connection · Social imaginaries of digital touch · Digital touch ethics and values The book concludes with a discussion of the significance of social understanding and methods in the context of Interdisciplinary collaborations to explore touch, towards the design of digital touch communication, ‘ways of feeling’, that are useable, appropriate, ethical and socially aware
NON-VERBAL COMMUNICATION WITH PHYSIOLOGICAL SENSORS. THE AESTHETIC DOMAIN OF WEARABLES AND NEURAL NETWORKS
Historically, communication implies the transfer of information between bodies, yet this
phenomenon is constantly adapting to new technological and cultural standards. In a
digital context, it’s commonplace to envision systems that revolve around verbal modalities.
However, behavioural analysis grounded in psychology research calls attention to
the emotional information disclosed by non-verbal social cues, in particular, actions that
are involuntary. This notion has circulated heavily into various interdisciplinary computing
research fields, from which multiple studies have arisen, correlating non-verbal
activity to socio-affective inferences. These are often derived from some form of motion
capture and other wearable sensors, measuring the ‘invisible’ bioelectrical changes that
occur from inside the body.
This thesis proposes a motivation and methodology for using physiological sensory
data as an expressive resource for technology-mediated interactions. Initialised from a
thorough discussion on state-of-the-art technologies and established design principles
regarding this topic, then applied to a novel approach alongside a selection of practice
works to compliment this. We advocate for aesthetic experience, experimenting with
abstract representations. Atypically from prevailing Affective Computing systems, the
intention is not to infer or classify emotion but rather to create new opportunities for rich
gestural exchange, unconfined to the verbal domain.
Given the preliminary proposition of non-representation, we justify a correspondence
with modern Machine Learning and multimedia interaction strategies, applying an iterative,
human-centred approach to improve personalisation without the compromising
emotional potential of bodily gesture. Where related studies in the past have successfully
provoked strong design concepts through innovative fabrications, these are typically limited
to simple linear, one-to-one mappings and often neglect multi-user environments;
we foresee a vast potential. In our use cases, we adopt neural network architectures to
generate highly granular biofeedback from low-dimensional input data.
We present the following proof-of-concepts: Breathing Correspondence, a wearable
biofeedback system inspired by Somaesthetic design principles; Latent Steps, a real-time auto-encoder to represent bodily experiences from sensor data, designed for dance performance;
and Anti-Social Distancing Ensemble, an installation for public space interventions,
analysing physical distance to generate a collective soundscape. Key findings are
extracted from the individual reports to formulate an extensive technical and theoretical
framework around this topic. The projects first aim to embrace some alternative perspectives
already established within Affective Computing research. From here, these concepts
evolve deeper, bridging theories from contemporary creative and technical practices with
the advancement of biomedical technologies.Historicamente, os processos de comunicação implicam a transferência de informação
entre organismos, mas este fenómeno está constantemente a adaptar-se a novos padrões
tecnológicos e culturais. Num contexto digital, é comum encontrar sistemas que giram
em torno de modalidades verbais. Contudo, a análise comportamental fundamentada
na investigação psicológica chama a atenção para a informação emocional revelada por
sinais sociais não verbais, em particular, acções que são involuntárias. Esta noção circulou
fortemente em vários campos interdisciplinares de investigação na área das ciências da
computação, dos quais surgiram múltiplos estudos, correlacionando a actividade nãoverbal
com inferências sócio-afectivas. Estes são frequentemente derivados de alguma
forma de captura de movimento e sensores “wearable”, medindo as alterações bioeléctricas
“invisíveis” que ocorrem no interior do corpo.
Nesta tese, propomos uma motivação e metodologia para a utilização de dados sensoriais
fisiológicos como um recurso expressivo para interacções mediadas pela tecnologia.
Iniciada a partir de uma discussão aprofundada sobre tecnologias de ponta e princípios
de concepção estabelecidos relativamente a este tópico, depois aplicada a uma nova abordagem,
juntamente com uma selecção de trabalhos práticos, para complementar esta.
Defendemos a experiência estética, experimentando com representações abstractas. Contrariamente
aos sistemas de Computação Afectiva predominantes, a intenção não é inferir
ou classificar a emoção, mas sim criar novas oportunidades para uma rica troca gestual,
não confinada ao domínio verbal.
Dada a proposta preliminar de não representação, justificamos uma correspondência
com estratégias modernas de Machine Learning e interacção multimédia, aplicando uma
abordagem iterativa e centrada no ser humano para melhorar a personalização sem o
potencial emocional comprometedor do gesto corporal. Nos casos em que estudos anteriores
demonstraram com sucesso conceitos de design fortes através de fabricações
inovadoras, estes limitam-se tipicamente a simples mapeamentos lineares, um-para-um,
e muitas vezes negligenciam ambientes multi-utilizadores; com este trabalho, prevemos
um potencial alargado. Nos nossos casos de utilização, adoptamos arquitecturas de redes
neurais para gerar biofeedback altamente granular a partir de dados de entrada de baixa dimensão.
Apresentamos as seguintes provas de conceitos: Breathing Correspondence, um sistema
de biofeedback wearable inspirado nos princípios de design somaestético; Latent
Steps, um modelo autoencoder em tempo real para representar experiências corporais
a partir de dados de sensores, concebido para desempenho de dança; e Anti-Social Distancing
Ensemble, uma instalação para intervenções no espaço público, analisando a
distância física para gerar uma paisagem sonora colectiva. Os principais resultados são
extraídos dos relatórios individuais, para formular um quadro técnico e teórico alargado
para expandir sobre este tópico. Os projectos têm como primeiro objectivo abraçar algumas
perspectivas alternativas às que já estão estabelecidas no âmbito da investigação
da Computação Afectiva. A partir daqui, estes conceitos evoluem mais profundamente,
fazendo a ponte entre as teorias das práticas criativas e técnicas contemporâneas com o
avanço das tecnologias biomédicas
Immersive Participation:Futuring, Training Simulation and Dance and Virtual Reality
Dance knowledge can inform the development of scenario design in immersive digital simulation environments by strengthening a participant’s capacity to learn through the body. This study engages with processes of participatory practice that question how the transmission and transfer of dance knowledge/embodied knowledge in immersive digital environments is activated and applied in new contexts. These questions are relevant in both arts and industry and have the potential to add value and knowledge through crossdisciplinary collaboration and exchange. This thesis consists of three different research projects all focused on observation, participation, and interviews with experts on embodiment in digital simulation. The projects were chosen to provide a range of perspectives across dance, industry and futures studies. Theories of embodied cognition, in particular the notions of the extended body, distributed cognition, enactment and mindfulness, offer critical lenses through which to explore the relationship of embodied integration and participation within immersive digital environments. These areas of inquiry lead to the consideration of how language from the field of computer science can assist in describing somatic experience in digital worlds through a discussion of the emerging concepts of mindfulness, wayfinding, guided movement and digital kinship. These terms serve as an example of how the mutability of language became part of the process as terms applied in disparate disciplines were understood within varying contexts. The analytic tools focus on applying a posthuman view, speculation through a futures ethnography, and a cognitive ethnographical approach to my research project. These approaches allowed me to examine an ecology of practices in order to identify methods and processes that can facilitate the transmission and transfer of embodied knowledge within a community of practice. The ecological components include dance, healthcare, transport, education and human/computer interaction. These fields drove the data collection from a range of sources including academic papers, texts, specialists’ reports, scientific papers, interviews and conversations with experts and artists.The aim of my research is to contribute both a theoretical and a speculative understanding of processes, as well as tools applicable in the transmission of embodied knowledge in virtual dance and arts environments as well as digital simulation across industry. Processes were understood theoretically through established studies in embodied cognition applied to workbased training, reinterpreted through my own movement study. Futures methodologies paved the way for speculative processes and analysis. Tools to choreograph scenario design in immersive digital environments were identified through the recognition of cross purpose language such as mindfulness, wayfinding, guided movement and digital kinship. Put together, the major contribution of this research is a greater understanding of the value of dance knowledge applied to simulation developed through theoretical and transformational processes and creative tools