Using sound in multi-touch interfaces to change materiality and touch behavior

Current development in multimodal interfaces allows us to interact with digitally represented objects. Sadly, these representations are often poor due to technical limitations in representing some of the sensorial properties. Here we explore the possibility of overcoming these limitations by exploiting multisensory integration processes and propose a sound-based interaction technique to alter the perceived materiality of a surface being touched and to shape users' touch behavior. The latter can be seen both as a cue of, and as a means to reinforce, the altered perception. We designed a prototype that dynamically alters the texture-related sound feedback based on touch behavior, as in natural surface touch interactions. A user study showed that the frequency of the sound feedback alters texture perception (coldness and material type) and touch behavior (velocity and pressure). We conclude by discussing lessons learnt from this work in terms of HCI applications and questions opened by this research. Copyright is held by the owner/author(s)

Body x Materials: A workshop exploring the role of material-enabled body-based multisensory experiences

Over the last 15 years, HCI and Interaction Design have experienced a “material turn” characterized by a growing interest in the materiality of technology and computation, and in methods that support exploring, envisioning, and crafting with and through materials. The community has experienced a similar turn focused on the body, on how to best design for and from a first-person, lived experience, and the moving and sensual body. In this workshop, we focus on the intersection of these two turns. The emerging developments in multimodal interfaces open opportunities to bring in materiality to the digital world as well as to transform the materiality of objects and bodies in the real-world, including the materiality of our own body. The different sensory qualities of (touchable and untouchable, physical and digital) objects and bodies, including our own, can be brought into the design of digital technologies to enrich, augment, and transform embodied experiences. In this “materials revolution” [15], what are the current theories, approaches, methods, and tools that emphasize the critical role of materiality to body-based interactions with technology? To explore this, in this workshop we will focus on five related themes: material enabling expression, material as a catalyst for human action, material enabling reflection and awareness, material enabling transformation and material supporting the design process for the re-creation of the existing and the yet-to-exist. This workshop with technology presentations, panel sessions with experts, and multidisciplinary discussions will: (i) bring together researchers who work on (re)creating sensory properties of materials through technology with those who investigate experiential effects of materials and materialenabled interactions, (ii) discuss methods, opportunities, difficulties in designing materiality and material-enabled interactions, and (iii) form a multidisciplinary community to build synergies and collaborations

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

ICS Materials

This present book covers a series of outstanding reputation researchers’ contributions on the topic of ICS Materials: a new class of emerging materials with properties and qualities concerning interactivity, connectivity and intelligence. In the general framework of ICS Materials’ domain, each chapter deals with a specific aspect following the characteristic perspective of each researcher. As result, methods, tools, guidelines emerged that are relevant and applicable to several contexts such as product, interaction design, materials science and many more

Haptic Media Scenes

The aim of this thesis is to apply new media phenomenological and enactive embodied cognition approaches to explain the role of haptic sensitivity and communication in personal computer environments for productivity. Prior theory has given little attention to the role of haptic senses in influencing cognitive processes, and do not frame the richness of haptic communication in interaction design—as haptic interactivity in HCI has historically tended to be designed and analyzed from a perspective on communication as transmissions, sending and receiving haptic signals. The haptic sense may not only mediate contact confirmation and affirmation, but also rich semiotic and affective messages—yet this is a strong contrast between this inherent ability of haptic perception, and current day support for such haptic communication interfaces. I therefore ask: How do the haptic senses (touch and proprioception) impact our cognitive faculty when mediated through digital and sensor technologies? How may these insights be employed in interface design to facilitate rich haptic communication? To answer these questions, I use theoretical close readings that embrace two research fields, new media phenomenology and enactive embodied cognition. The theoretical discussion is supported by neuroscientific evidence, and tested empirically through case studies centered on digital art. I use these insights to develop the concept of the haptic figura, an analytical tool to frame the communicative qualities of haptic media. The concept gauges rich machine- mediated haptic interactivity and communication in systems with a material solution supporting active haptic perception, and the mediation of semiotic and affective messages that are understood and felt. As such the concept may function as a design tool for developers, but also for media critics evaluating haptic media. The tool is used to frame a discussion on opportunities and shortcomings of haptic interfaces for productivity, differentiating between media systems for the hand and the full body. The significance of this investigation is demonstrating that haptic communication is an underutilized element in personal computer environments for productivity and providing an analytical framework for a more nuanced understanding of haptic communication as enabling the mediation of a range of semiotic and affective messages, beyond notification and confirmation interactivity

ICS Materials

This present book covers a series of outstanding reputation researchers’ contributions on the topic of ICS Materials: a new class of emerging materials with properties and qualities concerning interactivity, connectivity and intelligence. In the general framework of ICS Materials’ domain, each chapter deals with a specific aspect following the characteristic perspective of each researcher. As result, methods, tools, guidelines emerged that are relevant and applicable to several contexts such as product, interaction design, materials science and many more

Altering one's body-perception through e-textiles and haptic metaphors

Tajadura-Jiménez A, Väljamäe A and Kuusk K (2020) Altering One's Body-Perception Through E-Textiles and Haptic Metaphors. Front. Robot. AI 7:7.Technologies change rapidly our perception of reality, moving from augmented to virtual to magical. While e-textiles are a key component in exergame or space suits, the transformative potential of the internal side of garments to create embodied experiences still remains largely unexplored. This paper is the result from an art-science collaborative project that combines recent neuroscience findings, body-centered design principles and 2D vibrotactile array-based fabrics to alter one's body perception. We describe an iterative design process intertwined with two user studies on the effects on body-perceptions and emotional responses of various vibration patterns within textile that were designed as spatial haptic metaphors. Our results show potential in considering materials (e.g., rocks) as sensations to design for body perceptions (e.g., being heavy, strong) and emotional responses. We discuss these results in terms of sensory effects on body perception and synergetic impact to research on embodiment in virtual environments, human-computer interaction, and e-textile design. The work brings a new perspective to the sensorial design of embodied experiences which is based on "material perception" and haptic metaphors, and highlights potential opportunities opened by haptic clothing to change body-perception.This work was partially supported by PSI2016-79004-R Magic Shoes project grant (AEI/FEDER, UE), from Ministerio de Economía, Industria y Competitividad of Spain and the Magic Lining VERTIGO project as part of the STARTS program of the European Commission, based on technological elements from the project Magic Shoes. AT-J was supported by RYC- 2014–15421 grant from the Ministerio de Economía, Industria y Competitividad of Spain and AV was supported by the Estonian Research Council grant PUT1518