Exploration of mid-air haptics experience design

Ultrasonic Mid-air Haptics (UMH) is a novel technology that uses the mechanical properties of sound waves to create a pressure point in mid-air. This pressure point, called focal point, can slightly bend the skin and be felt in mid-air without any attachment to the body. This thesis focuses on both studying how to integrate this technology with other senses (i.e. vision and audition) and exploring the range of tactile sensations it can provide. The first two projects presented in this document present the integration of ultrasonic mid-air haptics with audio-visual content. The first project describes the process of creating a unique haptic experience that was part of a six-weeks multisensory exhibition in a museum. The second project moved from the museum to a controlled environment and explored the creation of haptic experiences based on physiologic measurements for six short films. Both studies showed the positive value of adding ultrasonic mid-air haptics to traditional media through higher reported arousal and participants’ high enthusiasm for multisensory content. In the two latter projects of this thesis, it was explored how we could extend the range of possible tactile sensations provided by UMHs. We introduced a new technique called Spatio-Temporal Modulation (STM). It enabled the creation of brand-new tactile experiences, including more salient shapes and wider range of textures. We also provided some guidelines on how to control some of the tactile properties of the sensation, including strength,roughness,or regularity. The findings of those four projects contribute to the growing body of knowledge of UMHs. A summary of the key contributions is provided at the end of the thesis as well as several leads for future works

Not just seeing, but also feeling art: mid-air haptic experiences integrated in a multisensory art exhibition

The use of the senses of vision and audition as interactive means has dominated the field of Human-Computer Interaction (HCI) for decades, even though nature has provided us with many more senses for perceiving and interacting with the world around us. That said, it has become attractive for {HCI} researchers and designers to harness touch, taste, and smell in interactive tasks and experience design. In this paper, we present research and design insights gained throughout an interdisciplinary collaboration on a six-week multisensory display – Tate Sensorium – exhibited at the Tate Britain art gallery in London, UK. This is a unique and first time case study on how to design art experiences whilst considering all the senses (i.e., vision, sound, touch, smell, and taste), in particular touch, which we exploited by capitalizing on a novel haptic technology, namely, mid-air haptics. We first describe the overall set up of Tate Sensorium and then move on to describing in detail the design process of the mid-air haptic feedback and its integration with sound for the Full Stop painting by John Latham (1961). This was the first time that mid-air haptic technology was used in a museum context over a prolonged period of time and integrated with sound to enhance the experience of visual art. As part of an interdisciplinary team of curators, sensory designers, sound artists, we selected a total of three variations of the mid-air haptic experience (i.e., haptic patterns), which were alternated at dedicated times throughout the six-week exhibition. We collected questionnaire-based feedback from 2500 visitors and conducted 50 interviews to gain quantitative and qualitative insights on visitors’ experiences and emotional reactions. Whilst the questionnaire results are generally very positive with only a small variation of the visitors’ arousal ratings across the three tactile experiences designed for the Full Stop painting, the interview data shed light on the differences in the visitors’ subjective experiences. Our findings suggest multisensory designers and art curators can ensure a balance between surprising experiences versus the possibility of free exploration for visitors. In addition, participants expressed that experiencing art with the combination of mid-air haptic and sound was immersive and provided an up-lifting experience of touching without touch. We are convinced that the insights gained from this large-scale and real-world field exploration of multisensory experience design exploiting a new and emerging technology provide a solid starting point for the {HCI} community, creative industries, and art curators to think beyond conventional art experiences. Specifically, our work demonstrates how novel mid-air technology can make art more emotionally engaging and stimulating, especially abstract art that is often open to interpretation

Multi-Sensory Interaction for Blind and Visually Impaired People

This book conveyed the visual elements of artwork to the visually impaired through various sensory elements to open a new perspective for appreciating visual artwork. In addition, the technique of expressing a color code by integrating patterns, temperatures, scents, music, and vibrations was explored, and future research topics were presented. A holistic experience using multi-sensory interaction acquired by people with visual impairment was provided to convey the meaning and contents of the work through rich multi-sensory appreciation. A method that allows people with visual impairments to engage in artwork using a variety of senses, including touch, temperature, tactile pattern, and sound, helps them to appreciate artwork at a deeper level than can be achieved with hearing or touch alone. The development of such art appreciation aids for the visually impaired will ultimately improve their cultural enjoyment and strengthen their access to culture and the arts. The development of this new concept aids ultimately expands opportunities for the non-visually impaired as well as the visually impaired to enjoy works of art and breaks down the boundaries between the disabled and the non-disabled in the field of culture and arts through continuous efforts to enhance accessibility. In addition, the developed multi-sensory expression and delivery tool can be used as an educational tool to increase product and artwork accessibility and usability through multi-modal interaction. Training the multi-sensory experiences introduced in this book may lead to more vivid visual imageries or seeing with the mind’s eye

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Haptic Media Scenes

The aim of this thesis is to apply new media phenomenological and enactive embodied cognition approaches to explain the role of haptic sensitivity and communication in personal computer environments for productivity. Prior theory has given little attention to the role of haptic senses in influencing cognitive processes, and do not frame the richness of haptic communication in interaction design—as haptic interactivity in HCI has historically tended to be designed and analyzed from a perspective on communication as transmissions, sending and receiving haptic signals. The haptic sense may not only mediate contact confirmation and affirmation, but also rich semiotic and affective messages—yet this is a strong contrast between this inherent ability of haptic perception, and current day support for such haptic communication interfaces. I therefore ask: How do the haptic senses (touch and proprioception) impact our cognitive faculty when mediated through digital and sensor technologies? How may these insights be employed in interface design to facilitate rich haptic communication? To answer these questions, I use theoretical close readings that embrace two research fields, new media phenomenology and enactive embodied cognition. The theoretical discussion is supported by neuroscientific evidence, and tested empirically through case studies centered on digital art. I use these insights to develop the concept of the haptic figura, an analytical tool to frame the communicative qualities of haptic media. The concept gauges rich machine- mediated haptic interactivity and communication in systems with a material solution supporting active haptic perception, and the mediation of semiotic and affective messages that are understood and felt. As such the concept may function as a design tool for developers, but also for media critics evaluating haptic media. The tool is used to frame a discussion on opportunities and shortcomings of haptic interfaces for productivity, differentiating between media systems for the hand and the full body. The significance of this investigation is demonstrating that haptic communication is an underutilized element in personal computer environments for productivity and providing an analytical framework for a more nuanced understanding of haptic communication as enabling the mediation of a range of semiotic and affective messages, beyond notification and confirmation interactivity

Is Multimedia Multisensorial? - A Review of Mulsemedia Systems

© 2018 Copyright held by the owner/author(s). Mulsemedia - multiple sensorial media - makes possible the inclusion of layered sensory stimulation and interaction through multiple sensory channels. e recent upsurge in technology and wearables provides mulsemedia researchers a vehicle for potentially boundless choice. However, in order to build systems that integrate various senses, there are still some issues that need to be addressed. is review deals with mulsemedia topics remained insu ciently explored by previous work, with a focus on multi-multi (multiple media - multiple senses) perspective, where multiple types of media engage multiple senses. Moreover, it addresses the evolution of previously identi ed challenges in this area and formulates new exploration directions.This article was funded by the European Union’s Horizon 2020 Research and Innovation program under Grant Agreement no. 688503

The Impact of Haptic Imagery in Art Supplies: Evaluation and Need for Touch

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Driven Marketing, specialization in Digital Marketing and AnalyticsWe live in a fast-paced world, where information is instantaneously consumed through the speed of the internet, and where brands need to stand out from the others to get their customer’s attention, which incentives the constant evolution of technologies and marketing strategies. Sensory Marketing has proven to be an effective way of capturing the consumers’ senses, and to provide unexpected shopping experiences that attract the attention of new consumers. The exploration of sensorial stimulation has been increasingly used in physical stores, but has been quite left behind when it comes to the online sales environments, especially if we focus on the touch sense. However, it is possible to stimulate the tactile sensation through images, videos and even text descriptions. It is called Haptic Imagery, and it triggers the memories of previous touch experiences, by stimulating the brain to tactile sense, and which can be used to overcome the lack of physical stimulation in digital stores. The aim of this study is to investigate how digital stores can take advantage of Haptic Imagery, to influence the buyer’s product judgement and purchase intention. Moreover, the present research uses the Need for Touch scale (NFT), developed by Peck and Childers (2003), to understand the individual differences between participants’ necessity for touching a product before buying it, and to study its relation to the artists’ willingness to buy their art supplies online. For this purpose, a survey was constructed and distributed among art students from two Art Universities, who are believed to be the right target consumers for art supplies. The main findings of the present study contradict the overarching literature, since the results in relation to product quality perception and purchase intention were not different between the group that saw Haptic Imagery and the one that saw Non-Haptic Imagery. Nevertheless, it was possible to verify that individuals with high NFT felt more frustration for not being able to touch the product, and therefore were less probable to buy it in online stores than individuals with lower NFT values, which confirms the previous findings on the subject and validates the use of Peck and Childers’ NFT scale