Designing a notation for the senses

Understanding the role of the non-visual senses is difficult, as there is at present no recording medium for the olfactory, gustatory, tactile or even aural environment which is useful to the practice of urban design. In any case, recording has a different aim from drawing and notation

Haptic Media Scenes

The aim of this thesis is to apply new media phenomenological and enactive embodied cognition approaches to explain the role of haptic sensitivity and communication in personal computer environments for productivity. Prior theory has given little attention to the role of haptic senses in influencing cognitive processes, and do not frame the richness of haptic communication in interaction design—as haptic interactivity in HCI has historically tended to be designed and analyzed from a perspective on communication as transmissions, sending and receiving haptic signals. The haptic sense may not only mediate contact confirmation and affirmation, but also rich semiotic and affective messages—yet this is a strong contrast between this inherent ability of haptic perception, and current day support for such haptic communication interfaces. I therefore ask: How do the haptic senses (touch and proprioception) impact our cognitive faculty when mediated through digital and sensor technologies? How may these insights be employed in interface design to facilitate rich haptic communication? To answer these questions, I use theoretical close readings that embrace two research fields, new media phenomenology and enactive embodied cognition. The theoretical discussion is supported by neuroscientific evidence, and tested empirically through case studies centered on digital art. I use these insights to develop the concept of the haptic figura, an analytical tool to frame the communicative qualities of haptic media. The concept gauges rich machine- mediated haptic interactivity and communication in systems with a material solution supporting active haptic perception, and the mediation of semiotic and affective messages that are understood and felt. As such the concept may function as a design tool for developers, but also for media critics evaluating haptic media. The tool is used to frame a discussion on opportunities and shortcomings of haptic interfaces for productivity, differentiating between media systems for the hand and the full body. The significance of this investigation is demonstrating that haptic communication is an underutilized element in personal computer environments for productivity and providing an analytical framework for a more nuanced understanding of haptic communication as enabling the mediation of a range of semiotic and affective messages, beyond notification and confirmation interactivity

Creative Haptic Interface Design for the Aging Population

Audiovisual human-computer-interfaces still make up the majority of content to the public; however, haptic interfaces offer unique advantage over the dominant information infrastructure, particularly for users with a disability or diminishing cognitive and physical skills like the elderly. The tactile sense allows users to integrate new, unobstructive channels for digital information into their sensorium, one that is less likely to be overwhelmed compared to vision and audition. Haptics research focus on the development of hardware, improving resolution, modality, and fidelity of the actuators. Despite the technological limitations, haptic interfaces are shown to reinforce physical skill acquisition, therapy, and communication. This chapter will present key characteristics intuitive tactile interfaces should capture for elderly end-users; sample projects will showcase unique applications and designs that identify the limitations of the UI

Sensory augmentation with distal touch: The tactile helmet project

The Tactile Helmet is designed to augment a wearer's senses with a long range sense of touch. Tactile specialist animals such as rats and mice are capable of rapidly acquiring detailed information about their environment from their whiskers by using task-sensitive strategies. Providing similar information about the nearby environment, in tactile form, to a human operator could prove invaluable for search and rescue operations, or for partially-sighted people. Two key aspects of the Tactile Helmet are sensory augmentation, and active sensing. A haptic display is used to provide the user with ultrasonic range information. This can be interpreted in addition to, rather than instead of, visual or auditory information. Active sensing systems "are purposive and information-seeking sensory systems, involving task specific control of the sensory apparatus" [1]. The integration of an accelerometer allows the device to actively gate the delivery of sensory information to the user, depending on their movement. Here we describe the hardware, sensory transduction and characterisation of the Tactile Helmet device, before outlining potential use cases and benefits of the system. © 2013 Springer-Verlag Berlin Heidelberg

Towards disappearing user interfaces for ubiquitous computing: human enhancement from sixth sense to super senses

The enhancement of human senses electronically is possible when pervasive computers interact unnoticeably with humans in Ubiquitous Computing. The design of computer user interfaces towards “disappearing” forces the interaction with humans using a content rather than a menu driven approach, thus the emerging requirement for huge number of non-technical users interfacing intuitively with billions of computers in the Internet of Things is met. Learning to use particular applications in Ubiquitous Computing is either too slow or sometimes impossible so the design of user interfaces must be naturally enough to facilitate intuitive human behaviours. Although humans from different racial, cultural and ethnic backgrounds own the same physiological sensory system, the perception to the same stimuli outside the human bodies can be different. A novel taxonomy for Disappearing User Interfaces (DUIs) to stimulate human senses and to capture human responses is proposed. Furthermore, applications of DUIs are reviewed. DUIs with sensor and data fusion to simulate the Sixth Sense is explored. Enhancement of human senses through DUIs and Context Awareness is discussed as the groundwork enabling smarter wearable devices for interfacing with human emotional memories

Head-mounted Sensory Augmentation Device: Comparing Haptic and Audio Modality

This paper investigates and compares the effectiveness of haptic and audio modality for navigation in low visibility environment using a sensory augmentation device. A second generation head-mounted vibrotactile interface as a sensory augmentation prototype was developed to help users to navigate in such environments. In our experiment, a subject navigates along a wall relying on the haptic or audio feedbacks as navigation commands. Haptic/audio feedback is presented to the subjects according to the information measured from the walls to a set of 12 ultrasound sensors placed around a helmet and a classification algorithm by using multilayer perceptron neural network. Results showed the haptic modality leads to significantly lower route deviation in navigation compared to auditory feedback. Furthermore, the NASA TLX questionnaire showed that subjects reported lower cognitive workload with haptic modality although both modalities were able to navigate the users along the wall

Haptic Reassurance in the Pitch Black for an Immersive Theatre Experience

An immersive theatre experience was designed to raise awareness and question perceptions of ‘blindness’, through enabling both sighted and blind members to experience a similar reality. A multimodal experience was created, comprising ambient sounds and narratives – heard through headphones – and an assortment of themed tactile objects, intended to be felt. In addition, audience members were each provided with a novel haptic device that was designed to enhance their discovery of a pitch-black space. An in the wild study of the cultural experience showed how blind and sighted audience members had different ‘felt’ experiences, but that neither was a lesser one. Furthermore, the haptic device was found to encourage enactive exploration and provide reassurance of the environment for both sighted and blind people, rather than acting simply as a navigation guide. We discuss the potential of using haptic feedback to create cultural experiences for both blind and sighted people; rethinking current utilitarian framing of it as assistive technology

Engineering data compendium. Human perception and performance. User's guide

The concept underlying the Engineering Data Compendium was the product of a research and development program (Integrated Perceptual Information for Designers project) aimed at facilitating the application of basic research findings in human performance to the design and military crew systems. The principal objective was to develop a workable strategy for: (1) identifying and distilling information of potential value to system design from the existing research literature, and (2) presenting this technical information in a way that would aid its accessibility, interpretability, and applicability by systems designers. The present four volumes of the Engineering Data Compendium represent the first implementation of this strategy. This is the first volume, the User's Guide, containing a description of the program and instructions for its use

Touch: an enquiry

This thesis forms part of a practice-based enquiry into ‘touch’ from a painter’s perspective. I am investigating touch as a three-way interplay: 1. the act of touch at work in the studio; 2. the sense of touch experienced with and in my body while engaging with my own work and with the paintings of Robert Ryman; and 3. the presence of touch, materially and metaphorically, in the work as a result of my reflective practice in response to an analysis of 1 and 2. I examine the making of two sections of studio work, Touch Screen and Where’s the ’Ouch?; analyse the findings of three days spent with the Ryman installation in the Hallen für Neue Kunst, Schaffhausen, Switzerland; and reflect on the making of a third section of my studio work, Blanks in response to the Hallen experience