Sensory augmentation and the tactile sublime

This paper responds to recent developments in the field of sensory augmentation by analysing several technological devices that augment the sensory apparatus using the tactile sense. First, I will define the term sensory augmentation, as the use of technological modification to enhance the sensory apparatus, and elaborate on the preconditions for successful tactile sensory augmentation. These are the adaptability of the brain to unfamiliar sensory input and the specific qualities of the skin lending themselves to be used for the perception of additional sensory information. Two devices, Moon Ribas’ Seismic Sense and David Eagleman’s vest, will then be discussed as potential facilitators of aesthetic experiences in virtue of the tactile sensory augmentation that these devices allow. I will connect the experiences afforded by these devices to the Kantian categories of the mathematical and the dynamical sublime, and to existing accounts of tactile sublimity. Essentially, the objects these devices make sensible, earthquakes for the Seismic Sense and digital information for the vest, produce pleasurable feelings of potential danger, awe, and respect. The subsequent acclimation to this new way of sensing and the aim to comprehend its sensed object are then discussed as possible objections to the interpretation of these experiences as sublime, and as aesthetic in general. To exemplify these issues and concretise my thesis of tactile sensory augmentation as a trigger of the sublime, I will outline an experiment to use the vest as an aid for faster decision making on the stock market

Sensory augmentation through tissue conduction

One hundred volunteers have undergone short (5 min) listening tests in a novel multi-transducer bone-and-tissue conduction apparatus for spatial audio. The subjects subsequently described their experiences in an unstructured qualitative elicitation exercise. Their responses were aggregated to identify key themes and differences. Emergent themes are: enjoyable, informative, spatial and strange. Tactile supplementation of spatial audio display was noted in a positive light. We note that some spatial attributes are more perceptible than others. The implications for perceptual augmentation are discussed, particularly in relation to conductive hearing deficits. We conclude that the technique has potential for development and discusses future research directions.N/

TESSA - Toolkit for experimentation with multimodal sensory substitution and augmentation

TESSA is a toolkit for experimenting with sensory augmentation. It includes hardware and software to facilitate rapid prototyping of interfaces that can enhance one sense using information gathered from another sense. The toolkit contains a range of sensors (e.g. ultrasonics, temperature sensors) and actuators (e.g. tactors or stereo sound), designed modularly so that inputs and outputs can be easily swapped in and out and customized using TESSA’s graphical user interface (GUI), with “real time” feedback. The system runs on a Raspberry Pi with a built-in touchscreen, providing a compact and portable form that is amenable for field trials. At CHI Interactivity, the audience will have the opportunity to experience sensory augmentation effects using this system, and design their own sensory augmentation interfaces

Sensory Augmentation for the Blind

Common navigational aids used by blind travelers during large-scale navigation divert attention away from important cues of the immediate environment (i.e., approaching vehicles). Sensory augmentation devices, relying on principles similar to those at work in sensory substitution, can potentially bypass the bottleneck of attention through sub-cognitive implementation of a set of rules coupling motor actions with sensory stimulation. We provide a late blind subject with a vibrotactile belt that continually signals the direction of magnetic north. The subject completed a set of behavioral tests before and after an extended training period. The tests were complemented by questionnaires and interviews. This newly supplied information improved performance on different time scales. In a pointing task we demonstrate an instant improvement of performance based on the signal provided by the device. Furthermore, the signal was helpful in relevant daily tasks, often complicated for the blind, such as keeping a direction over longer distances or taking shortcuts in familiar environments. A homing task with an additional attentional load demonstrated a significant improvement after training. The subject found the directional information highly expedient for the adjustment of his inner maps of familiar environments and describes an increase in his feeling of security when exploring unfamiliar environments with the belt. The results give evidence for a firm integration of the newly supplied signals into the behavior of this late blind subject with better navigational performance and more courageous behavior in unfamiliar environments. Most importantly, the complementary information provided by the belt lead to a positive emotional impact with enhanced feeling of security. The present experimental approach demonstrates the positive potential of sensory augmentation devices for the help of handicapped people

Low-fi skin vision: A case study in rapid prototyping a sensory substitution system

We describe the design process we have used to develop a minimal, twenty vibration motor Tactile Vision Sensory Substitution (TVSS) system which enables blind-folded subjects to successfully track and bat a rolling ball and thereby experience 'skin vision'. We have employed a low-fi rapid prototyping approach to build this system and argue that this methodology is particularly effective for building embedded interactive systems. We support this argument in two ways. First, by drawing on theoretical insights from robotics, a discipline that also has to deal with the challenge of building complex embedded systems that interact with their environments; second, by using the development of our TVSS as a case study: describing the series of prototypes that led to our successful design and highlighting what we learnt at each stage

The Feeling of Color: A Haptic Feedback Device for the Visually Disabled

Tapson J, Gurari N, Diaz J, et al. The Feeling of Color: A Haptic Feedback Device for the Visually Disabled. Presented at the Biomedical Circuits and Systems Conference (BIOCAS), Baltimore, MD.We describe a sensory augmentation system designed to provide the visually disabled with a sense of color. Our system consists of a glove with short-range optical color sensors mounted on its fingertips, and a torso-worn belt on which tactors (haptic feedback actuators) are mounted. Each fingertip sensor detects the observed objectpsilas color. This information is encoded to the tactor through vibrations in respective locations and varying modulations. Early results suggest that detection of primary colors is possible with near 100% accuracy and moderate latency, with a minimum amount of training

Tactile Language for a Head-Mounted Sensory Augmentation Device

Sensory augmentation is one of the most exciting domains for research in human-machine biohybridicity. The current paper presents the design of a 2nd generation vibrotactile helmet as a sensory augmentation prototype that is being developed to help users to navigate in low visibility environments. The paper outlines a study in which the user navigates along a virtual wall whilst the position and orientation of the user’s head is tracked by a motion capture system. Vibrotactile feedback is presented according to the user’s distance from the virtual wall and their head orientation. The research builds on our previous work by developing a simplified “tactile language” for communicating navigation commands. A key goal is to identify language tokens suitable to a head-mounted tactile interface that are maximally informative, minimize information overload, intuitive, and that have the potential to become ‘experientially transparent

Head-mounted Sensory Augmentation Device: Comparing Haptic and Audio Modality

This paper investigates and compares the effectiveness of haptic and audio modality for navigation in low visibility environment using a sensory augmentation device. A second generation head-mounted vibrotactile interface as a sensory augmentation prototype was developed to help users to navigate in such environments. In our experiment, a subject navigates along a wall relying on the haptic or audio feedbacks as navigation commands. Haptic/audio feedback is presented to the subjects according to the information measured from the walls to a set of 12 ultrasound sensors placed around a helmet and a classification algorithm by using multilayer perceptron neural network. Results showed the haptic modality leads to significantly lower route deviation in navigation compared to auditory feedback. Furthermore, the NASA TLX questionnaire showed that subjects reported lower cognitive workload with haptic modality although both modalities were able to navigate the users along the wall

Potential Mechanisms of Sensory Augmentation Systems on Human Balance Control

Numerous studies have demonstrated the real-time use of visual, vibrotactile, auditory, and multimodal sensory augmentation technologies for reducing postural sway during static tasks and improving balance during dynamic tasks. The mechanism by which sensory augmentation information is processed and used by the CNS is not well understood. The dominant hypothesis, which has not been supported by rigorous experimental evidence, posits that observed reductions in postural sway are due to sensory reweighting: feedback of body motion provides the CNS with a correlate to the inputs from its intact sensory channels (e.g., vision, proprioception), so individuals receiving sensory augmentation learn to increasingly depend on these intact systems. Other possible mechanisms for observed postural sway reductions include: cognition (processing of sensory augmentation information is solely cognitive with no selective adjustment of sensory weights by the CNS), “sixth” sense (CNS interprets sensory augmentation information as a new and distinct sensory channel), context-specific adaptation (new sensorimotor program is developed through repeated interaction with the device and accessible only when the device is used), and combined volitional and non-volitional responses. This critical review summarizes the reported sensory augmentation findings spanning postural control models, clinical rehabilitation, laboratory-based real-time usage, and neuroimaging to critically evaluate each of the aforementioned mechanistic theories. Cognition and sensory re-weighting are identified as two mechanisms supported by the existing literature

Sensory augmentation with distal touch: The tactile helmet project

The Tactile Helmet is designed to augment a wearer's senses with a long range sense of touch. Tactile specialist animals such as rats and mice are capable of rapidly acquiring detailed information about their environment from their whiskers by using task-sensitive strategies. Providing similar information about the nearby environment, in tactile form, to a human operator could prove invaluable for search and rescue operations, or for partially-sighted people. Two key aspects of the Tactile Helmet are sensory augmentation, and active sensing. A haptic display is used to provide the user with ultrasonic range information. This can be interpreted in addition to, rather than instead of, visual or auditory information. Active sensing systems "are purposive and information-seeking sensory systems, involving task specific control of the sensory apparatus" [1]. The integration of an accelerometer allows the device to actively gate the delivery of sensory information to the user, depending on their movement. Here we describe the hardware, sensory transduction and characterisation of the Tactile Helmet device, before outlining potential use cases and benefits of the system. © 2013 Springer-Verlag Berlin Heidelberg