Design and Development of a Multimodal Vest for Virtual Immersion and Guidance

This paper is focused on the development of a haptic vest to enhance immersion and realism in virtual environments, through vibrotactile feedback. The first steps to achieve touch-based communication are presented in order to set an actuation method based on vibration motors. Resulting vibrotactile patterns helping users to move inside virtual reality (VR). The research investigates human torso resolution and perception of vibration patterns, evaluating different kind of actuators at different locations on the vest. Finally, determining an appropriate distribution of vibration patterns allowed the generation of sensations that, for instance, help to guide in a mixed or virtual reality environment

Evaluation of Presence in Virtual Environments: Haptic Vest and User's Haptic Skills

This paper presents the integration of a haptic vest with a multimodal virtual environment, consisting of video, audio, and haptic feedback, with the main objective of determining how users, who interact with the virtual environment, benefit from tactile and thermal stimuli provided by the haptic vest. Some experiments are performed using a game application of a train station after an explosion. The participants of this experiment have to move inside the environment, while receiving several stimuli to check if any improvement in presence or realism in that environment is reflected on the vest. This is done by comparing the experimental results with those similar scenarios, obtained without haptic feedback. These experiments are carried out by three groups of participants who are classified on the basis of their experience in haptics and virtual reality devices. Some differences among the groups have been found, which can be related to the levels of realism and synchronization of all the elements in the multimodal environment that fulfill the expectations and maximum satisfaction level. According to the participants in the experiment, two different levels of requirements are to be defined by the system to comply with the expectations of professional and conventional users

Multisensory inclusive design with sensory substitution

Sensory substitution techniques are perceptual and cognitive phenomena used to represent one sensory form with an alternative. Current applications of sensory substitution techniques are typically focused on the development of assistive technologies whereby visually impaired users can acquire visual information via auditory and tactile cross-modal feedback. But despite their evident success in scientific research and furthering theory development in cognition, sensory substitution techniques have not yet gained widespread adoption within sensory-impaired populations. Here we argue that shifting the focus from assistive to mainstream applications may resolve some of the current issues regarding the use of sensory substitution devices to improve outcomes for those with disabilities. This article provides a tutorial guide on how to use research into multisensory processing and sensory substitution techniques from the cognitive sciences to design new inclusive cross-modal displays. A greater focus on developing inclusive mainstream applications could lead to innovative technologies that could be enjoyed by every person.</p

How much spatial information is lost in the sensory substitution process? Comparing visual, tactile, and auditory approaches

Sensory substitution devices (SSDs) can convey visuospatial information through spatialised auditory or tactile stimulation using wearable technology. However, the level of information loss associated with this transformation is unknown. In this study novice users discriminated the location of two objects at 1.2m using devices that transformed a 16x 8 depth map into spatially distributed patterns of light, sound, or touch on the abdomen. Results showed that through active sensing, participants could discriminate the vertical position of objects to a visual angle of 1°, 14°, and 21°, and their distance to 2cm, 8cm, and 29cm using these visual, auditory, and haptic SSDs respectively. Visual SSDs significantly outperformed auditory and tactile SSDs on vertical localisation, whereas for depth perception, all devices significantly differed from one another (visual > auditory > haptic). Our findings highlight the high level of acuity possible for SSDs even with low spatial resolutions (e.g. 16 8) and quantify the level of information loss attributable to this transformation for the SSD user. Finally, we discuss ways of closing this ‘modality gap’ found in SSDs and conclude that this process is best benchmarked against performance with SSDs that return to their primary modality (e.g. visuospatial into visual)

HapticSnakes: multi-haptic feedback wearable robots for immersive virtual reality

Haptic feedback plays a large role in enhancing immersion and presence in VR. However, previous research and commercial products have limitations in terms of variety and locations of delivered feedbacks. To address these challenges, we present HapticSnakes, which are snake-like waist-worn robots that can deliver multiple types of feedback in various body locations, including taps-, gestures-, airflow-, brushing- and gripper-based feedbacks. We developed two robots, one is lightweight and suitable for taps and gestures, while the other is capable of multiple types of feedback. We presented a design space based on our implementations and conducted two evaluations. Since taps are versatile, easy to deliver and largely unexplored, our first evaluation focused on distinguishability of tap strengths and locations on the front and back torso. Participants had highest accuracy in distinguishing feedback on the uppermost regions and had superior overall accuracy in distinguishing feedback strengths over locations. Our second user study investigated HapticSnakes' ability to deliver multiple feedback types within VR experiences, as well as users' impressions of wearing our robots and receiving novel feedback in VR. The results indicate that participants had distinct preferences for feedbacks and were in favor of using our robots throughout. Based on the results of our evaluations, we extract design considerations and discuss research challenges and opportunities for developing multi-haptic feedback robots. - 2019, The Author(s).Open Access funding provided by the Qatar National Library. The presented work is supported in part through Program for Leading Graduate Schools, “Graduate Program for Embodiment Informatics” by Japan’s Ministry of Education, Culture, Sports, Science and Technology. We would also like to thank Mr. Thomas Höglund for his contribution to the mechanical design and control software of the HapticSnakes system.Scopu

Designing Tactile Interfaces for Abstract Interpersonal Communication, Pedestrian Navigation and Motorcyclists Navigation

The tactile medium of communication with users is appropriate for displaying information in situations where auditory and visual mediums are saturated. There are situations where a subject's ability to receive information through either of these channels is severely restricted by the environment they are in or through any physical impairments that the subject may have. In this project, we have focused on two groups of users who need sustained visual and auditory focus in their task: Soldiers on the battle field and motorcyclists. Soldiers on the battle field use their visual and auditory capabilities to maintain awareness of their environment to guard themselves from enemy assault. One of the major challenges to coordination in a hazardous environment is maintaining communication between team members while mitigating cognitive load. Compromise in communication between team members may result in mistakes that can adversely affect the outcome of a mission. We have built two vibrotactile displays, Tactor I and Tactor II, each with nine actuators arranged in a three-by-three matrix with differing contact areas that can represent a total of 511 shapes. We used two dimensions of tactile medium, shapes and waveforms, to represent verb phrases and evaluated ability of users to perceive verb phrases the tactile code. We evaluated the effectiveness of communicating verb phrases while the users were performing two tasks simultaneously. The results showed that performing additional visual task did not affect the accuracy or the time taken to perceive tactile codes. Another challenge in coordinating Soldiers on a battle field is navigating them to respective assembly areas. We have developed HaptiGo, a lightweight haptic vest that provides pedestrians both navigational intelligence and obstacle detection capabilities. HaptiGo consists of optimally-placed vibro-tactile sensors that utilize natural and small form factor interaction cues, thus emulating the sensation of being passively guided towards the intended direction. We evaluated HaptiGo and found that it was able to successfully navigate users with timely alerts of incoming obstacles without increasing cognitive load, thereby increasing their environmental awareness. Additionally, we show that users are able to respond to directional information without training. The needs of motorcyclists are di erent from those of Soldiers. Motorcyclists' need to maintain visual and auditory situational awareness at all times is crucial since they are highly exposed on the road. Route guidance systems, such as the Garmin, have been well tested on automobilists, but remain much less safe for use by motorcyclists. Audio/visual routing systems decrease motorcyclists' situational awareness and vehicle control, and thus increase the chances of an accident. To enable motorcyclists to take advantage of route guidance while maintaining situational awareness, we created HaptiMoto, a wearable haptic route guidance system. HaptiMoto uses tactile signals to encode the distance and direction of approaching turns, thus avoiding interference with audio/visual awareness. Evaluations show that HaptiMoto is intuitive for motorcyclists, and a safer alternative to existing solutions

Accelerated and improved motor learning and rehabilitation using kinesthetic feedback

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2006.Includes bibliographical references (p. 69-71).About 21 million people in the United States [roughly 8%] have a basic motor skill inability [13], many stemming not from atrophy, but an improper mapping from the brain to the motor system. Devices exist today to aid people in rebuilding their motor system mappings, but do so in bulky, and inconvenient ways, since many of the users have adequate muscle strength, but the inability to control it properly. Hundreds of millions of people in the world participate in the arts, most of which involve motion of some sort. Typically, to become able to properly perform/paint/dance/etc, training is necessary. We learn from visual and auditory feedback, and sometimes, from the touch of a teacher. This research aims to improve the efficacy of such training with robotic touch, to enable people to become better, faster. This research proposes an augmented sensory feedback system - a lightweight comfortable wearable device that utilizes the communication channel of direct touch on the body, to give real-time feedback to the wearer about their performance in motor skill tasks. Using vibrotactile signals to indicate joint error in a user's motion, we enable a user to wear a full-body suit that provides subtle cues for the brain, as they perform a variety of motor skill tasks.(cont.) The hope is that utilizing tactile real-time feedback will act as a dance teacher or physical therapist does: by giving muscle aid through informational touch cues, not only through force or torque. This will enable people to undergo constant therapy/training, over all joints of the body simultaneously. with higher accuracy than a therapist/teacher provides. The device will enable more rapid motor rehabilitation and postural retraining to combat repetitive strain injuries (RSIs). It will also allow allow communication between a motion expert and a student in real-time [by comparing the student's performance to an expert's]. to aid in higher level motor learning skills such as sports and dance. It will function as a tool to accelerate and deepen peoples motor learning capabilities. This thesis focuses on actuator selection and feedback mechanisms for such a suit, in a low-joint-number test, comprising elements of the upper arm. Initial tests on a 5 degree-of-freedom suit show a decrease in motion errors of roughly 21% (p = 0.015), with 15% lower steady-state error (p = 0.007) and a 7% accelerated rate of learning (p = 0.007).by Jeff Lieberman.S.M