Spatial-Temporal Characteristics of Multisensory Integration

abstract: We experience spatial separation and temporal asynchrony between visual and haptic information in many virtual-reality, augmented-reality, or teleoperation systems. Three studies were conducted to examine the spatial and temporal characteristic of multisensory integration. Participants interacted with virtual springs using both visual and haptic senses, and their perception of stiffness and ability to differentiate stiffness were measured. The results revealed that a constant visual delay increased the perceived stiffness, while a variable visual delay made participants depend more on the haptic sensations in stiffness perception. We also found that participants judged stiffness stiffer when they interact with virtual springs at faster speeds, and interaction speed was positively correlated with stiffness overestimation. In addition, it has been found that participants could learn an association between visual and haptic inputs despite the fact that they were spatially separated, resulting in the improvement of typing performance. These results show the limitations of Maximum-Likelihood Estimation model, suggesting that a Bayesian inference model should be used.Dissertation/ThesisDoctoral Dissertation Human Systems Engineering 201

Perception of delay in haptic telepresence systems

Time delay is recognized as an important issue in haptic telepresence systems as it is inherent to long-distance data transmission. What factors influence haptic delay perception in a time-delayed environment are, however, largely unknown. In this article, we examine the impact of manual movement frequency and amplitude in a sinusoidal exploratory movement as well as the stiffness of the haptic environment on the detection threshold for delay in haptic feedback. The results suggest that the detection of delay in force feedback depends on the movement frequency and amplitude, while variation of the absolute feedback force level does not influence the detection threshold. A model based on the exploration movement is proposed and guidelines for system design with respect to the time delay in haptic feedback are provided

How force perception changes in different refresh rate conditions

n this work we consider the role of different refresh rates of the force feedback physical engine for haptics environments, such as robotic surgery and virtual reality surgical training systems. Two experimental force feedback tasks are evaluated in a virtual environment. Experiment I is a passive contact task, where the hand-grip is held waiting for the force feedback perception given by the contact with virtual objects. Experiment II is an active contact task, where a tool is moved in a direction until the contact perception with a pliable object. Different stiffnesses and refresh rates are factorially manipulated. To evaluate differences in the two tasks, we account for latency time inside the wall, penetration depth, and maximum force exerted against the object surface. The overall result of these experiments shows an improved sensitivity in almost all variables considered with refresh rates of 500 and 1,000 Hz compared with a refresh rate of 250 Hz, but no improved sensitivity is showed among them

How force perception changes in different refresh rate conditions

n this work we consider the role of different refresh rates of the force feedback physical engine for haptics environments, such as robotic surgery and virtual reality surgical training systems. Two experimental force feedback tasks are evaluated in a virtual environment. Experiment I is a passive contact task, where the hand-grip is held waiting for the force feedback perception given by the contact with virtual objects. Experiment II is an active contact task, where a tool is moved in a direction until the contact perception with a pliable object. Different stiffnesses and refresh rates are factorially manipulated. To evaluate differences in the two tasks, we account for latency time inside the wall, penetration depth, and maximum force exerted against the object surface. The overall result of these experiments shows an improved sensitivity in almost all variables considered with refresh rates of 500 and 1,000 Hz compared with a refresh rate of 250 Hz, but no improved sensitivity is showed among them

Wearable Vibrotactile Haptic Device for Stiffness Discrimination during Virtual Interactions

In this paper, we discuss the development of cost effective, wireless, and wearable vibrotactile haptic device for stiffness perception during an interaction with virtual objects. Our experimental setup consists of haptic device with five vibrotactile actuators, virtual reality environment tailored in Unity 3D integrating the Oculus Rift Head Mounted Display (HMD) and the Leap Motion controller. The virtual environment is able to capture touch inputs from users. Interaction forces are then rendered at 500 Hz and fed back to the wearable setup stimulating fingertips with ERM vibrotactile actuators. Amplitude and frequency of vibrations are modulated proportionally to the interaction force to simulate the stiffness of a virtual object. A quantitative and qualitative study is done to compare the discrimination of stiffness on virtual linear spring in three sensory modalities: visual only feedback, tactile only feedback, and their combination. A common psychophysics method called the Two Alternative Forced Choice (2AFC) approach is used for quantitative analysis using Just Noticeable Difference (JND) and Weber Fractions (WF). According to the psychometric experiment result, average Weber fraction values of 0.39 for visual only feedback was improved to 0.25 by adding the tactile feedback

A survey of haptics in serious gaming

Serious gaming often requires high level of realism for training and learning purposes. Haptic technology has been proved to be useful in many applications with an additional perception modality complementary to the audio and the vision. It provides novel user experience to enhance the immersion of virtual reality with a physical control-layer. This survey focuses on the haptic technology and its applications in serious gaming. Several categories of related applications are listed and discussed in details, primarily on haptics acts as cognitive aux and main component in serious games design. We categorize haptic devices into tactile, force feedback and hybrid ones to suit different haptic interfaces, followed by description of common haptic gadgets in gaming. Haptic modeling methods, in particular, available SDKs or libraries either for commercial or academic usage, are summarized. We also analyze the existing research difficulties and technology bottleneck with haptics and foresee the future research directions

The Analysis of design and manufacturing tasks using haptic and immersive VR - Some case studies

The use of virtual reality in interactive design and manufacture has been researched extensively but the practical application of this technology in industry is still very much in its infancy. This is surprising as one would have expected that, after some 30 years of research commercial applications of interactive design or manufacturing planning and analysis would be widespread throughout the product design domain. One of the major but less well known advantages of VR technology is that logging the user gives a great deal of rich data which can be used to automatically generate designs or manufacturing instructions, analyse design and manufacturing tasks, map engineering processes and, tentatively, acquire expert knowledge. The authors feel that the benefits of VR in these areas have not been fully disseminated to the wider industrial community and - with the advent of cheaper PC-based VR solutions - perhaps a wider appreciation of the capabilities of this type of technology may encourage companies to adopt VR solutions for some of their product design processes. With this in mind, this paper will describe in detail applications of haptics in assembly demonstrating how user task logging can lead to the analysis of design and manufacturing tasks at a level of detail not previously possible as well as giving usable engineering outputs. The haptic 3D VR study involves the use of a Phantom and 3D system to analyse and compare this technology against real-world user performance. This work demonstrates that the detailed logging of tasks in a virtual environment gives considerable potential for understanding how virtual tasks can be mapped onto their real world equivalent as well as showing how haptic process plans can be generated in a similar manner to the conduit design and assembly planning HMD VR tool reported in PART A. The paper concludes with a view as to how the authors feel that the use of VR systems in product design and manufacturing should evolve in order to enable the industrial adoption of this technology in the future

Force-Aware Interface via Electromyography for Natural VR/AR Interaction

While tremendous advances in visual and auditory realism have been made for virtual and augmented reality (VR/AR), introducing a plausible sense of physicality into the virtual world remains challenging. Closing the gap between real-world physicality and immersive virtual experience requires a closed interaction loop: applying user-exerted physical forces to the virtual environment and generating haptic sensations back to the users. However, existing VR/AR solutions either completely ignore the force inputs from the users or rely on obtrusive sensing devices that compromise user experience. By identifying users' muscle activation patterns while engaging in VR/AR, we design a learning-based neural interface for natural and intuitive force inputs. Specifically, we show that lightweight electromyography sensors, resting non-invasively on users' forearm skin, inform and establish a robust understanding of their complex hand activities. Fuelled by a neural-network-based model, our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. Through an interactive psychophysical study, we show that human perception of virtual objects' physical properties, such as stiffness, can be significantly enhanced by our interface. We further demonstrate that our interface enables ubiquitous control via finger tapping. Ultimately, we envision our findings to push forward research towards more realistic physicality in future VR/AR.Comment: ACM Transactions on Graphics (SIGGRAPH Asia 2022

空中ジェスチャ・インタラクションにおける触覚提示の高臨場感化および安定化に関する研究

Tohoku University昆陽雅司課