Challenges and Opportunities for Designing Tactile Codecs from Audio Codecs

Haptic communications allows physical interaction over long distances and greatly complements conventional means of communications, such as audio and video. However, whilst standardized codecs for video and audio are well established, there is a lack of standardized codecs for haptics. This causes vendor lock-in and thereby greatly limits scalability, increases cost and prevents advanced usage scenarios with multi-sensors/actuators and multi-users. The aim of this paper is to introduce a new approach for understanding and encoding tactile signals, i.e. the sense of touch, among haptic interactions. Inspired by various audio codecs, we develop a similar methodology for tactile codecs. Notably, we demonstrate that tactile and audio signals are similar in both time and frequency domains, thereby allowing audio coding techniques to be adapted to tactile codecs with appropriate adjustments. We also present the differences between audio and tactile signals that should be considered in future designs. Moreover, in order to evaluate the performance of a tactile codec, we propose a potential direction of designing an objective quality metric which complements haptic mean opinion scores (h-MOS). This, we hope, will open the door for designing and assessing tactile codecs

The Penn Haptic Texture Toolkit for Modeling, Rendering, and Evaluating Haptic Virtual Textures

The Penn Haptic Texture Toolkit (HaTT) is a collection of 100 haptic texture and friction models, the recorded data from which the models were made, images of the textures, and the code and methods necessary to render these textures using an impedance-type haptic device such as a SensAble Phantom Omni. This toolkit was developed to provide haptics researchers with a method by which to compare and validate their texture modeling and rendering methods. The included rendering code has the additional benefit of allowing others, both researchers and designers, to incorporate our textures into their virtual environments, which will lead to a richer experience for the user

Introducing wearable haptics for rendering velocity feedback in VR serious games for neuro-rehabilitation of children

Rehabilitation in virtual reality offers advantages in terms of flexibility and parametrization of exercises, repeatability, and continuous data recording and analysis of the progress of the patient, also promoting high engagement and cognitive challenges. Still, most of the proposed virtual settings provide a high quality, immersive visual and audio feedback, without involving the sense of touch. In this paper, we show the design, implementation, and first evaluation of a gaming scenario for upper limb rehabilitation of children with cerebral palsy. In particular, we took care to introduce haptic feedback as a useful source of sensory information for the proposed task, considering—at the same time—the strict constraints for haptic wearable devices to comply with patient’s comfort, residual motor abilities, and with the embedded tracking features of the latest VR technologies. To show the potential of haptics in a rehabilitation setup, the proposed device and rendering method have been used to improve the velocity control of upper limb movements during the VR exercise, given its importance as a motor recovery metric. Eight healthy participants were enrolled, and results showed that haptic feedback can lead to lower speed tracking errors and higher movement smoothness, making the proposed setup suitable to be used in a rehabilitation context as a way to promote movement fluidity during exercises

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

A perspective review on integrating VR/AR with haptics into STEM education for multi-sensory learning

As a result of several governments closing educational facilities in reaction to the COVID-19 pandemic in 2020, almost 80% of the world’s students were not in school for several weeks. Schools and universities are thus increasing their efforts to leverage educational resources and provide possibilities for remote learning. A variety of educational programs, platforms, and technologies are now accessible to support student learning; while these tools are important for society, they are primarily concerned with the dissemination of theoretical material. There is a lack of support for hands-on laboratory work and practical experience. This is particularly important for all disciplines related to science, technology, engineering, and mathematics (STEM), where labs and pedagogical assets must be continuously enhanced in order to provide effective study programs. In this study, we describe a unique perspective to achieving multi-sensory learning through the integration of virtual and augmented reality (VR/AR) with haptic wearables in STEM education. We address the implications of a novel viewpoint on established pedagogical notions. We want to encourage worldwide efforts to make fully immersive, open, and remote laboratory learning a reality.European Union through the Erasmus+ Program under Grant 2020-1-NO01-KA203-076540, project title Integrating virtual and AUGMENTED reality with WEARable technology into engineering EDUcation (AugmentedWearEdu), https://augmentedwearedu.uia.no/ [34] (accessed on 27 March 2022). This work was also supported by the Top Research Centre Mechatronics (TRCM), University of Agder (UiA), Norwa

A Perspective Review on Integrating VR/AR with Haptics into STEM Education for Multi-Sensory Learning

As a result of several governments closing educational facilities in reaction to the COVID-19 pandemic in 2020, almost 80% of the world’s students were not in school for several weeks. Schools and universities are thus increasing their efforts to leverage educational resources and provide possibilities for remote learning. A variety of educational programs, platforms, and technologies are now accessible to support student learning; while these tools are important for society, they are primarily concerned with the dissemination of theoretical material. There is a lack of support for hands-on laboratory work and practical experience. This is particularly important for all disciplines related to science, technology, engineering, and mathematics (STEM), where labs and pedagogical assets must be continuously enhanced in order to provide effective study programs. In this study, we describe a unique perspective to achieving multi-sensory learning through the integration of virtual and augmented reality (VR/AR) with haptic wearables in STEM education. We address the implications of a novel viewpoint on established pedagogical notions. We want to encourage worldwide efforts to make fully immersive, open, and remote laboratory learning a reality.publishedVersio

Analysis domain model for shared virtual environments

The field of shared virtual environments, which also encompasses online games and social 3D environments, has a system landscape consisting of multiple solutions that share great functional overlap. However, there is little system interoperability between the different solutions. A shared virtual environment has an associated problem domain that is highly complex raising difficult challenges to the development process, starting with the architectural design of the underlying system. This paper has two main contributions. The first contribution is a broad domain analysis of shared virtual environments, which enables developers to have a better understanding of the whole rather than the part(s). The second contribution is a reference domain model for discussing and describing solutions - the Analysis Domain Model

Enhancing the use of Haptic Devices in Education and Entertainment

This research was part of the two-years Horizon 2020 European Project "weDRAW". The aim of the project was that "specific sensory systems have specific roles to learn specific concepts". This work explores the use of the haptic modality, stimulated by the means of force-feedback devices, to convey abstract concepts inside virtual reality. After a review of the current use of haptic devices in education, available haptic software and game engines, we focus on the implementation of an haptic plugin for game engines (HPGE, based on state of the art rendering library CHAI3D) and its evaluation in human perception experiments and multisensory integration