ElectroCutscenes: Realistic Haptic Feedback in Cutscenes of Virtual Reality Games Using Electric Muscle Stimulation

Cutscenes in Virtual Reality (VR) games enhance story telling by delivering output in the form of visual, auditory, or haptic feedback (e.g., using vibrating handheld controllers). Since they lack interaction in the form of user input, cutscenes would significantly benefit from improved feedback. We introduce the concept and implementation of ElectroCutscenes, a concept in which Electric Muscle Stimulation (EMS) is leveraged to elicit physical user movements to correspond to those of personal avatars in cutscenes of VR games while the user stays passive. Through a user study (N=22) in which users passively received kinesthetic feedback resulting in involuntarily movements, we show that ElectroCutscenes significantly increases perceived presence and realism compared to controller-based vibrotactile and no haptic feedback. Furthermore, we found preliminary evidence that combining visual and EMS feedback can evoke movements that are not actuated by either of them alone. We discuss how to enhance realism and presence of cutscenes in VR games even when EMS can partially rather than completely actuate the desired body movements

Exploring the effects of replicating shape, weight and recoil effects on VR shooting controllers

Commercial Virtual Reality (VR) controllers with realistic force feedback are becoming available, to increase the realism and immersion of first-person shooting (FPS) games in VR. These controllers attempt to mimic not only the shape and weight of real guns but also their recoil effects (linear force feedback parallel to the barrel, when the gun is shot). As these controllers become more popular and affordable, this paper investigates the actual effects that these properties (shape, weight, and especially directional force feedback) have on performance for general VR users (e.g. users with no marksmanship experience), drawing conclusions for both consumers and device manufacturers. We created a prototype replicating the properties exploited by commercial VR controllers (i.e. shape, weight and adjustable force feedback) and used it to assess the effect of these parameters in user performance, across a series of user studies. We first analysed the benefits on user performance of adding weight and shape vs a conventional controller (e.g. Vive controller). We then explore the implications of adding linear force feedback (LFF), as well as replicating the shape and weight. Our studies show negligible effects on the immediate shooting performance with some improvements in subjective appreciation, which are already present with low levels of LFF. While higher levels of LFF do not increase subjective appreciations any further, they lead users to reach their maximum distance skillset more quickly. This indicates that while adding low levels of LFF can be enough to influence user’s immersion/engagement for gaming contexts, controllers with higher levels of LFF might be better suited for training environments and/or when dealing with particularly demanding aiming tasks

The Hitchhiker’s Guide to the Metaverse

Non peer reviewe

Multi-point STM: Effects of Drawing Speed and Number of Focal Points on Users’ Responses using Ultrasonic Mid-Air Haptics

Spatiotemporal modulation (STM) is used to render tactile patterns with ultrasound arrays. Previous research only explored the effects of single-point STM parameters, such as drawing speed (Vd). Here we explore the effects of multi-point STM on both perceptual (intensity) and emotional (valence/arousal) responses. This introduces a new control parameter for STM - the number of focal points (Nfp) – on top of conventional STM parameter (Vd). Our results from a study with 30 participants showed a negative effect of Nfp on perceived intensity and arousal, but no significant effects on valence. We also found the effects of Vd still aligned with prior results for single-point, even when different Nfp were used, suggesting that effects observed from single-point also apply to multi-point STM. We finally derive recommendations, such as using single-point STM to produce stimuli with higher intensity and/or arousal, or using multi-point STM for milder and more relaxing (less arousing) experience

Auto-Pa\'izo Games: Towards Understanding the Design of Games that Aim to Unify a Player's Physical Body and the Virtual World

Most digital bodily games focus on the body as they use movement as input. However, they also draw the player's focus away from the body as the output occurs on visual displays, creating a divide between the physical body and the virtual world. We propose a novel approach - the ''Body as a Play Material'' - where a player uses their body as both input and output to unify the physical body and the virtual world. To showcase this approach, we designed three games where a player uses one of their hands (input) to play against the other hand (output) by loaning control over its movements to an Electrical Muscle Stimulation (EMS) system. We conducted a thematic analysis on the data obtained from a field study with 12 participants to articulate four player experience themes. We discuss our results about how participants appreciated the engagement with the variety of bodily movements for play and the ambiguity of using their body as a play material. Ultimately, our work aims to unify the physical body and the virtual world.Comment: This paper will be published at Annual Symposium on Computer-Human Interaction in Play (CHI PLAY) 202

Ubiquitous haptic feedback in human-computer interaction through electrical muscle stimulation

[no abstract

Haptic Media Scenes

The aim of this thesis is to apply new media phenomenological and enactive embodied cognition approaches to explain the role of haptic sensitivity and communication in personal computer environments for productivity. Prior theory has given little attention to the role of haptic senses in influencing cognitive processes, and do not frame the richness of haptic communication in interaction design—as haptic interactivity in HCI has historically tended to be designed and analyzed from a perspective on communication as transmissions, sending and receiving haptic signals. The haptic sense may not only mediate contact confirmation and affirmation, but also rich semiotic and affective messages—yet this is a strong contrast between this inherent ability of haptic perception, and current day support for such haptic communication interfaces. I therefore ask: How do the haptic senses (touch and proprioception) impact our cognitive faculty when mediated through digital and sensor technologies? How may these insights be employed in interface design to facilitate rich haptic communication? To answer these questions, I use theoretical close readings that embrace two research fields, new media phenomenology and enactive embodied cognition. The theoretical discussion is supported by neuroscientific evidence, and tested empirically through case studies centered on digital art. I use these insights to develop the concept of the haptic figura, an analytical tool to frame the communicative qualities of haptic media. The concept gauges rich machine- mediated haptic interactivity and communication in systems with a material solution supporting active haptic perception, and the mediation of semiotic and affective messages that are understood and felt. As such the concept may function as a design tool for developers, but also for media critics evaluating haptic media. The tool is used to frame a discussion on opportunities and shortcomings of haptic interfaces for productivity, differentiating between media systems for the hand and the full body. The significance of this investigation is demonstrating that haptic communication is an underutilized element in personal computer environments for productivity and providing an analytical framework for a more nuanced understanding of haptic communication as enabling the mediation of a range of semiotic and affective messages, beyond notification and confirmation interactivity

Facilitating Self-monitored Physical Rehabilitation with Virtual Reality and Haptic feedback

Physical rehabilitation is essential to recovery from joint replacement operations. As a representation, total knee arthroplasty (TKA) requires patients to conduct intensive physical exercises to regain the knee's range of motion and muscle strength. However, current joint replacement physical rehabilitation methods rely highly on therapists for supervision, and existing computer-assisted systems lack consideration for enabling self-monitoring, making at-home physical rehabilitation difficult. In this paper, we investigated design recommendations that would enable self-monitored rehabilitation through clinical observations and focus group interviews with doctors and therapists. With this knowledge, we further explored Virtual Reality(VR)-based visual presentation and supplemental haptic motion guidance features in our implementation VReHab, a self-monitored and multimodal physical rehabilitation system with VR and vibrotactile and pneumatic feedback in a TKA rehabilitation context. We found that the third point of view real-time reconstructed motion on a virtual avatar overlaid with the target pose effectively provides motion awareness and guidance while haptic feedback helps enhance users' motion accuracy and stability. Finally, we implemented \systemname to facilitate self-monitored post-operative exercises and validated its effectiveness through a clinical study with 10 patients