MetaSpace II: Object and full-body tracking for interaction and navigation in social VR

MetaSpace II (MS2) is a social Virtual Reality (VR) system where multiple users can not only see and hear but also interact with each other, grasp and manipulate objects, walk around in space, and get tactile feedback. MS2 allows walking in physical space by tracking each user's skeleton in real-time and allows users to feel by employing passive haptics i.e., when users touch or manipulate an object in the virtual world, they simultaneously also touch or manipulate a corresponding object in the physical world. To enable these elements in VR, MS2 creates a correspondence in spatial layout and object placement by building the virtual world on top of a 3D scan of the real world. Through the association between the real and virtual world, users are able to walk freely while wearing a head-mounted device, avoid obstacles like walls and furniture, and interact with people and objects. Most current virtual reality (VR) environments are designed for a single user experience where interactions with virtual objects are mediated by hand-held input devices or hand gestures. Additionally, users are only shown a representation of their hands in VR floating in front of the camera as seen from a first person perspective. We believe, representing each user as a full-body avatar that is controlled by natural movements of the person in the real world (see Figure 1d), can greatly enhance believability and a user's sense immersion in VR.Comment: 10 pages, 9 figures. Video: http://living.media.mit.edu/projects/metaspace-ii

Transitioning from Transmedia to Transreality Storyboarding to Improve the Co-Creation of the Experience Space

Transmedia storytelling is a digital based marketing approach in present day consumer markets. Typically applied to spanning or segueing stories or experiences across media such as film, books, comics and video-games to reach broader target audiences, often triggering a narrative, into which customers can participate and co-create the narrative. Common aims at customer engagement have been through shared stories on present day social media. However, for the creative-consumer, sharing on social media falls short of fully immersive storytelling ecology. Creatives (traditional designers and consumers) would benefit through tools and processes for incrementally expanding dimensions, mediums, fidelity, and shared interactions and senses across multiple media and interactive realities. This paper presents use cases of Transreality Storyboarding Framework (TSF), a design framework that affords creation of experience spaces for consumer-product engagement. Further, we propose a TSF app, to allow non-expert designers/everyday-consumers to contribute to storytelling, participation and production of product experience spaces

From ‘hands up’ to ‘hands on’: harnessing the kinaesthetic potential of educational gaming

Traditional approaches to distance learning and the student learning journey have focused on closing the gap between the experience of off-campus students and their on-campus peers. While many initiatives have sought to embed a sense of community, create virtual learning environments and even build collaborative spaces for team-based assessment and presentations, they are limited by technological innovation in terms of the types of learning styles they support and develop. Mainstream gaming development – such as with the Xbox Kinect and Nintendo Wii – have a strong element of kinaesthetic learning from early attempts to simulate impact, recoil, velocity and other environmental factors to the more sophisticated movement-based games which create a sense of almost total immersion and allow untethered (in a technical sense) interaction with the games’ objects, characters and other players. Likewise, gamification of learning has become a critical focus for the engagement of learners and its commercialisation, especially through products such as the Wii Fit. As this technology matures, there are strong opportunities for universities to utilise gaming consoles to embed levels of kinaesthetic learning into the student experience – a learning style which has been largely neglected in the distance education sector. This paper will explore the potential impact of these technologies, to broadly imagine the possibilities for future innovation in higher education

A Perspective Review on Integrating VR/AR with Haptics into STEM Education for Multi-Sensory Learning

As a result of several governments closing educational facilities in reaction to the COVID-19 pandemic in 2020, almost 80% of the world’s students were not in school for several weeks. Schools and universities are thus increasing their efforts to leverage educational resources and provide possibilities for remote learning. A variety of educational programs, platforms, and technologies are now accessible to support student learning; while these tools are important for society, they are primarily concerned with the dissemination of theoretical material. There is a lack of support for hands-on laboratory work and practical experience. This is particularly important for all disciplines related to science, technology, engineering, and mathematics (STEM), where labs and pedagogical assets must be continuously enhanced in order to provide effective study programs. In this study, we describe a unique perspective to achieving multi-sensory learning through the integration of virtual and augmented reality (VR/AR) with haptic wearables in STEM education. We address the implications of a novel viewpoint on established pedagogical notions. We want to encourage worldwide efforts to make fully immersive, open, and remote laboratory learning a reality.publishedVersio

Exploring how to use virtual tours to create an interactive customer remote experience

This paper investigates the use of Virtual Reality (VR) to develop virtual tour applications for marketing purposes. The aim is to explore how virtual technologies can support the creation of knowledge about a specific food product and the achievement of user engagement by a multi-sensory virtual tour of the real production site. The study provides design guidelines to create a valuable, multisensory experience by VR tours and demonstrate how the adoption of a user-driven approach, instead of a technology-driven approach, allows to achieve a positive intention to buy. The case study was represented by one of the excellences among Italian food products, the Parmigiano Reggiano (PR) cheese. The PR virtual tour was validated by a user testing campaign, involving more than 70 users: users' reactions and feedback were collected by human physiological data monitoring and questionnaires' administration. The research results demonstrated how virtual technologies could effectively help people to create a solid knowledge about a food product to support the marketing process and to form an intention to buy thanks to a better understanding of the quality of the local and traditional productions

A perspective review on integrating VR/AR with haptics into STEM education for multi-sensory learning

As a result of several governments closing educational facilities in reaction to the COVID-19 pandemic in 2020, almost 80% of the world’s students were not in school for several weeks. Schools and universities are thus increasing their efforts to leverage educational resources and provide possibilities for remote learning. A variety of educational programs, platforms, and technologies are now accessible to support student learning; while these tools are important for society, they are primarily concerned with the dissemination of theoretical material. There is a lack of support for hands-on laboratory work and practical experience. This is particularly important for all disciplines related to science, technology, engineering, and mathematics (STEM), where labs and pedagogical assets must be continuously enhanced in order to provide effective study programs. In this study, we describe a unique perspective to achieving multi-sensory learning through the integration of virtual and augmented reality (VR/AR) with haptic wearables in STEM education. We address the implications of a novel viewpoint on established pedagogical notions. We want to encourage worldwide efforts to make fully immersive, open, and remote laboratory learning a reality.European Union through the Erasmus+ Program under Grant 2020-1-NO01-KA203-076540, project title Integrating virtual and AUGMENTED reality with WEARable technology into engineering EDUcation (AugmentedWearEdu), https://augmentedwearedu.uia.no/ [34] (accessed on 27 March 2022). This work was also supported by the Top Research Centre Mechatronics (TRCM), University of Agder (UiA), Norwa

Beyond cute: exploring user types and design opportunities of virtual reality pet games

Virtual pet games, such as handheld games like Tamagotchi or video games like Petz, provide players with artificial pet companions or entertaining pet-raising simulations. Prior research has found that virtual pets have the potential to promote learning, collaboration, and empathy among users. While virtual reality (VR) has become an increasingly popular game medium, litle is known about users' expectations regarding game avatars, gameplay, and environments for VR-enabled pet games. We surveyed 780 respondents in an online survey and interviewed 30 participants to understand users' motivation, preferences, and game behavior in pet games played on various medium, and their expectations for VR pet games. Based on our findings, we generated three user types that reflect users' preferences and gameplay styles in VR pet games. We use these types to highlight key design opportunities and recommendations for VR pet games

Virtual Representations for Cybertherapy: A Relaxation Experience for Dementia Patients

The development of serious games has enabled new challenges for the healthcare sector in psychological, cognitive and motor rehabilitation. Thanks to Virtual Reality, stimulating and interactive experiences can be reproduced in a safe and controlled environment. This chapter illustrates the experimentation conducted in the hospital setting for the non-pharmacological treatment of cognitive disorders associated with Dementia. The therapy aims to relax patients of the agitation cluster through a gaming approach through the immersion in multisensory and natural settings in which sound and visual stimuli are provided. The study is supported by a technological architecture, including the Virtual Wall system for stereoscopic wall projection and rigid body tracking

DataLev: Mid-air Data Physicalisation Using Acoustic Levitation

Data physicalisation is a technique that encodes data through the geometric and material properties of an artefact, allowing users to engage with data in a more immersive and multi-sensory way. However, current methods of data physicalisation are limited in terms of their reconfgurability and the types of materials that can be used. Acoustophoresis—a method of suspending and manipulating materials using sound waves—ofers a promising solution to these challenges. In this paper, we present DataLev, a design space and platform for creating reconfgurable, multimodal data physicalisations with enriched materiality using acoustophoresis. We demonstrate the capabilities of DataLev through eight examples and evaluate its performance in terms of reconfgurability and materiality. Our work ofers a new approach to data physicalisation, enabling designers to create more dynamic, engaging, and expressive artefacts