A framework study on the use of immersive XR technologies in the cultural heritage domain

Most cultural promotion and dissemination are nowadays performed through the digitization of heritage sites and museums, a necessary requirement to meet the new needs of the public. Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR) have the potential to improve the experience quality and educational effect of these sites by stimulating users’ senses in a more natural and vivid way. In this respect, head-mounted display (HMD) devices allow visitors to enhance the experience of cultural sites by digitizing information and integrating additional virtual cues about cultural artifacts, resulting in a more immersive experience that engages the visitor both physically and emotionally. This study contributes to the development and incorporation of AR, MR, and VR applications in the cultural heritage domain by providing an overview of relevant studies utilizing fully immersive systems, such as headsets and CAVE systems, emphasizing the advantages that they bring when compared to handheld devices. We propose a framework study to identify the key features of headset-based Extended Reality (XR) technologies used in the cultural heritage domain that boost immersion, sense of presence, and agency. Furthermore, we highlight core characteristics that favor the adoption of these systems over more traditional solutions (e.g., handheld devices), as well as unsolved issues that must be addressed to improve the guests’ experience and the appreciation of the cultural heritage. An extensive search of Google Scholar, Scopus, IEEE Xplore, ACM Digital Library, and Wiley Online Library databases was conducted, including papers published from January 2018 to September 2022. To improve review reporting, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used. Sixty-five papers met the inclusion criteria and were classified depending on the study’s purpose: education, entertainment, edutainment, touristic guidance systems, accessibility, visitor profiling, and management. Immersive cultural heritage systems allow visitors to feel completely immersed and present in the virtual environment, providing a stimulating and educational cultural experience that can improve the quality and learning purposes of cultural visits. Nonetheless, the analyzed studies revealed some limitations that must be faced to give a further impulse to the adoption of these technologies in the cultural heritage domain

THE DIGITAL RESTITUTION OF LOT 3317: USING UNDERWATER IMAGE BASED MODELLING TO GENERATE VALUE IN VIRTUAL HERITAGE EXPERIENCES

The intent of this paper is to demonstrate the cultural value that Virtual Heritage experiences can generate. As a proof of concept, game resolution assets were optimized from high polygon image based models (IBMs) and rendered in real-time, within a Head Mounted Display (HMD). The case study of lot 3317 was chosen to facilitate heuristic explorations pertinent to the history of the St. Lawrence Seaway, Canada. This case study explores methods in creating and validating heritage values, while valourizing assets as a cultural resources for use in the future. While at the same time, challenging the current spatial dynamics by conceptually reducing the distance between the viewer and the flooded landscape. In addition to demonstrating the cultural values generated through the veneration of lot 3317, the paper aims to provide a model for optimizing and post processing meshes produced through Under-Water Image-Base Modelling

Roadmap on 3D integral imaging: Sensing, processing, and display

This Roadmap article on three-dimensional integral imaging provides an overview of some of the research activities in the field of integral imaging. The article discusses various aspects of the field including sensing of 3D scenes, processing of captured information, and 3D display and visualization of information. The paper consists of a series of 15 sections from the experts presenting various aspects of the field on sensing, processing, displays, augmented reality, microscopy, object recognition, and other applications. Each section represents the vision of its author to describe the progress, potential, vision, and challenging issues in this field

Taking real steps in virtual nature: a randomized blinded trial

Studies show that green exercise (i.e., physical activity in the presence of nature) can provide the synergistic psychophysiological benefits of both physical exercise and nature exposure. The present study aimed to investigate the extent to which virtual green exercise may extend these benefits to people that are unable to engage in active visits to natural environments, as well as to promote enhanced exercise behavior. After watching a video validated to elicit sadness, participants either performed a treadmill walk while exposed to one of two virtual conditions, which were created using different techniques (360° video or 3D model), or walked on a treadmill while facing a blank wall (control). Quantitative and qualitative data were collected in relation to three overarching themes: “Experience,” “Physical engagement” and “Psychophysiological recovery.” Compared to control, greater enjoyment was found in the 3D model, while lower walking speed was found in the 360° video. No significant differences among conditions were found with respect to heart rate, perceived exertion, or changes in blood pressure and affect. The analysis of qualitative data provided further understanding on the participants’ perceptions and experiences. These findings indicate that 3D model-based virtual green exercise can provide some additional benefits compared to indoor exercise, while 360° video-based virtual green exercise may result in lower physical engagement.publishedVersio

Digital Cognitive Companions for Marine Vessels : On the Path Towards Autonomous Ships

As for the automotive industry, industry and academia are making extensive efforts to create autonomous ships. The solutions for this are very technology-intense. Many building blocks, often relying on AI technology, need to work together to create a complete system that is safe and reliable to use. Even when the ships are fully unmanned, humans are still foreseen to guide the ships when unknown situations arise. This will be done through teleoperation systems.In this thesis, methods are presented to enhance the capability of two building blocks that are important for autonomous ships; a positioning system, and a system for teleoperation.The positioning system has been constructed to not rely on the Global Positioning System (GPS), as this system can be jammed or spoofed. Instead, it uses Bayesian calculations to compare the bottom depth and magnetic field measurements with known sea charts and magnetic field maps, in order to estimate the position. State-of-the-art techniques for this method typically use high-resolution maps. The problem is that there are hardly any high-resolution terrain maps available in the world. Hence we present a method using standard sea-charts. We compensate for the lower accuracy by using other domains, such as magnetic field intensity and bearings to landmarks. Using data from a field trial, we showed that the fusion method using multiple domains was more robust than using only one domain. In the second building block, we first investigated how 3D and VR approaches could support the remote operation of unmanned ships with a data connection with low throughput, by comparing respective graphical user interfaces (GUI) with a Baseline GUI following the currently applied interfaces in such contexts. Our findings show that both the 3D and VR approaches outperform the traditional approach significantly. We found the 3D GUI and VR GUI users to be better at reacting to potentially dangerous situations than the Baseline GUI users, and they could keep track of the surroundings more accurately. Building from this, we conducted a teleoperation user study using real-world data from a field-trial in the archipelago, where the users should assist the positioning system with bearings to landmarks. The users experienced the tool to give a good overview, and despite the connection with the low throughput, they managed through the GUI to significantly improve the positioning accuracy