Immersive Neural Graphics Primitives

Neural radiance field (NeRF), in particular its extension by instant neural graphics primitives, is a novel rendering method for view synthesis that uses real-world images to build photo-realistic immersive virtual scenes. Despite its potential, research on the combination of NeRF and virtual reality (VR) remains sparse. Currently, there is no integration into typical VR systems available, and the performance and suitability of NeRF implementations for VR have not been evaluated, for instance, for different scene complexities or screen resolutions. In this paper, we present and evaluate a NeRF-based framework that is capable of rendering scenes in immersive VR allowing users to freely move their heads to explore complex real-world scenes. We evaluate our framework by benchmarking three different NeRF scenes concerning their rendering performance at different scene complexities and resolutions. Utilizing super-resolution, our approach can yield a frame rate of 30 frames per second with a resolution of 1280x720 pixels per eye. We discuss potential applications of our framework and provide an open source implementation online.Comment: Submitted to IEEE VR, currently under revie

Novel drug targets in cell wall biosynthesis exploited by gene disruption in Pseudomonas aeruginosa

This study demonstrates that cell wall targets contribute significantly to intracellular survival, in vivo growth, and pathogenesis of P. aeruginosa. In conclusion, these findings establish a link between cell wall targets and virulence of P. aeruginosa and thus may lead to development of novel drugs for the treatment of P. aeruginosa infection

Effects Of Embodiment On Generic And Content-Specific Intelligent Virtual Agents As Exhibition Guides

Intelligent Virtual Agents (IVAs) received enormous attention in recent years due to significant improvements in voice communication technologies and the convergence of different research fields such as Machine Learning, Internet of Things, and Virtual Reality (VR). Interactive conversational IVAs can appear in different forms such as voice-only or with embodied audio-visual representations showing, for example, human-like contextually related or generic three-dimensional bodies. In this paper, we analyzed the benefits of different forms of virtual agents in the context of a VR exhibition space. Our results suggest positive evidence showing large benefits of both embodied and thematically related audio-visual representations of IVAs. We discuss implications and suggestions for content developers to design believable virtual agents in the context of such installations

Mixed Reality Tunneling Effects for Stereoscopic Untethered Video-See-Through Head-Mounted Displays

We present mixed reality (MR) tunneling, a novel method to balance the trade-off between limited render performance and high visual quality of video see-through head-mounted displays (VST-HMDs) through fusing images of two types of camera sensors with different resolutions and frame rates. By merging a color video stream from an external stereoscopic camera with the grayscale VST commonly integrated into today’s standalone virtual reality (VR) headsets, we create a perceptually high-resolution and wide field of view VSTHMD prototype. The external high-resolution VST displayed at the central foveal to the para-peripheral region of the human visual field complements the low-resolution, low-latency grayscale VST at the far peripheral region, producing a tunneling effect, which simulates the human foveal and peripheral vision, with the potential to reduce cybersickness as in the tunneling effect in immersive VR. We propose two extensions to the MR tunneling method. The first one accommodates the user’s head movement speed by fading out the external VST when fast head movements are detected, thus potentially compensating for video streaming latency. The second one is a foveated MR tunneling effect, which displays the center of the external VST based on the tracked user eye movements. We evaluated the three MR tunneling methods in a within-subject study with 24 participants. The user study demonstrates the potential of our prototype and techniques based on the example of an assembly task that requires hand-eye coordination, untethered locomotion, and fine motor skills. The results demonstrate that, although not significant, the MR tunneling effects lead to higher overall usability, less perceived motion sickness, and a better sense of presence

Interacting with Neural Radiance Fields in Immersive Virtual Reality

Recent advancements in the neural radiance field (NeRF) technology, in particular its extension by instant neural graphics primitives, provide tremendous opportunities for the use of real-time immersive virtual reality (VR) applications. Moreover, the recent release of an immersive neural graphics primitives framework (immersive-ngp) brings real-time, stereoscopic NeRF rendering to the Unity game engine. However, the system and application research combining NeRF and human-computer interaction in VR is still at the very beginning.In this demo, we present multiple interactive system features for immersive-ngp with design principles focusing on improving the usability and interactivity of the framework for small to medium-scale NeRF scenes. We demonstrate that these new feature implementations such as exocentric manipulation, VR tunneling effects, and immersive scene appearance editing enable novel VR-NeRF experiences, for example, for customized experiences in inspecting a particle accelerator environment

Immersive Neural Graphics Primitives

Neural radiance field (NeRF), in particular its extension by instant neural graphics primitives, is a novel rendering method for view synthesis that uses real-world images to build photo-realistic immersive virtual scenes. Despite its potential, research on the combination of NeRF and virtual reality (VR) remains sparse. Currently, there is no integration into typical VR systems available, and the performance and suitability of NeRF implementations for VR have not been evaluated, for instance, for different scene complexities or screen resolutions. In this paper, we present and evaluate a NeRF-based framework that is capable of rendering scenes in immersive VR allowing users to freely move their heads to explore complex real-world scenes. We evaluate our framework by benchmarking three different NeRF scenes concerning their rendering performance at different scene complexities and resolutions. Utilizing super-resolution, our approach can yield a frame rate of 30 frames per second with a resolution of 1280x720 pixels per eye. We discuss potential applications of our framework and provide an open source implementation online

Interactive VRS-NeRF: Lightning fast Neural Radiance Field Rendering for Virtual Reality

In this demo, we build upon VRS-NeRF to provide an implementation for real-time variable rate shading (VRS) in virtual reality (VR). Here, we demo a recent advancement of Neural Radiance Fields (NeRF) merging multiple pixels into one reducing the overall rendering time. This system allows the real-time visualization of small to medium NeRF scenes. Further, we demonstrate the applicability of VRS-NeRF for 2D applications and further show its generalizability by providing two additional methods for the determination of the shading rate and an adaptive method for adjusting the step size during ray marching

Taming Cyclops: Mixed Reality Head-Mounted Displays as Laser Safety Goggles for Advanced Optics Laboratories

In this poster paper, we present a mixed reality application for laser eye protection based on a video see-through head-mounted display. With our setup, laser lab users perceive the real environment through the head-mounted display, using it as a substitute for laser safety goggles required by health and safety regulations. We designed and evaluated our prototype with a human-centered computing approach at the Deutsches Elektronen-Synchrotron where there exists some of the most advanced and extreme optics laboratory working conditions. We demonstrated that virtual reality headsets can be an attractive future alternative to conventional laser safety goggles

Bringing Instant Neural Graphics Primitives to Immersive Virtual Reality

Neural radiance field (NeRF), in particular, its extension by instant neural graphics primitives is a novel rendering method for view synthesis that uses real-world images to build photo-realistic immersive virtual scenes. Despite its enormous potential for virtual reality (VR) applications, there is currently little robust integration of NeRF into typical VR systems available for research and benchmarking in the VR community. In this poster paper, we present an extension to instant neural graphics primitives and bring stereoscopic, high-resolution, low-latency, 6-DoF NeRF rendering to the Unity game engine for immersive VR applications. Link to the repository: https://github.com/uhhhci/immersive-ng

Mixed Reality for Laser Safety at Advanced Optics Laboratories

Nowadays, high-power and multi-spectral lasers are increasingly demanded in many scientific experiments and industrial processes. However, direct exposure to these laser sources can lead to permanent eye damage. Individuals conducting research and development with these laser sources are required to wear laser safety goggles as a form of eye protection. Currently, most laser safety goggles are based on optical spectral filters, which could filter up to 99 % of the visible spectrum, rendering researchers working in hazardous and complex laboratory environments visually impaired.In this work, we present a novel laser eye protection method that could provide full-band laser protection without reducing the user’s visibility of the environment. We demonstrate a system based on a virtual reality (VR) headset with a stereoscopic see-through camera, which could be constructed in a way that all laser and ambient light is blocked from the human eyes. We revisit an empirical evaluation of our prototype conducted at the laser science and technology group (FS-LA) at DESY with 18 participants, including 14 laser experts. The evaluation results reveal that MR technology has the potential to offer a safe alternative to conventional laser safety goggles and has tremendous prospects in improving the complex and hazardous working conditions at advanced optics laboratories