A deep-dream virtual reality platform for studying altered perceptual phenomenology

Altered states of consciousness, such as psychotic or pharmacologically-induced hallucinations, provide a unique opportunity to examine the mechanisms underlying conscious perception. However, the phenomenological properties of these states are difficult to isolate experimentally from other, more general physiological and cognitive 36 effects of psychoactive substances or psychopathological conditions. Thus, simulating phenomenological aspects of altered states in the absence of these other more general effects provides an important experimental tool for consciousness science and psychiatry. Here we describe such a tool, which we call the Hallucination Machine. It comprises a novel combination of two powerful technologies: deep convolutional neural networks (DCNNs) and panoramic videos of natural scenes, viewed immersively through a head-mounted display (panoramic VR). By doing this, we are able to simulate visual hallucinatory experiences in a biologically plausible and ecologically valid way. Two experiments illustrate potential applications of the Hallucination Machine. First, we show that the system induces visual phenomenology qualitatively similar to classical psychedelics. In a second experiment, we find that simulated hallucinations do not evoke the temporal distortion commonly associated with altered states. Overall, the Hallucination Machine offers a valuable new technique for simulating altered phenomenology without directly altering the underlying neurophysiology

SELF-IMAGE MULTIMEDIA TECHNOLOGIES FOR FEEDFORWARD OBSERVATIONAL LEARNING

This dissertation investigates the development and use of self-images in augmented reality systems for learning and learning-based activities. This work focuses on self- modeling, a particular form of learning, actively employed in various settings for therapy or teaching. In particular, this work aims to develop novel multimedia systems to support the display and rendering of augmented self-images. It aims to use interactivity (via games) as a means of obtaining imagery for use in creating augmented self-images. Two multimedia systems are developed, discussed and analyzed. The proposed systems are validated in terms of their technical innovation and their clinical efficacy in delivering behavioral interventions for young children on the autism spectrum

Emulating Perceptual Experience of Color Vision Deficiency with Virtual Reality

Abstract. One of the major goals of Universal Design is to create experiences that are inclusive to all users, including those affected by Color Vision Deficiency. Color Vision Deficiency might have a significant impact on a users’ perception of the content or the environment. There is a range of tools already available, that can be used to either aid or automate the process of readability testing for digital interfaces and content in respect to Color Vision Deficiency. Two different approaches to addressing this issue can be found. A brief review of such methodologies is provided in this paper. The first approach (user-end) attempts to solve the problem by altering mediation between the user and the content. The second (design-end) allows the designer to view an image, or color scheme altered to recreate the perceptual experience of a user affected by Color Vision Deficiency and asses the design from the perspective of a color-blind user. With an implemented proof-of-concept we investigate the potential use of Virtual Reality Head-Mounted Displays to employ similar methodology, to allow designers or interior decorators to experience physical environments (i.e.: classroom, library or a cafeteria) from the perspective of a color-blind person. Such tools might increase the designers’ empathy towards color-blind users but also allow them to identify visual components, such as infographics or advertisement, in a physical environment that are poorly visible to color-blind users. Such tools could be developed by taking advantage of a modern Head-Mounted Displays six degrees of freedom tracking, a 360 camera and color processing filters applied during post-processing at run-time, allowing a designer to easily switch between different types of colorblindness emulation

Seeing other perspectives: Evaluating the use of virtual and augmented reality to simulate visual impairments (OpenVisSim)

Simulations of visual impairment are often used to educate and inform the public. However, evidence regarding their accuracy remains lacking. Here we evaluated the effectiveness of modern digital technologies to simulate the everyday difficulties caused by glaucoma. Twenty5 three normally-sighted adults performed two everyday tasks that glaucoma patients often report difficulties with: a visual search task in which they attempted to locate a mobile phone in virtual domestic environments (Virtual Reality; VR), and a visual mobility task in which impairments were overlaid onto a real-world environment using Augmented Reality (AR). On some trials, a gaze-contingent simulated scotoma --- generated using perimetric data from a patient with advanced glaucoma --- was presented in either the superior or inferior hemifield. The main outcome measure was task completion time. Eye and head movements were also tracked and used to assess individual differences in looking behaviors. The results showed that the simulated impairments substantial impaired performance in both the VR (visual search) and AR (visual mobility) tasks (both P < 0.001). Furthermore, and in line with previous patient data: impairments were greatest when the simulated VFL was inferior versus superior (P < 0.001), participants made more eye and head movements in the inferior VFL condition (P < 0.001), and participants rated the inferior VFL condition as more difficult (P < 0.001). Notably, the difference in performance between the inferior and superior conditions was almost as great as the difference between a superior VFL and no impairment at all (VR: 71%; AR: 70%). We conclude that modern digital simulators are able to replicate and objectively quantify some of the key everyday difficulties associated with visual impairments. Advantages, limitations and possible applications of current technologies are discussed. Instructions are also given for how to freely obtain the software described (OpenVisSim)