An eye tracking based virtual reality system for use inside magnetic resonance imaging systems

Abstract

Copyright © The Author(s) 2021. Patients undergoing Magnetic Resonance Imaging (MRI) often experience anxiety and sometimes distress prior to and during scanning. Here a full MRI compatible virtual reality (VR) system is described and tested with the aim of creating a radically different experience. Potential benefits could accrue from the strong sense of immersion that can be created with VR, which could create sense experiences designed to avoid the perception of being enclosed and could also provide new modes of diversion and interaction that could make even lengthy MRI examinations much less challenging. Most current VR systems rely on head mounted displays combined with head motion tracking to achieve and maintain a visceral sense of a tangible virtual world, but this technology and approach encourages physical motion, which would be unacceptable and could be physically incompatible for MRI. The proposed VR system uses gaze tracking to control and interact with a virtual world. MRI compatible cameras are used to allow real time eye tracking and robust gaze tracking is achieved through an adaptive calibration strategy in which each successive VR interaction initiated by the subject updates the gaze estimation model. A dedicated VR framework has been developed including a rich virtual world and gaze-controlled game content. To aid in achieving immersive experiences physical sensations, including noise, vibration and proprioception associated with patient table movements, have been made congruent with the presented virtual scene. A live video link allows subject-carer interaction, projecting a supportive presence into the virtual world.ERC Grant Agreement No. 319456 (dHCP project); National Institute for Health Research (NIHR) Biomedical Research Centre; Engineering and Physical Sciences Research Council [Grant Number EP/L016737/1]; Medical Research Council UK (MRC) Clinician Scientist Fellowship [MR/P008712/1] and Transition Support Award [MR/V036874/1]; EU H2020 COGIMON [ICT 644727], PH-CODING [FETOPEN 829186], TRIMANUAL[MSCA 843408]; Wellcome EPSRC Centre for Medical Engineering at King’s College London [WT 203148/Z/16/Z]