Deep Neural Network and Data Augmentation Methodology for off-axis iris segmentation in wearable headsets

A data augmentation methodology is presented and applied to generate a large dataset of off-axis iris regions and train a low-complexity deep neural network. Although of low complexity the resulting network achieves a high level of accuracy in iris region segmentation for challenging off-axis eye-patches. Interestingly, this network is also shown to achieve high levels of performance for regular, frontal, segmentation of iris regions, comparing favorably with state-of-the-art techniques of significantly higher complexity. Due to its lower complexity, this network is well suited for deployment in embedded applications such as augmented and mixed reality headsets

Tone Reproduction in Virtual Reality

High dynamic range imaging has become very popular over the years in the field of computer graphics and games. The process of tone reproduction compresses the dynamic range of brightness in a scene to the lower range of display devices, thus making it an essential process in the graphics rendering pipeline. Various tone mapping operators have been tested for static viewing conditions. However, perceptual and temporal adaptation may vary for immersive viewing in a Virtual Reality environment. This thesis implements Ward et al. model (1994), Ward et al. model, Histogram Adjustment (1997) and Irawan, Ferwerda and Marschner model (2005) for static and immersive inputs. Faculty and students from the college took part in a personal survey to rate the tone mapped results based on their level of resemblance to real-life outdoor environments as well as the level of visibility in the lighter and darker regions. The proposed hypothesis states that immersion produces a measurable effect on our preference for a suitable tone reproduction model. This hypothesis is tested with the help of null hypothesis testing methods and some regression analysis on the data gathered from the survey

Remote Visual Observation of Real Places Through Virtual Reality Headsets

Virtual Reality has always represented a fascinating yet powerful opportunity that has attracted studies and technology developments, especially since the latest release on the market of powerful high-resolution and wide field-of-view VR headsets. While the great potential of such VR systems is common and accepted knowledge, issues remain related to how to design systems and setups capable of fully exploiting the latest hardware advances. The aim of the proposed research is to study and understand how to increase the perceived level of realism and sense of presence when remotely observing real places through VR headset displays. Hence, to produce a set of guidelines that give directions to system designers about how to optimize the display-camera setup to enhance performance, focusing on remote visual observation of real places. The outcome of this investigation represents unique knowledge that is believed to be very beneficial for better VR headset designs towards improved remote observation systems. To achieve the proposed goal, this thesis presents a thorough investigation of existing literature and previous researches, which is carried out systematically to identify the most important factors ruling realism, depth perception, comfort, and sense of presence in VR headset observation. Once identified, these factors are further discussed and assessed through a series of experiments and usability studies, based on a predefined set of research questions. More specifically, the role of familiarity with the observed place, the role of the environment characteristics shown to the viewer, and the role of the display used for the remote observation of the virtual environment are further investigated. To gain more insights, two usability studies are proposed with the aim of defining guidelines and best practices. The main outcomes from the two studies demonstrate that test users can experience an enhanced realistic observation when natural features, higher resolution displays, natural illumination, and high image contrast are used in Mobile VR. In terms of comfort, simple scene layouts and relaxing environments are considered ideal to reduce visual fatigue and eye strain. Furthermore, sense of presence increases when observed environments induce strong emotions, and depth perception improves in VR when several monocular cues such as lights and shadows are combined with binocular depth cues. Based on these results, this investigation then presents a focused evaluation on the outcomes and introduces an innovative eye-adapted High Dynamic Range (HDR) approach, which the author believes to be of great improvement in the context of remote observation when combined with eye-tracked VR headsets. Within this purpose, a third user study is proposed to compare static HDR and eye-adapted HDR observation in VR, to assess that the latter can improve realism, depth perception, sense of presence, and in certain cases even comfort. Results from this last study confirmed the author expectations, proving that eye-adapted HDR and eye tracking should be used to achieve best visual performances for remote observation in modern VR systems

Expert evaluation of aspects related to virtual reality systems and suggestions for future studies

Abstract. In this bachelor’s thesis, we review existing quantitative and qualitative research on virtual reality systems. We then present suggestions for performing a future study to combine the objective and subjective measurements of virtual reality experience. Additionally, we adapted an existing heuristics-based expert evaluation method to suit evaluating virtual reality systems. Using our method, we performed the expert evaluation on a selection of five modern consumer virtual reality systems to understand the connections between the subjective experience and the physical variables related to the virtual reality system. From this evaluation, we present findings that are used to construct discussion and to draw conclusions on these said connections. We found the most prominent conclusion to be that the experience of virtual reality is highly subjective and dependent on the content being viewed in virtual reality. Additionally, we concluded that some of the most important aspects in need of improvement are display resolution, lens design, user ergonomics, and lack of wirelessness. Finally, we state that two optimization problems are present; the first one being the optimization required to design a virtual reality system and the second one being the act of choosing a system to match a consumer’s preferred content.Tiivistelmä. Tässä kandidaatin tutkielmassa käymme läpi aiempaa kvantitatiivista ja kvalitatiivista tutkimusta virtuaalitodellisuusjärjestelmistä. Esitämme myös ehdotuksia myöhempää tutkimusta varten virtuaalitodellisuuteen liittyvien objektiivisten ja subjektiivisten mittausten yhdistämiseksi. Tämän lisäksi adaptoimme aiemman heuristiikkapohjaisen asiantuntija-arvioinnin sopimaan virtuaalitodellisuusjärjestelmien arviointiin. Käyttäen metodiamme toteutimme asiantuntija-arvioinnin viidellä modernilla kuluttajakäyttöön tarkoitetulla virtuaalitodellisuusjärjestelmällä ymmärtääksemme yhteyksiä subjektiivisen kokemuksen ja niiden fysikaalisten muuttujien välillä, jotka liittyvät virtuaalitodellisuusjärjestelmiin. Esitämme tämän asiantuntija-arvioinnin löydöksiä, ja luomme niiden avulla keskustelua, jonka avulla teemme mainittuihin yhteyksiin liittyviä johtopäätöksiä. Tärkein johtopäätöksemme oli se, että virtuaalitodellisuuden kokemus on erittäin subjektiivinen ja riippuvainen siitä sisällöstä, jota virtuaalitodellisuudessa koetaan. Aiemman lisäksi toteamme, että merkittävimpiä kehitystä kaipaavia osa-alueita ovat näytön resoluutio, linssien suunnittelu, käyttäjäergonomia ja langattomuuden puute. Viimeisenä totesimme, että virtuaalitodellisuusjärjestelmiin liittyy kaksi optimointiongelmaa; ensimmäinen liittyy järjestelmän suunnittelussa tapahtuvaan optimointiin, ja toinen liittyy sellaisen järjestelmän valitsemiseen, joka sopii kunkin kuluttajan suosimaan tarkoitukseen

DEPTH PERCEPTION IN VIRTUAL PERIPERSONAL SPACE: AN INVESTIGATION OF MOTION PARALLAX ON PERCEPTION- VS ACTION-ESTIMATIONS

The goal of the current experiment was to investigate whether the addition of Motion Parallax will allow participants to make more accurate distance estimations, in both the real and virtual worlds, as well as to determine whether perception- and action-estimations were affected similarly. Due to rising number of COVID-19 cases in 2020, all in-person testing needed to cease with only one participant being tested with the full set of conditions in the final experimental configuration and one participant having been completed the motion parallax conditions only. As a result, the two participants were combined and only the motion parallax conditions were analyzed. Due to low statistical power, no significant main effects, nor significant interactions were discovered. Once the COVID-19 pandemic has subsidised, I am intending to collect data from all twenty-four participants with the full array of conditions in order to complete the current project. An increase in distance-estimation accuracy, especially in virtual reality conditions is still expected to be found

Design methodology for 360-degree immersive video applications

360-degree immersive video applications for Head Mounted Display (HMD) devices offer great potential in providing engaging forms of experiential media solutions. Design challenges emerge though by this new kind of immersive media due to the 2D form of resources used for their construction, the lack of depth, the limited interaction, and the need to address the sense of presence. In addition, the use of Virtual Reality (VR) is related to cybersickness effects imposing further implications in moderate motion design tasks. This research project provides a systematic methodological approach in addressing those challenges and implications in 360-degree immersive video applications design. By studying and analysing methods and techniques efficiently used in the area of VR and Games design, a rigorous methodological design process is proposed. This process is introduced by the specification of the iVID (Immersive Video Interaction Design) framework. The efficiency of the iVID framework and the design methods and techniques it proposes is evaluated through two phases of user studies. Two different 360-degree immersive video prototypes have been created to serve the studies purposes. The analysis of the purposes of the studies ed to the definition of a set of design guidelines to be followed along with the iVID framework for designing 360-degree video-based experiences that are engaging and immersive

Exploring Adaptation-Based Techniques to Create Comfortable Virtual Reality Experiences

Virtual reality (VR) is transitioning from research to widespread consumer use. However, VR sickness - a type of motion sickness associated with VR usage - is believed to be a major impediment to the mass adoption of VR and it is estimatedto affect more than two-thirds of VR users. Previous research also shows that VR sickness affects some vulnerable groups, such as women, more severely than other groups. Although several strategies have been developed to mitigate VR sickness, most of them are not equally effective for all users since the effectiveness of any particular strategy varies across individuals. There are also concerns that some widely used VR sickness mitigation strategies, such as field-of-view (FOV) restriction, may have negative consequences on women. This thesis aims: 1) to provide theoretical understanding of the aspects of VR systems that cause VR sickness to affect some user more than others, with focus on sex differences, and 2) to develop adaptation-based strategies that could mitigate VR sickness for all VR users irrespective of their differences.Towards these goals, I first investigate the effectiveness of FOV restriction in reducing VR sickness across genders, and it’s effects on women’s spatial navigation ability. Then, based on findings from the first set of studies, I develop andempirically evaluate a novel adaptive eye gaze-contingent FOV restrictor that allows users to have a wider visual field while blocking their peripheral FOV. The wider visual field would be beneficial for women’s spatial navigation performance in virtual environments. Finally, I provide a novel standardized adaptation-based training paradigm that supplements existing VR sickness mitigation techniques by allowing the user to best prepare themselves for continued VR use. Evaluation of this strategy suggests that it could reduce and even eliminate VR sickness in susceptible individuals irrespective of their individual differences

Perceptual models for high-refresh-rate rendering

Rendering realistic images requires substantial computational power. With new high-refresh-rate displays as well as the renaissance of virtual reality (VR) and augmented reality (AR), one cannot expect that GPU performance will scale fast enough to meet the requirements of immersive photo-realistic rendering with current rendering techniques. In this dissertation, I follow the dual of the well-known computer vision approach: vision is inverse graphics: to improve graphical algorithms, I consider the operation of the human visual system. I propose to model and exploit the limitations of the visual system in the context of novel high-refresh-rate displays; specifically, I focus on spatio-temporal perception, a topic that has received remarkably less attention than spatial-only perception so far. I present three main contributions. First, I demonstrate the validity of the perceptual approach by presenting a conceptually simple rendering technique motivated by our eyes' limited sensitivity to high spatio-temporal change which reduces the rendering load and transmission requirement of current-generation VR headsets without introducing perceivable visual artefacts. Second, I present two visual models related to motion perception: (a) a metric for detecting flicker; and (b) a comprehensive visual model to predict perceived motion quality on monitors with arbitrary refresh rates and monitor resolutions. Third, I propose an adaptive rendering algorithm that utilises the proposed models. All algorithms operate on physical colorimetric units (instead of display-referenced pixel values), for which I provide the appropriate display measurements and models. All proposed algorithms and visual models are calibrated and validated with psychophysical experiments

Assessing the Effects of Illuminance and Correlated Color Temperature on Emotional Responses and Lighting Preferences Using Virtual Reality

This paper presents a novel approach to assessing human lighting adjustment behavior and preference in diverse lighting conditions through the evaluation of emotional feedback and behavioral data using VR. Participants (n= 27) were exposed to different lighting (n=17) conditions with different levels of illuminance and correlated color temperature (CCT) with a randomized order in a virtual office environment. Results from this study significantly advanced our understanding of preferred lighting conditions in virtual reality environments, influenced by a variety of factors such as illuminance, color temperature, order of presentation, and participant demographics. Through a comprehensive analysis of user adjustment profiles, we obtained insightful data that can guide the optimization of lighting design across various settings

The use of augmented reality on sensory evaluation of liked or disliked products : A Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Food Innovation at Lincoln University

Augmented reality (AR) applications in the food industry, are considered innovative to enrich the interactions between consumers and food products, especially for benefitting online retailers. Otherwise, the boosting market of dairy-free foods, alternative proteins and upcycled foods are driven by consumers’ pursuit of healthy and functional foods, vegetarian diets and environment sustainability. This study combined and applied these recent food trends in the sensory evaluation of food products. AR HoloLens two headsets were used to set up two product’s AR environments: (1) AR coconut view (ARC), and (2) AR dairy view (ARD). Three products were testes: (1) a dairy-free yoghurt based on a vanilla bean-flavoured coconut milk-based yoghurt (coconut yoghurt), and (2) a dairy yoghurt based on a vanilla bean-flavoured cow milk-based yoghurt (dairy-yoghurt), and (3) a transitional yoghurt named “mixed yoghurt” (made by half coconut yoghurt and half dairy yoghurt) to avoid appearance bias. The study aimed to investigate the hedonic ratings, just-about-right (JAR) ratings, check-all-that-apply (CATA) attribute terms, emotional response, purchase intent and consumer shopping behaviours of these three yoghurts (coconut yoghurt, dairy yoghurt and mixed yoghurt) under ARC, ARD and sensory booths (SB) using a 3 x 3 factorial design. The results showed that the liking scores of dairy yoghurt and mixed yoghurt were generally higher than coconut yoghurt regardless of the environments. The interaction effect of yoghurts and environments were statistically significant in terms of appearance, taste/flavour, sweetness, mouthfeel, aftertaste and overall liking. Moreover, AR contextual environments potentially improve the liking scores of the evaluated yoghurt samples. JAR and penalty analysis revealed that most of the consumers rated the sourness, sweetness, and mouthfeel of the dairy and mixed yoghurt to be just-about-right. At the same time, consumers penalized the coconut yoghurt for being “too much” in sourness, “too little” in sweetness, and “too thin” in mouthfeel. For the CATA analysis, attribute terms positively associated with overall liking (such as “sweet”, “smooth” and “creamy”), which were selected for coconut yoghurt and dairy yoghurt; whereas, the attribute terms against overall liking (such as “firm”, “heavy” and “astringent”) were selected for coconut yoghurt. No effects of environments on emotional responses were found among yoghurt types. Regarding purchase intent and the investigations of yoghurt consumption behaviour, the purchase intent of dairy yoghurt and mixed yoghurt were higher than coconut yoghurt, and “taste” and “health” were considered to be the most critical reasons for yoghurt consumption. The use of augmented reality in the sensory evaluation could affect the discrimination of food products. For instance, the significant difference of liking scores between mixed yoghurt and coconut yoghurt was found under SB, but the difference was not significant under ARC. Whether ecological validity could be improved by augmented reality in sensory studies will be investigated in further study