Bridge the Gap Between VQA and Human Behavior on Omnidirectional Video: A Large-Scale Dataset and a Deep Learning Model

Omnidirectional video enables spherical stimuli with the $360 \times 180^ \circ$ viewing range. Meanwhile, only the viewport region of omnidirectional video can be seen by the observer through head movement (HM), and an even smaller region within the viewport can be clearly perceived through eye movement (EM). Thus, the subjective quality of omnidirectional video may be correlated with HM and EM of human behavior. To fill in the gap between subjective quality and human behavior, this paper proposes a large-scale visual quality assessment (VQA) dataset of omnidirectional video, called VQA-OV, which collects 60 reference sequences and 540 impaired sequences. Our VQA-OV dataset provides not only the subjective quality scores of sequences but also the HM and EM data of subjects. By mining our dataset, we find that the subjective quality of omnidirectional video is indeed related to HM and EM. Hence, we develop a deep learning model, which embeds HM and EM, for objective VQA on omnidirectional video. Experimental results show that our model significantly improves the state-of-the-art performance of VQA on omnidirectional video.Comment: Accepted by ACM MM 201

Omnidirectional video

Omnidirectional video enables direct surround immersive viewing of a scene by warping the original image into the correct perspective given a viewing direction. However, novel views from viewpoints off the camera path can only be obtained if we solve the 3D motion and calibration problem. In this paper we address the case of a parabolic catadioptric camera – a paraboloidal mirror in front of an orthographic lens – and we introduce a new representation, called the circle space, for points and lines in such images. In this circle space, we formulate an epipolar constraint involving a 4x4 fundamental matrix. We prove that the intrinsic parameters can be inferred in closed form from the 2D subspace of the new fundamental matrix from two views if they are constant or from three views if they vary. Three dimensional motion and structure can then be estimated from the decomposition of the fundamental matrix

User Experience of Interactive Omnidirectional Video: Case Hugo Simberg VR

Virtual reality technology is gaining popularity and market saturation with new types of head- mounted display devices reaching the market in the recent years. With more conventional uses, such as industrial simulators for flight lessons, the newer technologies have found their way on the consumer market, being used at home for entertainment and gaming. An emerging way to create these types of applications is using omnidirectional video as the basis. These Interactive omnidirectional video applications can be developed comparatively quickly, while still offering some challenges in the development process. User Experience Design in the field of virtual reality has been gaining more foothold in the field of virtual reality application development. User experience offers a multitude of design frameworks and approaches to find the user needs and to ensure for the best user experience. While this design process is suitable for development on this field, the user experience and design implications regarding virtual reality and interactive omnidirectional remain quite limited. User Experience design methods were used in a development process for an interactive omnidirectional video application, Hugo Simberg VR, which allows the user to explore a museum and a cathedral virtually. This thesis discusses this undertaken design process, from the initial exploratory design phases to the iterative refining design phases, and the evaluation of the finished application. The results of the evaluation identified nine experiential dimensions in interactive omnidirectional video, of which six were perceived with positive valence, and three with negative valence. These findings show that these dimensions should be considered in the design of interactive omnidirectional videos

Diversity techniques for omnidirectional telemetry coverage of the HiMAT research vehicle

The highly maneuverable aircraft technology (HiMAT) remotely piloted research vehicle (RPRV) was flight tested and a number of technological advances applicable to future fighter aircraft were demonstrated. The aircraft control system uses airborne and ground-based computers which communicate via uplink and downlink telemetry. Antenna radiation patterns are normally much less than ideal for continuous reception or transmission for all aircraft attitudes. After flight qualification and testing on other aircraft, a frequency diversity concept and an antenna diversity concept were implemented on the HiMAT vehicle to obtain omnidirectional telemetry coverage