301 research outputs found
Blind Omnidirectional Image Quality Assessment with Viewport Oriented Graph Convolutional Networks
Quality assessment of omnidirectional images has become increasingly urgent
due to the rapid growth of virtual reality applications. Different from
traditional 2D images and videos, omnidirectional contents can provide
consumers with freely changeable viewports and a larger field of view covering
the spherical surface, which makes the objective
quality assessment of omnidirectional images more challenging. In this paper,
motivated by the characteristics of the human vision system (HVS) and the
viewing process of omnidirectional contents, we propose a novel Viewport
oriented Graph Convolution Network (VGCN) for blind omnidirectional image
quality assessment (IQA). Generally, observers tend to give the subjective
rating of a 360-degree image after passing and aggregating different viewports
information when browsing the spherical scenery. Therefore, in order to model
the mutual dependency of viewports in the omnidirectional image, we build a
spatial viewport graph. Specifically, the graph nodes are first defined with
selected viewports with higher probabilities to be seen, which is inspired by
the HVS that human beings are more sensitive to structural information. Then,
these nodes are connected by spatial relations to capture interactions among
them. Finally, reasoning on the proposed graph is performed via graph
convolutional networks. Moreover, we simultaneously obtain global quality using
the entire omnidirectional image without viewport sampling to boost the
performance according to the viewing experience. Experimental results
demonstrate that our proposed model outperforms state-of-the-art full-reference
and no-reference IQA metrics on two public omnidirectional IQA databases
ShadowNeuS: Neural SDF Reconstruction by Shadow Ray Supervision
By supervising camera rays between a scene and multi-view image planes, NeRF
reconstructs a neural scene representation for the task of novel view
synthesis. On the other hand, shadow rays between the light source and the
scene have yet to be considered. Therefore, we propose a novel shadow ray
supervision scheme that optimizes both the samples along the ray and the ray
location. By supervising shadow rays, we successfully reconstruct a neural SDF
of the scene from single-view pure shadow or RGB images under multiple lighting
conditions. Given single-view binary shadows, we train a neural network to
reconstruct a complete scene not limited by the camera's line of sight. By
further modeling the correlation between the image colors and the shadow rays,
our technique can also be effectively extended to RGB inputs. We compare our
method with previous works on challenging tasks of shape reconstruction from
single-view binary shadow or RGB images and observe significant improvements.
The code and data will be released.Comment: Project page: https://gerwang.github.io/shadowneus
Empirical essays on dynamic decision making
This thesis is a collection of empirical and experimental studies on dynamic decision
making.
Chapter 1 studies the non-linear incentive of academics in economics departments of
the U.K. high education institutions based on the data throughout the last four
RAEs/REFs (i.e., RAE1996, RAE2001, RAE2008, REF2014). The time-discontinuity
features of the RAEs/REFs and the constraints on job moving result in academics facing
non-linear incentives. The data shows that in a harsh working environment with a
periodical decline of the UK economics study, academic economists respond to such
incentives by postponing the publication of their high-quality outputs to the beginning
of the next assessment period, as expected.
Chapter 2 presents an experiment designed to study how people play a two-person two-stage
dynamic game with incomplete information and uncertainty and to study the
effect of different elicitation methods on equilibrium and level-k play. The experimental
data shows that around half of the subjects are strategic thinkers and level-k thinking
dominates in strategic thinking. Furthermore, the comparison between the direct-response
and
the
strategy
method
reveals
that
the
latter
method
has
a
negative
effect
on
players’
strategic
thinking.
Chapter 3 is an experimental study of the intertemporal consumption and saving
behaviour of agents who have a finite lifecycle in an endowment economy in the
presence of two different time profiles of taxes. A series of farsighted models (i.e.,
rational expectation and adaptive learning) and myopic models are introduced to
explain players’ saving behaviour in the presence of a tax decrease in the middle of their
lifecycle. In this setting, the data analysis shows that most of the subjects’ behaviours
are consistent with the suggestions of myopic models
No-Reference Quality Assessment for 360-degree Images by Analysis of Multi-frequency Information and Local-global Naturalness
360-degree/omnidirectional images (OIs) have achieved remarkable attentions
due to the increasing applications of virtual reality (VR). Compared to
conventional 2D images, OIs can provide more immersive experience to consumers,
benefitting from the higher resolution and plentiful field of views (FoVs).
Moreover, observing OIs is usually in the head mounted display (HMD) without
references. Therefore, an efficient blind quality assessment method, which is
specifically designed for 360-degree images, is urgently desired. In this
paper, motivated by the characteristics of the human visual system (HVS) and
the viewing process of VR visual contents, we propose a novel and effective
no-reference omnidirectional image quality assessment (NR OIQA) algorithm by
Multi-Frequency Information and Local-Global Naturalness (MFILGN).
Specifically, inspired by the frequency-dependent property of visual cortex, we
first decompose the projected equirectangular projection (ERP) maps into
wavelet subbands. Then, the entropy intensities of low and high frequency
subbands are exploited to measure the multi-frequency information of OIs.
Besides, except for considering the global naturalness of ERP maps, owing to
the browsed FoVs, we extract the natural scene statistics features from each
viewport image as the measure of local naturalness. With the proposed
multi-frequency information measurement and local-global naturalness
measurement, we utilize support vector regression as the final image quality
regressor to train the quality evaluation model from visual quality-related
features to human ratings. To our knowledge, the proposed model is the first
no-reference quality assessment method for 360-degreee images that combines
multi-frequency information and image naturalness. Experimental results on two
publicly available OIQA databases demonstrate that our proposed MFILGN
outperforms state-of-the-art approaches
Stereoscopic Omnidirectional Image Quality Assessment Based on Predictive Coding Theory
Objective quality assessment of stereoscopic omnidirectional images is a
challenging problem since it is influenced by multiple aspects such as
projection deformation, field of view (FoV) range, binocular vision, visual
comfort, etc. Existing studies show that classic 2D or 3D image quality
assessment (IQA) metrics are not able to perform well for stereoscopic
omnidirectional images. However, very few research works have focused on
evaluating the perceptual visual quality of omnidirectional images, especially
for stereoscopic omnidirectional images. In this paper, based on the predictive
coding theory of the human vision system (HVS), we propose a stereoscopic
omnidirectional image quality evaluator (SOIQE) to cope with the
characteristics of 3D 360-degree images. Two modules are involved in SOIQE:
predictive coding theory based binocular rivalry module and multi-view fusion
module. In the binocular rivalry module, we introduce predictive coding theory
to simulate the competition between high-level patterns and calculate the
similarity and rivalry dominance to obtain the quality scores of viewport
images. Moreover, we develop the multi-view fusion module to aggregate the
quality scores of viewport images with the help of both content weight and
location weight. The proposed SOIQE is a parametric model without necessary of
regression learning, which ensures its interpretability and generalization
performance. Experimental results on our published stereoscopic omnidirectional
image quality assessment database (SOLID) demonstrate that our proposed SOIQE
method outperforms state-of-the-art metrics. Furthermore, we also verify the
effectiveness of each proposed module on both public stereoscopic image
datasets and panoramic image datasets
- …