585 research outputs found
Visual Comfort Assessment for Stereoscopic Image Retargeting
In recent years, visual comfort assessment (VCA) for 3D/stereoscopic content
has aroused extensive attention. However, much less work has been done on the
perceptual evaluation of stereoscopic image retargeting. In this paper, we
first build a Stereoscopic Image Retargeting Database (SIRD), which contains
source images and retargeted images produced by four typical stereoscopic
retargeting methods. Then, the subjective experiment is conducted to assess
four aspects of visual distortion, i.e. visual comfort, image quality, depth
quality and the overall quality. Furthermore, we propose a Visual Comfort
Assessment metric for Stereoscopic Image Retargeting (VCA-SIR). Based on the
characteristics of stereoscopic retargeted images, the proposed model
introduces novel features like disparity range, boundary disparity as well as
disparity intensity distribution into the assessment model. Experimental
results demonstrate that VCA-SIR can achieve high consistency with subjective
perception
Quality assessment metric of stereo images considering cyclopean integration and visual saliency
In recent years, there has been great progress in the wider use of three-dimensional (3D) technologies. With increasing sources of 3D content, a useful tool is needed to evaluate the perceived quality of the 3D videos/images. This paper puts forward a framework to evaluate the quality of stereoscopic images contaminated by possible symmetric or asymmetric distortions. Human visual system (HVS) studies reveal that binocular combination models and visual saliency are the two key factors for the stereoscopic image quality assessment (SIQA) metric. Therefore inspired by such findings in HVS, this paper proposes a novel saliency map in SIQA metric for the cyclopean image called “cyclopean saliency”, which avoids complex calculations and produces good results in detecting saliency regions. Moreover, experimental results show that our metric significantly outperforms conventional 2D quality metrics and yields higher correlations with human subjective judgment than the state-of-art SIQA metrics. 3D saliency performance is also compared with “cyclopean saliency” in SIQA. It is noticed that the proposed metric is applicable to both symmetric and asymmetric distortions. It can thus be concluded that the proposed SIQA metric can provide an effective evaluation tool to assess stereoscopic image quality
End-to-end deep multi-score model for No-reference stereoscopic image quality assessment
Deep learning-based quality metrics have recently given significant
improvement in Image Quality Assessment (IQA). In the field of stereoscopic
vision, information is evenly distributed with slight disparity to the left and
right eyes. However, due to asymmetric distortion, the objective quality
ratings for the left and right images would differ, necessitating the learning
of unique quality indicators for each view. Unlike existing stereoscopic IQA
measures which focus mainly on estimating a global human score, we suggest
incorporating left, right, and stereoscopic objective scores to extract the
corresponding properties of each view, and so forth estimating stereoscopic
image quality without reference. Therefore, we use a deep multi-score
Convolutional Neural Network (CNN). Our model has been trained to perform four
tasks: First, predict the left view's quality. Second, predict the quality of
the left view. Third and fourth, predict the quality of the stereo view and
global quality, respectively, with the global score serving as the ultimate
quality. Experiments are conducted on Waterloo IVC 3D Phase 1 and Phase 2
databases. The results obtained show the superiority of our method when
comparing with those of the state-of-the-art. The implementation code can be
found at: https://github.com/o-messai/multi-score-SIQ
Stereoscopic Omnidirectional Image Quality Assessment Based on Predictive Coding Theory
Objective quality assessment of stereoscopic omnidirectional images is a
challenging problem since it is influenced by multiple aspects such as
projection deformation, field of view (FoV) range, binocular vision, visual
comfort, etc. Existing studies show that classic 2D or 3D image quality
assessment (IQA) metrics are not able to perform well for stereoscopic
omnidirectional images. However, very few research works have focused on
evaluating the perceptual visual quality of omnidirectional images, especially
for stereoscopic omnidirectional images. In this paper, based on the predictive
coding theory of the human vision system (HVS), we propose a stereoscopic
omnidirectional image quality evaluator (SOIQE) to cope with the
characteristics of 3D 360-degree images. Two modules are involved in SOIQE:
predictive coding theory based binocular rivalry module and multi-view fusion
module. In the binocular rivalry module, we introduce predictive coding theory
to simulate the competition between high-level patterns and calculate the
similarity and rivalry dominance to obtain the quality scores of viewport
images. Moreover, we develop the multi-view fusion module to aggregate the
quality scores of viewport images with the help of both content weight and
location weight. The proposed SOIQE is a parametric model without necessary of
regression learning, which ensures its interpretability and generalization
performance. Experimental results on our published stereoscopic omnidirectional
image quality assessment database (SOLID) demonstrate that our proposed SOIQE
method outperforms state-of-the-art metrics. Furthermore, we also verify the
effectiveness of each proposed module on both public stereoscopic image
datasets and panoramic image datasets
Towards Top-Down Stereoscopic Image Quality Assessment via Stereo Attention
Stereoscopic image quality assessment (SIQA) plays a crucial role in
evaluating and improving the visual experience of 3D content. Existing
binocular properties and attention-based methods for SIQA have achieved
promising performance. However, these bottom-up approaches are inadequate in
exploiting the inherent characteristics of the human visual system (HVS). This
paper presents a novel network for SIQA via stereo attention, employing a
top-down perspective to guide the quality assessment process. Our proposed
method realizes the guidance from high-level binocular signals down to
low-level monocular signals, while the binocular and monocular information can
be calibrated progressively throughout the processing pipeline. We design a
generalized Stereo AttenTion (SAT) block to implement the top-down philosophy
in stereo perception. This block utilizes the fusion-generated attention map as
a high-level binocular modulator, influencing the representation of two
low-level monocular features. Additionally, we introduce an Energy Coefficient
(EC) to account for recent findings indicating that binocular responses in the
primate primary visual cortex are less than the sum of monocular responses. The
adaptive EC can tune the magnitude of binocular response flexibly, thus
enhancing the formation of robust binocular features within our framework. To
extract the most discriminative quality information from the summation and
subtraction of the two branches of monocular features, we utilize a
dual-pooling strategy that applies min-pooling and max-pooling operations to
the respective branches. Experimental results highlight the superiority of our
top-down method in simulating the property of visual perception and advancing
the state-of-the-art in the SIQA field. The code of this work is available at
https://github.com/Fanning-Zhang/SATNet.Comment: 13 pages, 4 figure
No-reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics
We present two contributions in this work: (i) a bivariate generalized Gaussian distribution (BGGD) model for the joint distribution of luminance and disparity subband coefficients of natural stereoscopic scenes and (ii) a no-reference (NR) stereo image quality assessment algorithm based on the BGGD model. We first empirically show that a BGGD accurately models the joint distribution of luminance and disparity subband coefficients. We then show that the model parameters form good discriminatory features for NR quality assessment. Additionally, we rely on the previously established result that luminance and disparity subband coefficients of natural stereo scenes are correlated, and show that correlation also forms a good feature for NR quality assessment. These features are computed for both the left and right luminance-disparity pairs in the stereo image and consolidated into one feature vector per stereo pair. This feature set and the stereo pair׳s difference mean opinion score (DMOS) (labels) are used for supervised learning with a support vector machine (SVM). Support vector regression is used to estimate the perceptual quality of a test stereo image pair. The performance of the algorithm is evaluated over popular databases and shown to be competitive with the state-of-the-art no-reference quality assessment algorithms. Further, the strength of the proposed algorithm is demonstrated by its consistently good performance over both symmetric and asymmetric distortion types. Our algorithm is called Stereo QUality Evaluator (StereoQUE)
Full-reference stereoscopic video quality assessment using a motion sensitive HVS model
Stereoscopic video quality assessment has become a major research topic in recent years. Existing stereoscopic video quality metrics are predominantly based on stereoscopic image quality metrics extended to the time domain via for example temporal pooling. These approaches do not explicitly consider the motion sensitivity of the Human Visual System (HVS). To address this limitation, this paper introduces a novel HVS model inspired by physiological findings characterising the motion sensitive response of complex cells in the primary visual cortex (V1 area). The proposed HVS model generalises previous HVS models, which characterised the behaviour of simple and complex cells but ignored motion sensitivity, by estimating optical flow to measure scene velocity at different scales and orientations. The local motion characteristics (direction and amplitude) are used to modulate the output of complex cells. The model is applied to develop a new type of full-reference stereoscopic video quality metrics which uniquely combine non-motion sensitive and motion sensitive energy terms to mimic the response of the HVS. A tailored two-stage multi-variate stepwise regression algorithm is introduced to determine the optimal contribution of each energy term. The two proposed stereoscopic video quality metrics are evaluated on three stereoscopic video datasets. Results indicate that they achieve average correlations with subjective scores of 0.9257 (PLCC), 0.9338 and 0.9120 (SRCC), 0.8622 and 0.8306 (KRCC), and outperform previous stereoscopic video quality metrics including other recent HVS-based metrics
No reference quality assessment of stereo video based on saliency and sparsity
With the popularity of video technology, stereoscopic video quality assessment (SVQA) has become increasingly important. Existing SVQA methods cannot achieve good performance because the videos' information is not fully utilized. In this paper, we consider various information in the videos together, construct a simple model to combine and analyze the diverse features, which is based on saliency and sparsity. First, we utilize the 3-D saliency map of sum map, which remains the basic information of stereoscopic video, as a valid tool to evaluate the videos' quality. Second, we use the sparse representation to decompose the sum map of 3-D saliency into coefficients, then calculate the features based on sparse coefficients to obtain the effective expression of videos' message. Next, in order to reduce the relevance between the features, we put them into stacked auto-encoder, mapping vectors to higher dimensional space, and adding the sparse restraint, then input them into support vector machine subsequently, and finally, get the quality assessment scores. Within that process, we take the advantage of saliency and sparsity to extract and simplify features. Through the later experiment, we can see the proposed method is fitting well with the subjective scores
- …