219 research outputs found
UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment
The volume of User Generated Content (UGC) has increased in recent years. The
challenge with this type of content is assessing its quality. So far, the
state-of-the-art metrics are not exhibiting a very high correlation with
perceptual quality. In this paper, we explore state-of-the-art metrics that
extract/combine natural scene statistics and deep neural network features. We
experiment with these by introducing saliency maps to improve perceptibility.
We train and test our models using public datasets, namely, YouTube-UGC and
KoNViD-1k. Preliminary results indicate that high correlations are achieved by
using only deep features while adding saliency is not always boosting the
performance. Our results and code will be made publicly available to serve as a
benchmark for the research community and can be found on our project page:
https://github.com/xinyiW915/SPIE-2023-Supplementary
Filling the gaps in video transcoder deployment in the cloud
Cloud-based deployment of content production and broadcast workflows has
continued to disrupt the industry after the pandemic. The key tools required
for unlocking cloud workflows, e.g., transcoding, metadata parsing, and
streaming playback, are increasingly commoditized. However, as video traffic
continues to increase there is a need to consider tools which offer
opportunities for further bitrate/quality gains as well as those which
facilitate cloud deployment. In this paper we consider preprocessing,
rate/distortion optimisation and cloud cost prediction tools which are only
just emerging from the research community. These tools are posed as part of the
per-clip optimisation approach to transcoding which has been adopted by large
streaming media processing entities but has yet to be made more widely
available for the industry.Comment: Camera-ready version of BEIT Conference at NAB 202
Video compression dataset and benchmark of learning-based video-quality metrics
Video-quality measurement is a critical task in video processing. Nowadays,
many implementations of new encoding standards - such as AV1, VVC, and LCEVC -
use deep-learning-based decoding algorithms with perceptual metrics that serve
as optimization objectives. But investigations of the performance of modern
video- and image-quality metrics commonly employ videos compressed using older
standards, such as AVC. In this paper, we present a new benchmark for
video-quality metrics that evaluates video compression. It is based on a new
dataset consisting of about 2,500 streams encoded using different standards,
including AVC, HEVC, AV1, VP9, and VVC. Subjective scores were collected using
crowdsourced pairwise comparisons. The list of evaluated metrics includes
recent ones based on machine learning and neural networks. The results
demonstrate that new no-reference metrics exhibit a high correlation with
subjective quality and approach the capability of top full-reference metrics.Comment: 10 pages, 4 figures, 6 tables, 1 supplementary materia
Ada-DQA: Adaptive Diverse Quality-aware Feature Acquisition for Video Quality Assessment
Video quality assessment (VQA) has attracted growing attention in recent
years. While the great expense of annotating large-scale VQA datasets has
become the main obstacle for current deep-learning methods. To surmount the
constraint of insufficient training data, in this paper, we first consider the
complete range of video distribution diversity (\ie content, distortion,
motion) and employ diverse pretrained models (\eg architecture, pretext task,
pre-training dataset) to benefit quality representation. An Adaptive Diverse
Quality-aware feature Acquisition (Ada-DQA) framework is proposed to capture
desired quality-related features generated by these frozen pretrained models.
By leveraging the Quality-aware Acquisition Module (QAM), the framework is able
to extract more essential and relevant features to represent quality. Finally,
the learned quality representation is utilized as supplementary supervisory
information, along with the supervision of the labeled quality score, to guide
the training of a relatively lightweight VQA model in a knowledge distillation
manner, which largely reduces the computational cost during inference.
Experimental results on three mainstream no-reference VQA benchmarks clearly
show the superior performance of Ada-DQA in comparison with current
state-of-the-art approaches without using extra training data of VQA.Comment: 10 pages, 5 figures, to appear in ACM MM 202
- …