No-reference video quality assessment model based on artifact metrics for digital transmission applications

Tese (doutorado)—Universidade de Brasília, Instituto de Ciências Exatas, Departamento de Ciência da Computação, 2017.Um dos principais fatores para a redução da qualidade do conteúdo visual, em sistemas de imagem digital, são a presença de degradações introduzidas durante as etapas de processamento de sinais. Contudo, medir a qualidade de um vídeo implica em comparar direta ou indiretamente um vídeo de teste com o seu vídeo de referência. Na maioria das aplicações, os seres humanos são o meio mais confiável de estimar a qualidade de um vídeo. Embora mais confiáveis, estes métodos consomem tempo e são difíceis de incorporar em um serviço de controle de qualidade automatizado. Como alternativa, as métricas objectivas, ou seja, algoritmos, são geralmente usadas para estimar a qualidade de um vídeo automaticamente. Para desenvolver uma métrica objetiva é importante entender como as características perceptuais de um conjunto de artefatos estão relacionadas com suas forças físicas e com o incômodo percebido. Então, nós estudamos as características de diferentes tipos de artefatos comumente encontrados em vídeos comprimidos (ou seja, blocado, borrado e perda-de-pacotes) por meio de experimentos psicofísicos para medir independentemente a força e o incômodo desses artefatos, quando sozinhos ou combinados no vídeo. Nós analisamos os dados obtidos desses experimentos e propomos vários modelos de qualidade baseados nas combinações das forças perceptuais de artefatos individuais e suas interações. Inspirados pelos resultados experimentos, nós propomos uma métrica sem-referência baseada em características extraídas dos vídeos (por exemplo, informações DCT, a média da diferença absoluta entre blocos de uma imagem, variação da intensidade entre pixels vizinhos e atenção visual). Um modelo de regressão não-linear baseado em vetores de suporte (Support Vector Regression) é usado para combinar todas as características e estimar a qualidade do vídeo. Nossa métrica teve um desempenho muito melhor que as métricas de artefatos testadas e para algumas métricas com-referência (full-reference).The main causes for the reducing of visual quality in digital imaging systems are the unwanted presence of degradations introduced during processing and transmission steps. However, measuring the quality of a video implies in a direct or indirect comparison between test video and reference video. In most applications, psycho-physical experiments with human subjects are the most reliable means of determining the quality of a video. Although more reliable, these methods are time consuming and difficult to incorporate into an automated quality control service. As an alternative, objective metrics, i.e. algorithms, are generally used to estimate video quality quality automatically. To develop an objective metric, it is important understand how the perceptual characteristics of a set of artifacts are related to their physical strengths and to the perceived annoyance. Then, to study the characteristics of different types of artifacts commonly found in compressed videos (i.e. blockiness, blurriness, and packet-loss) we performed six psychophysical experiments to independently measure the strength and overall annoyance of these artifact signals when presented alone or in combination. We analyzed the data from these experiments and proposed several models for the overall annoyance based on combinations of the perceptual strengths of the individual artifact signals and their interactions. Inspired by experimental results, we proposed a no-reference video quality metric based in several features extracted from the videos (e.g. DCT information, cross-correlation of sub-sampled images, average absolute differences between block image pixels, intensity variation between neighbouring pixels, and visual attention). A non-linear regression model using a support vector (SVR) technique is used to combine all features to obtain an overall quality estimate. Our metric performed better than the tested artifact metrics and for some full-reference metrics

FPGA Implementation of Procedures for Video Quality Assessment

Video resolutions used in a variety of media are constantly rising. While manufacturers struggle to perfect their screens, it is also important to ensure high quality of displayed image. Overall quality can be measured using Mean Opinion Score (MOS). Video quality can be aected by miscellaneous artifacts, appearing at every stage of video creation and transmission. In this paper, we present a solution to calculate four distinct video quality metrics that can be applied to a real-time video quality assessment system. Our assessment module is capable of processing 8K resolution in real time set at the level of 30 frames per second. The throughput of 2.19 GB/s surpasses the performance of pure software solutions. The module was created using a high-level language to concentrate on the architectural optimization

IDENTIFIKASI DISTORSI BLUR PADA GAMBAR DIGITAL

Salah satu masalah yang sering muncul dalam dunia fotograï¬ adalah efek blur yang dapat diakibatkan baik oleh objek yang bergerak maupun gerakan kamera yang berhubungan dengan kecepatan rana (shutter speed) ketika gambar akan diambil. Paper ini menyajikan sebuah metode baru yang sederhana untuk mendeteksi kemunculan distorsi blur yang tidak diinginkan pada gambar digital. Metode yang diusulkan menggunakan transformasi discrete cosine transform (DCT) pada gambar yang telah mengalami distorsi dengan ukuran blok DCT yang bervariasi. Hasil dari pendeteksian ini kemudian digunakan untuk meningkatkan kualitas gambar melalui metode debluring berdasarkan korelasi pixel yang diterapkan pada area tertentu pada gambar yang mengandung distorsi blur ini. Hasil eksperimen menunjukkan bahwa kualitas gambar yang disempurnakan dihasilkan oleh metode debluring secara selektif menggunakan deteksi distorsi blur lokal akan lebih baik daripada yang tidak melalui proses seleksi. Dari berbagai ukuran blok yang digunakan dalam percobaan, blok berukuran 32×32 piksel menghasilkan kualitas gambar yang secara umum lebih baik. One of the problems that often arise in photography is a blurring effect that can be caused either by a moving object or camera movements that associated with the shutter speed when the picture is taken. This paper presents a simple new method for detecting the appearance of unwanted blur distortion in digital images. The proposed method uses the transformation of Discrete Cosine Transform (DCT) on the image that has been distorted with varying DCT block size. The results of the detection used to improve image quality through debluring method based on pixel correlation that applied to certain areas of the image that contains this blur distortion. The experimental results show that the enhanced picture quality produced by the method of selectively debluring using a local blur distortion detection is better than not through the selection process. From various block sizes used in the experiments, the block size of 32×32 pixel generates better picture quality

Computational inference and control of quality in multimedia services

Quality is the degree of excellence we expect of a service or a product. It is also one of the key factors that determine its value. For multimedia services, understanding the experienced quality means understanding how the delivered delity, precision and reliability correspond to the users' expectations. Yet the quality of multimedia services is inextricably linked to the underlying technology. It is developments in video recording, compression and transport as well as display technologies that enables high quality multimedia services to become ubiquitous. The constant evolution of these technologies delivers a steady increase in performance, but also a growing level of complexity. As new technologies stack on top of each other the interactions between them and their components become more intricate and obscure. In this environment optimizing the delivered quality of multimedia services becomes increasingly challenging. The factors that aect the experienced quality, or Quality of Experience (QoE), tend to have complex non-linear relationships. The subjectively perceived QoE is hard to measure directly and continuously evolves with the user's expectations. Faced with the diculty of designing an expert system for QoE management that relies on painstaking measurements and intricate heuristics, we turn to an approach based on learning or inference. The set of solutions presented in this work rely on computational intelligence techniques that do inference over the large set of signals coming from the system to deliver QoE models based on user feedback. We furthermore present solutions for inference of optimized control in systems with no guarantees for resource availability. This approach oers the opportunity to be more accurate in assessing the perceived quality, to incorporate more factors and to adapt as technology and user expectations evolve. In a similar fashion, the inferred control strategies can uncover more intricate patterns coming from the sensors and therefore implement farther-reaching decisions. Similarly to natural systems, this continuous adaptation and learning makes these systems more robust to perturbations in the environment, longer lasting accuracy and higher eciency in dealing with increased complexity. Overcoming this increasing complexity and diversity is crucial for addressing the challenges of future multimedia system. Through experiments and simulations this work demonstrates that adopting an approach of learning can improve the sub jective and objective QoE estimation, enable the implementation of ecient and scalable QoE management as well as ecient control mechanisms

Dynamic adaptation of streamed real-time E-learning videos over the internet

Even though the e-learning is becoming increasingly popular in the academic environment, the quality of synchronous e-learning video is still substandard and significant work needs to be done to improve it. The improvements have to be brought about taking into considerations both: the network requirements and the psycho- physical aspects of the human visual system. One of the problems of the synchronous e-learning video is that the head-and-shoulder video of the instructor is mostly transmitted. This video presentation can be made more interesting by transmitting shots from different angles and zooms. Unfortunately, the transmission of such multi-shot videos will increase packet delay, jitter and other artifacts caused by frequent changes of the scenes. To some extent these problems may be reduced by controlled reduction of the quality of video so as to minimise uncontrolled corruption of the stream. Hence, there is a need for controlled streaming of a multi-shot e-learning video in response to the changing availability of the bandwidth, while utilising the available bandwidth to the maximum. The quality of transmitted video can be improved by removing the redundant background data and utilising the available bandwidth for sending high-resolution foreground information. While a number of schemes exist to identify and remove the background from the foreground, very few studies exist on the identification and separation of the two based on the understanding of the human visual system. Research has been carried out to define foreground and background in the context of e-learning video on the basis of human psychology. The results have been utilised to propose methods for improving the transmission of e-learning videos. In order to transmit the video sequence efficiently this research proposes the use of Feed- Forward Controllers that dynamically characterise the ongoing scene and adjust the streaming of video based on the availability of the bandwidth. In order to satisfy a number of receivers connected by varied bandwidth links in a heterogeneous environment, the use of Multi-Layer Feed-Forward Controller has been researched. This controller dynamically characterises the complexity (number of Macroblocks per frame) of the ongoing video sequence and combines it with the knowledge of availability of the bandwidth to various receivers to divide the video sequence into layers in an optimal way before transmitting it into network. The Single-layer Feed-Forward Controller inputs the complexity (Spatial Information and Temporal Information) of the on-going video sequence along with the availability of bandwidth to a receiver and adjusts the resolution and frame rate of individual scenes to transmit the sequence optimised to give the most acceptable perceptual quality within the bandwidth constraints. The performance of the Feed-Forward Controllers have been evaluated under simulated conditions and have been found to effectively regulate the streaming of real-time e-learning videos in order to provide perceptually improved video quality within the constraints of the available bandwidth

Metodologia para medidas objetivas de qualidade de vídeo em sistemas de difusão de conteúdos audiovisuais

Tese (doutorado) - Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Elétrica, 2013.Este trabalho apresenta uma metodologia para a verificação da qualidade de vídeos em sistemas de difusão em massa. A metodologia proposta apresenta uma nova forma de medir a qualidade de vídeo de forma objetiva sem a utilização de referência para comparação entre as sequências de vídeo original e processada. As medidas no ambiente do usuário são feitas independentemente dos processamentos e da tecnologia da rede de distribuição. A medida da qualidade é feita por meio da inserção de marcas de testes nos vídeos, logo após a produção destes. O processamento e distribuição desses vídeos em um sistema de difusão em massa podem causar degradações que afetam as marcas inseridas. Os vídeos recebidos no ambiente do usuário são captados por meio de uma câmera de vídeo disponível em conjunto com o terminal, que identifica a marca inserida anteriormente à distribuição. Esta captura é feita pelo sistema óptico do ambiente do usuário após uma calibração do sistema. As marcas recebidas passam por um processamento de uma função de qualidade que tem como resultado um valor numérico indicando a qualidade deste vídeo. Como resultados da metodologia, são apresentadas comparações com sistemas de medição de qualidade de vídeo de referência completa utilizados em testes por organismos de padronização. As comparações com os sistemas VQM (Video Quality Metrics) e SSIM (Structural SIMilarity) foram feitas utilizando várias sequências de vídeo de testes, onde foi alcançada uma correlação estatística maior que 80% entre as medidas de qualidade resultantes da nova metodologia e as desses sistemas de referência. ______________________________________________________________________________ ABSTRACTThis work presents a method for verifying the quality of videos on mass broadcasting systems. The proposed method presents a new way of measuring the objective video quality without reference. The measurements in the user’s environment are made regardless of the processing and the broadcasting network technology. A quality measure is made by the insertion of test marks on videos, immediately after the content production. The processing and distribution of these videos on a mass broadcasting system can cause degradations that affect the inserted marks. The video received in the user’s environment is captured by a video camera available along with the video terminal. The captured video should identify the mark inserted before the distribution. This capture is made by the optical system in the user environment after a system calibration. The mark is decoded by the processing of a quality function which results in a numerical value indicating the quality of the video. The results of video quality metrics using this methodology were compared to stan- dardized full reference metrics, VQM (Video Quality Metrics) and SSIM (Structural Similarity), and the linear correlation between this proposed new metrics and the other two metrics was greater than 80%, indicating convergence between these metrics