Video Quality Assessment: Exploring the Impact of Frame Rate

Technology advancements in the past decades has led to an immense increase in data traffic over various networks. Videos constitute a major percentage of this traffic and their share is projected to increase at an accelerating speed in the coming years. Service providers aim to deliver videos that have high quality while at the same time keeping the data rate as low as possible. Effective and efficient objective Video Quality Assessment~(VQA) algorithms are essential in order to provide real time estimate of video quality so that the best compromise between data rate and quality can be achieved. Data rate of video transmission can be altered by controlling different parameters of the video, among which frame rate is one of the most important parameters. So far, only limited works have been done to study the impact of frame rate variations on video quality. The purpose of this work is to investigate the impact of varying frame rate on the quality of videos and to develop novel VQA models that integrate frame rate variations into the task of quality assessment. In order to achieve this goal, we first construct two new video databases that contain videos of diverse content, and spatial and temporal resolutions. We carry out subjective studies on these databases to obtain human opinions on video quality. The subjective study allows us to evaluate the performance of well known objective VQA algorithms on cross-frame rate videos. The results reveal that there is considerable disparity between the subjective scores and the predictions from state-of-the-art objective models that do not take frame rate into consideration. We explore statistical models for video quality analysis. In particular, we conduct cross-frame local phase statistical analysis, which provides new insights on video motion smoothness as an important factor that affects video quality across different frame rates. Our evaluations of the proposed motion smoothness metric using the subject-rated databases show that this novel measure provides a new means to capture the impact of frame rate on video quality, and demonstrates strong promise at improving the performance of objective video quality assessment models. We also propose the notions of perceptual temporal aliasing factor and perceptual spatiotemporal aliasing factor to incorporate the characteristics of human visual contrast sensitivity variations into the framework of spatial and temporal aliasing analysis. We incorporate the proposed aliasing factors into the VQA process to predict the quality of video under frame rate change, resolution change, and lossy compression. Our performance evaluation using the subjective database shows that the proposed perceptual aliasing factors are strong quality predictors across-frame rate, resolution, and data rate, significantly outperforming baseline VQA methods without aliasing modeling

Terahertz Security Image Quality Assessment by No-reference Model Observers

To provide the possibility of developing objective image quality assessment (IQA) algorithms for THz security images, we constructed the THz security image database (THSID) including a total of 181 THz security images with the resolution of 127*380. The main distortion types in THz security images were first analyzed for the design of subjective evaluation criteria to acquire the mean opinion scores. Subsequently, the existing no-reference IQA algorithms, which were 5 opinion-aware approaches viz., NFERM, GMLF, DIIVINE, BRISQUE and BLIINDS2, and 8 opinion-unaware approaches viz., QAC, SISBLIM, NIQE, FISBLIM, CPBD, S3 and Fish_bb, were executed for the evaluation of the THz security image quality. The statistical results demonstrated the superiority of Fish_bb over the other testing IQA approaches for assessing the THz image quality with PLCC (SROCC) values of 0.8925 (-0.8706), and with RMSE value of 0.3993. The linear regression analysis and Bland-Altman plot further verified that the Fish__bb could substitute for the subjective IQA. Nonetheless, for the classification of THz security images, we tended to use S3 as a criterion for ranking THz security image grades because of the relatively low false positive rate in classifying bad THz image quality into acceptable category (24.69%). Interestingly, due to the specific property of THz image, the average pixel intensity gave the best performance than the above complicated IQA algorithms, with the PLCC, SROCC and RMSE of 0.9001, -0.8800 and 0.3857, respectively. This study will help the users such as researchers or security staffs to obtain the THz security images of good quality. Currently, our research group is attempting to make this research more comprehensive.Comment: 13 pages, 8 figures, 4 table

Full Reference Objective Quality Assessment for Reconstructed Background Images

With an increased interest in applications that require a clean background image, such as video surveillance, object tracking, street view imaging and location-based services on web-based maps, multiple algorithms have been developed to reconstruct a background image from cluttered scenes. Traditionally, statistical measures and existing image quality techniques have been applied for evaluating the quality of the reconstructed background images. Though these quality assessment methods have been widely used in the past, their performance in evaluating the perceived quality of the reconstructed background image has not been verified. In this work, we discuss the shortcomings in existing metrics and propose a full reference Reconstructed Background image Quality Index (RBQI) that combines color and structural information at multiple scales using a probability summation model to predict the perceived quality in the reconstructed background image given a reference image. To compare the performance of the proposed quality index with existing image quality assessment measures, we construct two different datasets consisting of reconstructed background images and corresponding subjective scores. The quality assessment measures are evaluated by correlating their objective scores with human subjective ratings. The correlation results show that the proposed RBQI outperforms all the existing approaches. Additionally, the constructed datasets and the corresponding subjective scores provide a benchmark to evaluate the performance of future metrics that are developed to evaluate the perceived quality of reconstructed background images.Comment: Associated source code: https://github.com/ashrotre/RBQI, Associated Database: https://drive.google.com/drive/folders/1bg8YRPIBcxpKIF9BIPisULPBPcA5x-Bk?usp=sharing (Email for permissions at: ashrotreasuedu