Predicting visible flicker in temporally changing images

Novel display algorithms such as low-persistence displays, black frame insertion, and temporal resolution multiplexing in- troduce temporal change into images at 40-180 Hz, on the bound- ary of the temporal integration of the visual system. This can lead to flicker, a highly-objectionable artifact known to induce viewer discomfort. The critical flicker frequency (CFF) alone does not model this phenomenon well, as flicker sensitivity varies with contrast, and spatial frequency; a content-aware model is re- quired. In this paper, we introduce a visual model for predicting flicker visibility in temporally changing images. The model per- forms a multi-scale analysis on the difference between consecu- tive frames, normalizing values with the spatio-temporal contrast sensitivity function as approximated by the pyramid of visibility. The output of the model is a 2D detection probability map. We ran a subjective flicker marking experiment to fit the model pa- rameters, then analyze the difference between two display algo- rithms, black frame insertion and temporal resolution multiplex- ing, to demonstrate the application of our model

Visual motion processing in migraine: enhanced motion after-effects are related to display contrast, visual symptoms, visual triggers and attack frequency

BACKGROUND: Visual after-effects are illusions that occur after prolonged viewing of visual displays. The motion after-effect (MAE), for example, is an illusory impression of motion after viewing moving displays: subsequently, stationary displays appear to drift in the opposite direction. After-effects have been used extensively in basic vision research and in clinical settings, and are enhanced in migraine. OBJECTIVES: To assess associations between (1) MAE duration and visual symptoms experienced during/between migraine/headache attacks, and (2) visual stimuli reported as migraine/headache triggers. METHODS: The MAE was elicited after viewing motion for 45 seconds. MAE duration was tested for three test contrast displays (high, medium, low). Participants also completed a headache questionnaire that included migraine/headache triggers. RESULTS: For each test contrast, the MAE was prolonged in migraine. MAE duration was associated with photophobia; visual triggers (flicker, striped patterns); and migraine or headache frequency. CONCLUSIONS: Group differences on various visual tasks have been attributed to abnormal cortical processing in migraine, such as hyperexcitability, heightened responsiveness and/or a lack of intra-cortical inhibition. The results are not consistent with hyperexcitability simply from a general lack of inhibition. Alternative multi-stage models are discussed and suggestions for further research are recommended, including visual tests in clinical assessments/clinical trials

Spatiotemporal Video Quality Assessment Method via Multiple Feature Mappings

Progressed video quality assessment (VQA) methods aim to evaluate the perceptual quality of videos in many applications but often prompt to increase computational complexity. Problems derive from the complexity of the distorted videos that are of significant concern in the communication industry, as well as the spatial-temporal content of the two-fold (spatial and temporal) distortion. Therefore, the findings of the study indicate that the information in the spatiotemporal slice (STS) images are useful in measuring video distortion. This paper mainly focuses on developing on a full reference video quality assessment algorithm estimator that integrates several features of spatiotemporal slices (STSS) of frames to form a high-performance video quality. This research work aims to evaluate video quality by utilizing several VQA databases by the following steps: (1) we first arrange the reference and test video sequences into a spatiotemporal slice representation. A collection of spatiotemporal feature maps were computed on each reference-test video. These response features are then processed by using a Structural Similarity (SSIM) to form a local frame quality.  (2) To further enhance the quality assessment, we combine the spatial feature maps with the spatiotemporal feature maps and propose the VQA model, named multiple map similarity feature deviation (MMSFD-STS). (3) We apply a sequential pooling strategy to assemble the quality indices of frames in the video quality scoring. (4) Extensive evaluations on video quality databases show that the proposed VQA algorithm achieves better/competitive performance as compared with other state- of- the- art methods

Perceptual models for high-refresh-rate rendering

Rendering realistic images requires substantial computational power. With new high-refresh-rate displays as well as the renaissance of virtual reality (VR) and augmented reality (AR), one cannot expect that GPU performance will scale fast enough to meet the requirements of immersive photo-realistic rendering with current rendering techniques. In this dissertation, I follow the dual of the well-known computer vision approach: vision is inverse graphics: to improve graphical algorithms, I consider the operation of the human visual system. I propose to model and exploit the limitations of the visual system in the context of novel high-refresh-rate displays; specifically, I focus on spatio-temporal perception, a topic that has received remarkably less attention than spatial-only perception so far. I present three main contributions. First, I demonstrate the validity of the perceptual approach by presenting a conceptually simple rendering technique motivated by our eyes' limited sensitivity to high spatio-temporal change which reduces the rendering load and transmission requirement of current-generation VR headsets without introducing perceivable visual artefacts. Second, I present two visual models related to motion perception: (a) a metric for detecting flicker; and (b) a comprehensive visual model to predict perceived motion quality on monitors with arbitrary refresh rates and monitor resolutions. Third, I propose an adaptive rendering algorithm that utilises the proposed models. All algorithms operate on physical colorimetric units (instead of display-referenced pixel values), for which I provide the appropriate display measurements and models. All proposed algorithms and visual models are calibrated and validated with psychophysical experiments

Looking away from faces: influence of high-level visual processes on saccade programming

Human faces capture attention more than other visual stimuli. Here we investigated whether such face-specific biases rely on automatic (involuntary) or voluntary orienting responses. To this end, we used an anti-saccade paradigm, which requires the ability to inhibit a reflexive automatic response and to generate a voluntary saccade in the opposite direction of the stimulus. To control for potential low-level confounds in the eye-movement data, we manipulated the high-level visual properties of the stimuli while normalizing their global low-level visual properties. Eye movements were recorded in 21 participants who performed either pro- or anti-saccades to a face, car, or noise pattern, randomly presented to the left or right of a fixation point. For each trial, a symbolic cue instructed the observer to generate either a pro-saccade or an anti-saccade. We report a significant increase in anti-saccade error rates for faces compared to cars and noise patterns, as well as faster pro-saccades to faces and cars in comparison to noise patterns. These results indicate that human faces induce stronger involuntary orienting responses than other visual objects, i.e., responses that are beyond the control of the observer. Importantly, this involuntary processing cannot be accounted for by global low-level visual factors

Evidence that Viewers Prefer Higher Frame Rate Film

High frame rate (HFR) movie-making refers to the capture and projection of movies at frame rates several times higher than the traditional 24 frames per second. This higher frame rate theoretically improves the quality of motion portrayed in movies, and helps avoid motion blur, judder and other undesirable artefacts. However, there is considerable debate in the cinema industry regarding the acceptance of HFR content given anecdotal reports of hyper-realistic imagery that reveals too much set and costume detail. Despite the potential theoretical advantages, there has been little empirical investigation of the impact of high-frame rate techniques on the viewer experience. In this study we use stereoscopic 3D content, filmed and projected at multiple frame rates (24, 48 and 60 fps), with shutter angles ranging from 90 degrees to 358 degrees, to evaluate viewer preferences. In a paired-comparison paradigm we assessed preferences along a set of five attributes (realism, motion smoothness, blur/clarity, quality of depth and overall preference). The resulting data show a clear preference for higher frame rates, particularly when contrasting 24 fps with 48 or 60 fps. We found little impact of shutter angle on viewers’ choices, with the exception of one measure (motion smoothness) for one clip type. These data are the first empirical evidence of the advantages afforded by high frame rate capture and presentation in a cinema context.https://source.sheridancollege.ca/centres_sirt_works/1000/thumbnail.jp

A no-reference video quality metric using a natural video statistical model

The demand for high quality multimedia content is increasing rapidly, which has resulted in service providers employing Quality of Service (QoS) strategies to monitor the quality of delivered content. However, the QoS parameters commonly used do not correlate well with the actual quality perceived by the end-users. Numerous objective video quality assessment (VQA) metrics have been proposed to address this problem. However, most of these metrics rely on the availability of additional information from the original undistorted video to perform adequately, which will increase the bandwidth required. This paper presents a No-Reference (NR) VQA algorithm, which extracts a Natural Video Statistical Model using both spatial and temporal features to model the quality experienced by the end-users without needing additional information from the transmitter. These features are based on the observation that the statistics of natural scenes are regular on pristine content but are significantly altered in the presence of distortion. The proposed method achieves a Spearman Rank Order Correlation Coefficient (SROCC) of 0.8161 with subjective data, which is statistically identical and sometimes superior to existing state-of-the-art full and reduced reference VQA metrics.peer-reviewe

Content-prioritised video coding for British Sign Language communication.

Video communication of British Sign Language (BSL) is important for remote interpersonal communication and for the equal provision of services for deaf people. However, the use of video telephony and video conferencing applications for BSL communication is limited by inadequate video quality. BSL is a highly structured, linguistically complete, natural language system that expresses vocabulary and grammar visually and spatially using a complex combination of facial expressions (such as eyebrow movements, eye blinks and mouth/lip shapes), hand gestures, body movements and finger-spelling that change in space and time. Accurate natural BSL communication places specific demands on visual media applications which must compress video image data for efficient transmission. Current video compression schemes apply methods to reduce statistical redundancy and perceptual irrelevance in video image data based on a general model of Human Visual System (HVS) sensitivities. This thesis presents novel video image coding methods developed to achieve the conflicting requirements for high image quality and efficient coding. Novel methods of prioritising visually important video image content for optimised video coding are developed to exploit the HVS spatial and temporal response mechanisms of BSL users (determined by Eye Movement Tracking) and the characteristics of BSL video image content. The methods implement an accurate model of HVS foveation, applied in the spatial and temporal domains, at the pre-processing stage of a current standard-based system (H.264). Comparison of the performance of the developed and standard coding systems, using methods of video quality evaluation developed for this thesis, demonstrates improved perceived quality at low bit rates. BSL users, broadcasters and service providers benefit from the perception of high quality video over a range of available transmission bandwidths. The research community benefits from a new approach to video coding optimisation and better understanding of the communication needs of deaf people