Dataset and metrics for predicting local visible differences

A large number of imaging and computer graphics applications require localized information on the visibility of image distortions. Existing image quality metrics are not suitable for this task as they provide a single quality value per image. Existing visibility metrics produce visual difference maps, and are specifically designed for detecting just noticeable distortions but their predictions are often inaccurate. In this work, we argue that the key reason for this problem is the lack of large image collections with a good coverage of possible distortions that occur in different applications. To address the problem, we collect an extensive dataset of reference and distorted image pairs together with user markings indicating whether distortions are visible or not. We propose a statistical model that is designed for the meaningful interpretation of such data, which is affected by visual search and imprecision of manual marking. We use our dataset for training existing metrics and we demonstrate that their performance significantly improves. We show that our dataset with the proposed statistical model can be used to train a new CNN-based metric, which outperforms the existing solutions. We demonstrate the utility of such a metric in visually lossless JPEG compression, super-resolution and watermarking.</jats:p

Perceptual quality assessment of real-world images and videos

The development of online social-media venues and rapid advances in technology by camera and mobile device manufacturers have led to the creation and consumption of a seemingly limitless supply of visual content. However, a vast majority of these digital images and videos are often afflicted with annoying artifacts during acquisition, subsequent storage, and transmission over the network. All these factors impact the quality of the visual media as perceived by a human observer, thereby compromising their quality of experience (QoE). This dissertation focuses on constructing datasets that are representative of real-world image and video distortions as well as on designing algorithms that accurately predict the perceptual quality of images and videos. The primary goal of this research is to design and demonstrate automatic image and continuous-time video quality predictors that can effectively tackle the widely diverse authentic spatial, temporal, and network-induced distortions -- contrary to all present-day algorithms that operate on single, synthetic visual distortions and predict a single overall quality score for a given video. I introduce an image quality database which contains a large number of images captured using a representative variety of modern mobile devices and afflicted with a widely diverse authentic image distortions. I will also describe the design of an online crowdsourcing system which aided a very large-scale image quality assessment subjective study. This data collection facilitated the design of a new image quality predictor that is founded on the principles of natural scene statistics of images in different color spaces and transform domains. This new quality method is capable of assessing the quality of images with complex mixtures of distortions and yields high correlation with human perception. Pertaining to videos, this dissertation describes a video quality database created to understand the impact of network-induced distortions on an end user's quality of experience. I present the details of a large-scale subjective study that I conducted to gather continuous-time ground truth QoE scores on a collection of 180 videos afflicted with diverse stalling events. I also present my analysis of the temporal variations in the perceived QoE due to the time-varying video quality and present insights on the impact of relevant human cognitive aspects such as long-term and short-term memory and recency on quality perception. Next, I present a continuous-time objective QoE predicting model that effectively captures the complex interactions between the aforementioned human cognitive elements, spatial and temporal distortions, properties of stalling events, and models the state of any given client-side network buffer. I also show how the proposed framework can be extended by further supplementing with any number of additional inputs (or by eliminating any ineffective ones), based on the information available at the content providers during the design of adaptive stream-switching algorithms. This QoE predictor supports future research in the design of quality-aware stream-switching algorithms which could control the position, location, and length of stalls, given a network bandwidth budget and the end user's device information, such that the end user's QoE is maximized.Computer Science

In vivo functional and myeloarchitectonic mapping of human primary auditory areas

In contrast to vision, where retinotopic mapping alone can define areal borders, primary auditory areas such as A1 are best delineated by combining in vivo tonotopic mapping with postmortem cyto- or myeloarchitectonics from the same individual. We combined high-resolution (800 μm) quantitative T(1) mapping with phase-encoded tonotopic methods to map primary auditory areas (A1 and R) within the "auditory core" of human volunteers. We first quantitatively characterize the highly myelinated auditory core in terms of shape, area, cortical depth profile, and position, with our data showing considerable correspondence to postmortem myeloarchitectonic studies, both in cross-participant averages and in individuals. The core region contains two "mirror-image" tonotopic maps oriented along the same axis as observed in macaque and owl monkey. We suggest that these two maps within the core are the human analogs of primate auditory areas A1 and R. The core occupies a much smaller portion of tonotopically organized cortex on the superior temporal plane and gyrus than is generally supposed. The multimodal approach to defining the auditory core will facilitate investigations of structure-function relationships, comparative neuroanatomical studies, and promises new biomarkers for diagnosis and clinical studies

Space Station Human Factors Research Review. Volume 4: Inhouse Advanced Development and Research

A variety of human factors studies related to space station design are presented. Subjects include proximity operations and window design, spatial perceptual issues regarding displays, image management, workload research, spatial cognition, virtual interface, fault diagnosis in orbital refueling, and error tolerance and procedure aids

Underlying elements of image quality assessment: : Preference and terminology for communicating image quality characteristics

Image quality markedly affects the evaluation of images, and its control is crucial in studies using natural visual scenes as stimuli. Various image elements, such as sharpness or naturalness, can impact how observers view images and more directly how they evaluate their quality. To gain a better understanding of the types of interactions between these various elements, we conducted a study with a large set of images with multiple overlapping distortions, covering a wide range of quality variation. Observers assigned a quality rating on a 0-10 scale plus a verbal description of the images, explaining the elements on which their rating was based. Regression model predicting image quality ratings using 68 attributes uncovered the link between verbal descriptions and quality ratings and the importance of the image quality rating for each of the 68 image attributes. Brightness, naturalness, and good colors seem to be related to the highest image quality preference. However, the most important elements for predicting good image quality were related to image fidelity such as graininess and sharpness. This indicates that a certain level of image fidelity must be achieved before more subjective associations with, for instance, naturalness can emerge. Of the attributes, 72% had a negative impact on the preference judgment. This negative bias may be due to the fact that there are more ways that observers can perceive an image to fail than to excel when they are asked to evaluate image quality.Image quality markedly affects the evaluation of images, and its control is crucial in studies using natural visual scenes as stimuli. Various image elements, such as sharpness or naturalness, can impact how observers view images and, more directly, how they evaluate their quality. To gain a better understanding of the types of interactions between these various elements, we conducted a study with a large set of images with multiple overlapping distortions, covering a wide range of quality variation. Observers assigned a quality rating of the images on a 0–10 scale and gave a verbal description explaining the elements on which their rating was based. A regression model predicting image quality ratings using 68 attributes uncovered the link between verbal descriptions and quality ratings and the importance of the image quality rating for each of the 68 image attributes. Brightness, naturalness, and good colors seem to be related to the highest image quality preference. However, the most important elements for predicting good image quality were related to image fidelity such as graininess and sharpness. This indicates that a certain level of image fidelity must be achieved before more subjective associations with, for instance, naturalness can emerge. Of the attributes, 72% had a negative impact on the preference judgment. This negative bias may be due to the fact that there are more ways that observers can perceive an image to fail than to excel when they are asked to evaluate image quality.Peer reviewe

Vision Science and Technology at NASA: Results of a Workshop

A broad review is given of vision science and technology within NASA. The subject is defined and its applications in both NASA and the nation at large are noted. A survey of current NASA efforts is given, noting strengths and weaknesses of the NASA program