Underlying elements of image quality assessment: : Preference and terminology for communicating image quality characteristics

Image quality markedly affects the evaluation of images, and its control is crucial in studies using natural visual scenes as stimuli. Various image elements, such as sharpness or naturalness, can impact how observers view images and more directly how they evaluate their quality. To gain a better understanding of the types of interactions between these various elements, we conducted a study with a large set of images with multiple overlapping distortions, covering a wide range of quality variation. Observers assigned a quality rating on a 0-10 scale plus a verbal description of the images, explaining the elements on which their rating was based. Regression model predicting image quality ratings using 68 attributes uncovered the link between verbal descriptions and quality ratings and the importance of the image quality rating for each of the 68 image attributes. Brightness, naturalness, and good colors seem to be related to the highest image quality preference. However, the most important elements for predicting good image quality were related to image fidelity such as graininess and sharpness. This indicates that a certain level of image fidelity must be achieved before more subjective associations with, for instance, naturalness can emerge. Of the attributes, 72% had a negative impact on the preference judgment. This negative bias may be due to the fact that there are more ways that observers can perceive an image to fail than to excel when they are asked to evaluate image quality.Image quality markedly affects the evaluation of images, and its control is crucial in studies using natural visual scenes as stimuli. Various image elements, such as sharpness or naturalness, can impact how observers view images and, more directly, how they evaluate their quality. To gain a better understanding of the types of interactions between these various elements, we conducted a study with a large set of images with multiple overlapping distortions, covering a wide range of quality variation. Observers assigned a quality rating of the images on a 0–10 scale and gave a verbal description explaining the elements on which their rating was based. A regression model predicting image quality ratings using 68 attributes uncovered the link between verbal descriptions and quality ratings and the importance of the image quality rating for each of the 68 image attributes. Brightness, naturalness, and good colors seem to be related to the highest image quality preference. However, the most important elements for predicting good image quality were related to image fidelity such as graininess and sharpness. This indicates that a certain level of image fidelity must be achieved before more subjective associations with, for instance, naturalness can emerge. Of the attributes, 72% had a negative impact on the preference judgment. This negative bias may be due to the fact that there are more ways that observers can perceive an image to fail than to excel when they are asked to evaluate image quality.Peer reviewe

Kuvanlaatukokemuksen arvionnin instrumentit

This dissertation describes the instruments available for image quality evaluation, develops new methods for subjective image quality evaluation and provides image and video databases for the assessment and development of image quality assessment (IQA) algorithms. The contributions of the thesis are based on six original publications. The first publication introduced the VQone toolbox for subjective image quality evaluation. It created a platform for free-form experimentation with standardized image quality methods and was the foundation for later studies. The second publication focused on the dilemma of reference in subjective experiments by proposing a new method for image quality evaluation: the absolute category rating with dynamic reference (ACR-DR). The third publication presented a database (CID2013) in which 480 images were evaluated by 188 observers using the ACR-DR method proposed in the prior publication. Providing databases of image files along with their quality ratings is essential in the field of IQA algorithm development. The fourth publication introduced a video database (CVD2014) based on having 210 observers rate 234 video clips. The temporal aspect of the stimuli creates peculiar artifacts and degradations, as well as challenges to experimental design and video quality assessment (VQA) algorithms. When the CID2013 and CVD2014 databases were published, most state-of-the-art I/VQAs had been trained on and tested against databases created by degrading an original image or video with a single distortion at a time. The novel aspect of CID2013 and CVD2014 was that they consisted of multiple concurrent distortions. To facilitate communication and understanding among professionals in various fields of image quality as well as among non-professionals, an attribute lexicon of image quality, the image quality wheel, was presented in the fifth publication of this thesis. Reference wheels and terminology lexicons have a long tradition in sensory evaluation contexts, such as taste experience studies, where they are used to facilitate communication among interested stakeholders; however, such an approach has not been common in visual experience domains, especially in studies on image quality. The sixth publication examined how the free descriptions given by the observers influenced the ratings of the images. Understanding how various elements, such as perceived sharpness and naturalness, affect subjective image quality can help to understand the decision-making processes behind image quality evaluation. Knowing the impact of each preferential attribute can then be used for I/VQA algorithm development; certain I/VQA algorithms already incorporate low-level human visual system (HVS) models in their algorithms.Väitöskirja tarkastelee ja kehittää uusia kuvanlaadun arvioinnin menetelmiä, sekä tarjoaa kuva- ja videotietokantoja kuvanlaadun arviointialgoritmien (IQA) testaamiseen ja kehittämiseen. Se, mikä koetaan kauniina ja miellyttävänä, on psykologisesti kiinnostava kysymys. Työllä on myös merkitystä teollisuuteen kameroiden kuvanlaadun kehittämisessä. Väitöskirja sisältää kuusi julkaisua, joissa tarkastellaan aihetta eri näkökulmista. I. julkaisussa kehitettiin sovellus keräämään ihmisten antamia arvioita esitetyistä kuvista tutkijoiden vapaaseen käyttöön. Se antoi mahdollisuuden testata standardoituja kuvanlaadun arviointiin kehitettyjä menetelmiä ja kehittää niiden pohjalta myös uusia menetelmiä luoden perustan myöhemmille tutkimuksille. II. julkaisussa kehitettiin uusi kuvanlaadun arviointimenetelmä. Menetelmä hyödyntää sarjallista kuvien esitystapaa, jolla muodostettiin henkilöille mielikuva kuvien laatuvaihtelusta ennen varsinaista arviointia. Tämän todettiin vähentävän tulosten hajontaa ja erottelevan pienempiä kuvanlaatueroja. III. julkaisussa kuvaillaan tietokanta, jossa on 188 henkilön 480 kuvasta antamat laatuarviot ja niihin liittyvät kuvatiedostot. Tietokannat ovat arvokas työkalu pyrittäessä kehittämään algoritmeja kuvanlaadun automaattiseen arvosteluun. Niitä tarvitaan mm. opetusmateriaalina tekoälyyn pohjautuvien algoritmien kehityksessä sekä vertailtaessa eri algoritmien suorituskykyä toisiinsa. Mitä paremmin algoritmin tuottama ennuste korreloi ihmisten antamiin laatuarvioihin, sen parempi suorituskyky sillä voidaan sanoa olevan. IV. julkaisussa esitellään tietokanta, jossa on 210 henkilön 234 videoleikkeestä tekemät laatuarviot ja niihin liittyvät videotiedostot. Ajallisen ulottuvuuden vuoksi videoärsykkeiden virheet ovat erilaisia kuin kuvissa, mikä tuo omat haasteensa videoiden laatua arvioiville algoritmeille (VQA). Aikaisempien tietokantojen ärsykkeet on muodostettu esimerkiksi sumentamalla yksittäistä kuvaa asteittain, jolloin ne sisältävät vain yksiulotteisia vääristymiä. Nyt esitetyt tietokannat poikkeavat aikaisemmista ja sisältävät useita samanaikaisia vääristymistä, joiden interaktio kuvanlaadulle voi olla merkittävää. V. julkaisussa esitellään kuvanlaatuympyrä (image quality wheel). Se on kuvanlaadun käsitteiden sanasto, joka on kerätty analysoimalla 146 henkilön tuottamat 39 415 kuvanlaadun sanallista kuvausta. Sanastoilla on pitkät perinteet aistinvaraisen arvioinnin tutkimusperinteessä, mutta niitä ei ole aikaisemmin kehitetty kuvanlaadulle. VI. tutkimuksessa tutkittiin, kuinka arvioitsijoiden antamat käsitteet vaikuttavat kuvien laadun arviointiin. Esimerkiksi kuvien arvioitu terävyys tai luonnollisuus auttaa ymmärtämään laadunarvioinnin taustalla olevia päätöksentekoprosesseja. Tietoa voidaan käyttää esimerkiksi kuvan- ja videonlaadun arviointialgoritmien (I/VQA) kehitystyössä

Analysing the visual dynamics of spatial morphology

Recently there has been a revival of interest in visibility analysis of architectural configurations. The new analyses rely heavily on computing power and statistical analysis, two factors which, according to the postpositivist school of geography, should immediately cause us to be wary. Thedanger, they would suggest, is in the application of a reductionist formal mathematical description in order to `explain' multilayered sociospatial phenomena. The author presents an attempt to rationalise how we can use visibility analysis to explore architecture in this multilayered context by considering the dynamics that lead to the visual experience. In particular, it is recommended that we assess the visualprocess of inhabitation, rather than assess the visibility in vacuo. In order to investigate the possibilities and limitations of the methodology, an urban environment is analysed by means of an agent-based model of visual actors within the configuration. The results obtained from the model are compared with actual pedestrian movement and other analytic measurements of the area: the agents correlate well both with human movement patterns and with configurational relationship as analysed by space-syntax methods. The application of both methods in combination improves on the correlation with observed movement of either, which in turn implies that an understanding of both the process of inhabitation and the principles of configuration may play a crucial role in determining the social usage of space

Modeling the Effect of Images on Product Choices

Conjoint is one of the most popular methods in marketing research, widely used to understand how customers trade-off features of a product. Since product images have a strong influence on customer choice, it is natural to want to include images in conjoint studies, yet this has proven to be difficult, since images are difficult to parsimoniously characterize in the utility function. This paper proposes a novel approach to account for the effect of images on respondents’ choices, in which consumer heterogeneity in the appeal of the images is modeled through the covariance structure in a probit model. The covariance structure is informed by a separate task where respondents rate the images included in the study. In our application to midsize crossover vehicles, we show that our approach readily scales to a large number of images, fits better than several alternatives commonly used in practice, and makes more reasonable predictions about product substitution when a new product enters the market. We discuss how this approach could be used predict the effect of other difficult-to-characterize product attribute such as sound quality or taste on product choice

SAVOIAS: A Diverse, Multi-Category Visual Complexity Dataset

Visual complexity identifies the level of intricacy and details in an image or the level of difficulty to describe the image. It is an important concept in a variety of areas such as cognitive psychology, computer vision and visualization, and advertisement. Yet, efforts to create large, downloadable image datasets with diverse content and unbiased groundtruthing are lacking. In this work, we introduce Savoias, a visual complexity dataset that compromises of more than 1,400 images from seven image categories relevant to the above research areas, namely Scenes, Advertisements, Visualization and infographics, Objects, Interior design, Art, and Suprematism. The images in each category portray diverse characteristics including various low-level and high-level features, objects, backgrounds, textures and patterns, text, and graphics. The ground truth for Savoias is obtained by crowdsourcing more than 37,000 pairwise comparisons of images using the forced-choice methodology and with more than 1,600 contributors. The resulting relative scores are then converted to absolute visual complexity scores using the Bradley-Terry method and matrix completion. When applying five state-of-the-art algorithms to analyze the visual complexity of the images in the Savoias dataset, we found that the scores obtained from these baseline tools only correlate well with crowdsourced labels for abstract patterns in the Suprematism category (Pearson correlation r=0.84). For the other categories, in particular, the objects and advertisement categories, low correlation coefficients were revealed (r=0.3 and 0.56, respectively). These findings suggest that (1) state-of-the-art approaches are mostly insufficient and (2) Savoias enables category-specific method development, which is likely to improve the impact of visual complexity analysis on specific application areas, including computer vision.Comment: 10 pages, 4 figures, 4 table

Multivariate relationships among visual perceptual attributes and gross motor tasks with different environmental demands

This descriptive study investigates the underlying factors and interrelationships among five visual perceptual attributes and two performance stages of two gross motor tasks with different spatial/ temporal environmental demands. The Spatial Motor Task is a modification of the Scott Motor Ability Obstacle Race (Scott, 1943). The Spatial/temporal Motor Task is a modification of the Crawford Soccer Test Battery (Crawford, 1957). The basic difference in task demands is the moving ball in the latter task. Early and later performance stages are determined by averaging the first three scores on the first day and averaging the best three scores on the second day. The visual perceptual variables, selected on the basis of their role in performance of gross motor tasks as previously reported in research, are Coincidence Anticipation, Field Dependence/independence, Perceptual Speed, Peripheral Range, and Spatial Relations. Data were collected over a three week period of time during the spring 1977 semester. All assessments are made on a carefully scheduled basis by trained administrators. Eighty randomly selected undergraduate women enrolled in the general college physical education classes serve as subjects

Revealing doctors' prescribing choice dimensions with multivariate tools: A perceptual mapping approach

This article explores the use of multivariate techniques to build perceptual maps that show the perceived competitive positions of a set of ethical drugs. The data were drawn from a commercial panel of 283 family doctors working for the UK National Health Service. Factor analysis was applied to identify the main dimensions used by doctors to perceive and evaluate antihypertensive brands. Perceptual maps showing the competitive positions of 14 antihypertensives were produced using the factor scores of the medicines. Multiple regression analysis enabled the relative importance of each perceptual dimension to be determined. The results reveal that doctors use a small number of perceptual dimensions to evaluate competitive antihypertensive drugs, namely: 'medical support', 'long term efficacy', and the 'additional beneficial effects' provided by the brand. The implications for marketing managers in the ethical pharmaceutical industry are explained and ways in which the perceptual analysis can be used to guide strategic marketing decisions considered

A model of perceived dynamic range for HDR images

For High Dynamic Range (HDR) content, the dynamic range of an image is an important characteristic in algorithm design and validation, analysis of aesthetic attributes and content selection. Traditionally, it has been computed as the ratio between the maximum and minimum pixel luminance, a purely objective measure; however, the human visual system's perception of dynamic range is more complex and has been largely neglected in the literature. In this paper, a new methodology for measuring perceived dynamic range (PDR) of chromatic and achromatic HDR images is proposed. PDR can benefit HDR in a number of ways: for evaluating inverse tone mapping operators and HDR compression methods; aesthetically; or as a parameter for content selection in perceptual studies. A subjective study was conducted on a data set of 36 chromatic and achromatic HDR images. Results showed a strong agreement across participants' allocated scores. In addition, a high correlation between ratings of the chromatic and achromatic stimuli was found. Based on the results from a pilot study, five objective measures (pixel-based dynamic range, image key, area of bright regions, contrast and colorfulness) were selected as candidates for a PDR predictor model; two of which have been found to be significant contributors to the model. Our analyses show that this model performs better than individual metrics for both achromatic and chromatic stimuli