From pairwise comparisons and rating to a unified quality scale.

The goal of psychometric scaling is the quantification of perceptual experiences, understanding the relationship between an external stimulus, the internal representation and the response. In this paper, we propose a probabilistic framework to fuse the outcome of different psychophysical experimental protocols, namely rating and pairwise comparisons experiments. Such a method can be used for merging existing datasets of subjective nature and for experiments in which both measurements are collected. We analyze and compare the outcomes of both types of experimental protocols in terms of time and accuracy in a set of simulations and experiments with benchmark and real-world image quality assessment datasets, showing the necessity of scaling and the advantages of each protocol and mixing. Although most of our examples focus on image quality assessment, our findings generalize to any other subjective quality-of-experience task.This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement n◦ 725253–EyeCode), from EPSRC research grant EP/P007902/1 and from a Science Foundation Ireland (SFI) research grant under the Grant Number 15/RP/2776. Marıa Pérez-Ortiz did part of this work while at the University of Cambridge and University College London (under MURI grant EPSRC 542892)

Active sampling, scaling and dataset merging for large-scale image quality assessment

The field of subjective assessment is concerned with eliciting human judgements about a set of stimuli. Collecting such data is costly and time-consuming, especially when the subjective study is to be conducted in a controlled environment and using a specialized equipment. Thus, data from these studies are usually scarce. One of the areas, for which obtaining subjective measurements is difficult is image quality assessment. The results from these studies are used to develop and train automated or objective image quality metrics, which, with the advent of deep learning, require large amounts of versatile and heterogeneous data. I present three main contributions in this dissertation. First, I propose a new active sampling method for efficient collection of pairwise comparisons in subjective assessment experiments. In these experiments observers are asked to express a preference between two conditions. However, many pairwise comparison protocols require a large number of comparisons to infer accurate scores, which may be unfeasible when each comparison is time-consuming (e.g. videos) or expensive (e.g. medical imaging). This motivates the use of an active sampling algorithm that chooses only the most informative pairs for comparison. I demonstrate, with real and synthetic data, that my algorithm offers the highest accuracy of inferred scores given a fixed number of measurements compared to the existing methods. Second, I propose a probabilistic framework to fuse the outcomes of different psychophysical experimental protocols, namely rating and pairwise comparisons experiments. Such a method can be used for merging existing datasets of subjective nature and for experiments in which both measurements are collected. Third, with a new dataset merging technique and by collecting additional cross-dataset quality comparisons I create a Unified Photometric Image Quality (UPIQ) dataset with over 4,000 images by realigning and merging existing high-dynamic-range (HDR) and standard-dynamic-range (SDR) datasets. The realigned quality scores share the same unified quality scale across all datasets. I then use the new dataset to retrain existing HDR metrics and show that the dataset is sufficiently large for training deep architectures. I show the utility of the dataset and metrics in an application to image compression that accounts for viewing conditions, including screen brightness and the viewing distance

A practical guide and software for analysing pairwise comparison experiments

Most popular strategies to capture subjective judgments from humans involve the construction of a unidimensional relative measurement scale, representing order preferences or judgments about a set of objects or conditions. This information is generally captured by means of direct scoring, either in the form of a Likert or cardinal scale, or by comparative judgments in pairs or sets. In this sense, the use of pairwise comparisons is becoming increasingly popular because of the simplicity of this experimental procedure. However, this strategy requires non-trivial data analysis to aggregate the comparison ranks into a quality scale and analyse the results, in order to take full advantage of the collected data. This paper explains the process of translating pairwise comparison data into a measurement scale, discusses the benefits and limitations of such scaling methods and introduces a publicly available software in Matlab. We improve on existing scaling methods by introducing outlier analysis, providing methods for computing confidence intervals and statistical testing and introducing a prior, which reduces estimation error when the number of observers is low. Most of our examples focus on image quality assessment.Comment: Code available at https://github.com/mantiuk/pwcm

Effects of tapentadol on pain, motor symptoms and cognitive functions in Parkinson\u2019s disease

Background: Pain is a common and undertreated non-motor symptom in patients with Parkinson\u2019s disease (PD). Opioids have been seldom used in PD because they could worsen cognitive and motor functions. Objective: We aimed to assess efficacy and tolerability of tapentadol in PD patients. Methods: We retrospectively reviewed 21 PD patients treated with tapentadol extended release (ER) for chronic pain. Patients were evaluated before treatment and at 3 and 6 months during treatment for pain intensity (current, 24-hour average, and minimum and worst) with a 0-10 Numerical Rating Scale and the painDETECT questionnaire; for motor symptom severity with the Unified Parkinson\u2019s Disease Rating Scale part III and the Hoehn and Yahr scale; for cognitive functions with MiniMental Status Examination, Corsi\u2019s Block Tapping test, Digit Span, Digit-Symbol Substitution test, FAS test, Rey\u2019s Auditory Verbal Learning test, Trail Making-A and -B, and the 9 Hole-Peg Test; for anxiety and depression with the Hospital Anxiety and Depression Scale; and for the quality of life with the Short Form-12 for Quality of Life. Data were analyzed by one-way ANOVA and paired t-test, and by Friedman\u2019s and Wilcoxon\u2019s test. Statistical significance was taken in all cases as P < 0,05. Results: Pain intensity decreased over the course of treatment. No differences were found in PD symptom severity and dopaminergic drug dosages between pretreatment and treatment evaluations . No decrement in cognitive neuropsychological performances was found and an improvement was observed in Digit Span, Digit-Symbol Substitution test and FAS test. The levels of anxiety, depression and of quality of life improved. Overall tapentadol ER was well tolerated and most patients reported no or mild and short-lived gastroenterological and neurological side effects. Conclusions: These results indicate the potential efficacy and tolerability of medium-high dose of tapentadol ER for the treatment of pain in PD