5 research outputs found
A practical guide and software for analysing pairwise comparison experiments
Most popular strategies to capture subjective judgments from humans involve
the construction of a unidimensional relative measurement scale, representing
order preferences or judgments about a set of objects or conditions. This
information is generally captured by means of direct scoring, either in the
form of a Likert or cardinal scale, or by comparative judgments in pairs or
sets. In this sense, the use of pairwise comparisons is becoming increasingly
popular because of the simplicity of this experimental procedure. However, this
strategy requires non-trivial data analysis to aggregate the comparison ranks
into a quality scale and analyse the results, in order to take full advantage
of the collected data. This paper explains the process of translating pairwise
comparison data into a measurement scale, discusses the benefits and
limitations of such scaling methods and introduces a publicly available
software in Matlab. We improve on existing scaling methods by introducing
outlier analysis, providing methods for computing confidence intervals and
statistical testing and introducing a prior, which reduces estimation error
when the number of observers is low. Most of our examples focus on image
quality assessment.Comment: Code available at https://github.com/mantiuk/pwcm
Fully-automatic inverse tone mapping algorithm based on dynamic mid-level tone mapping
High Dynamic Range (HDR) displays can show images with higher color contrast levels and peak luminosities than the common Low Dynamic Range (LDR) displays. However, most existing video content is recorded and/or graded in LDR format. To show LDR content on HDR displays, it needs to be up-scaled using a so-called inverse tone mapping algorithm. Several techniques for inverse tone mapping have been proposed in the last years, going from simple approaches based on global and local operators to more advanced algorithms such as neural networks. Some of the drawbacks of existing techniques for inverse tone mapping are the need for human intervention, the high computation time for more advanced algorithms, limited low peak brightness, and the lack of the preservation of the artistic intentions. In this paper, we propose a fully-automatic inverse tone mapping operator based on mid-level mapping capable of real-time video processing. Our proposed algorithm allows expanding LDR images into HDR images with peak brightness over 1000 nits, preserving the artistic intentions inherent to the HDR domain. We assessed our results using the full-reference objective quality metrics HDR-VDP-2.2 and DRIM, and carrying out a subjective pair-wise comparison experiment. We compared our results with those obtained with the most recent methods found in the literature. Experimental results demonstrate that our proposed method outperforms the current state-of-the-art of simple inverse tone mapping methods and its performance is similar to other more complex and time-consuming advanced techniques
Predicting Engagement in Video Lectures
The explosion of Open Educational Resources (OERs) in the recent years
creates the demand for scalable, automatic approaches to process and evaluate
OERs, with the end goal of identifying and recommending the most suitable
educational materials for learners. We focus on building models to find the
characteristics and features involved in context-agnostic engagement (i.e.
population-based), a seldom researched topic compared to other contextualised
and personalised approaches that focus more on individual learner engagement.
Learner engagement, is arguably a more reliable measure than popularity/number
of views, is more abundant than user ratings and has also been shown to be a
crucial component in achieving learning outcomes. In this work, we explore the
idea of building a predictive model for population-based engagement in
education. We introduce a novel, large dataset of video lectures for predicting
context-agnostic engagement and propose both cross-modal and modality-specific
feature sets to achieve this task. We further test different strategies for
quantifying learner engagement signals. We demonstrate the use of our approach
in the case of data scarcity. Additionally, we perform a sensitivity analysis
of the best performing model, which shows promising performance and can be
easily integrated into an educational recommender system for OERs.Comment: In Proceedings of International Conference on Educational Data Mining
202
Psychometric scaling of TID2013 dataset
TID2013 is a subjective image quality assessment dataset with a wide range of distortion types and over 3000 images. The dataset has proven to be a challenging test for objective quality metrics. The dataset mean opinion scores were obtained by collecting pairwise comparison judgments using the Swiss tournament system, and averaging votes of observers. However, this approach differs from the usual analysis of multiple pairwise comparisons, which involves psychometric scaling of the comparison data using either Thurstone or Bradley-Terry models. In this paper we investigate how quality scores change when they are computed using such psychometric scaling instead of averaging vote counts. In order to properly scale TID2013 quality scores, we conduct four additional experiments of two different types, which we found necessary to produce a common quality scale: comparisons with reference images, and cross-content comparisons. We demonstrate on a fifth validation experiment that the two additional types of comparisons are necessary and in conjunction with psychometric scaling improve the consistency of quality scores, especially across images depicting different contents
Factors to Consider for Tailored Gamification
International audienceGamification is widely used to foster user motivation. Recent studies show that users can be more or less receptive to different game elements, based on their personality or player profile. Consequently, recent work on tailored gamification tries to identify links between user types and motivating game elements. However findings are very heterogeneous due to different contexts, different typologies to characterize users, and different implementations of game elements. Our work seeks to obtain more generalizable findings in order to identify the main factors that will support design choices when tailoring gamification to users' profiles and provide designers with concrete recommendations for designing tailored gamification systems. For this purpose, we ran a crowdsourced study with 300 participants to identify the motivational impact of game elements. Our study differs from previous work in three ways: first, it is independent from a specific user activity and domain; second, it considers three user typologies; and third, it clearly distinguishes motivational strategies and their implementation using multiple different game elements. Our results reveal that (1) different implementations of a same motivational strategy have different impacts on motivation, (2) dominant user type is not sufficient to differentiate users according to their preferences for game elements, (3) Hexad is the most appropriate user typology for tailored gamification and (4) the motiva-tional impact of certain game elements varies with the user activity or the domain of gamified systems