Quantifying Aspect Bias in Ordinal Ratings using a Bayesian Approach

User opinions expressed in the form of ratings can influence an individual's view of an item. However, the true quality of an item is often obfuscated by user biases, and it is not obvious from the observed ratings the importance different users place on different aspects of an item. We propose a probabilistic modeling of the observed aspect ratings to infer (i) each user's aspect bias and (ii) latent intrinsic quality of an item. We model multi-aspect ratings as ordered discrete data and encode the dependency between different aspects by using a latent Gaussian structure. We handle the Gaussian-Categorical non-conjugacy using a stick-breaking formulation coupled with P\'{o}lya-Gamma auxiliary variable augmentation for a simple, fully Bayesian inference. On two real world datasets, we demonstrate the predictive ability of our model and its effectiveness in learning explainable user biases to provide insights towards a more reliable product quality estimation.Comment: Accepted for publication in IJCAI 201

Using Response Times for Joint Modeling of Careless Responding and Attentive Response Styles

Questionnaires are by far the most common tool for measuring noncognitive constructs in psychology and educational sciences. Response bias may pose an additional source of variation between respondents that threatens validity of conclusions drawn from questionnaire data. We present a mixture modeling approach that leverages response time data from computer-administered questionnaires for the joint identification and modeling of two commonly encountered response bias that, so far, have only been modeled separately—careless and insufficient effort responding and response styles (RS) in attentive answering. Using empirical data from the Programme for International Student Assessment 2015 background questionnaire and the case of extreme RS as an example, we illustrate how the proposed approach supports gaining a more nuanced understanding of response behavior as well as how neglecting either type of response bias may impact conclusions on respondents’ content trait levels as well as on their displayed response behavior. We further contrast the proposed approach against a more heuristic two-step procedure that first eliminates presumed careless respondents from the data and subsequently applies model-based approaches accommodating RS. To investigate the trustworthiness of results obtained in the empirical application, we conduct a parameter recovery study

Approximate inference in graphical models

Probability theory provides a mathematically rigorous yet conceptually flexible calculus of uncertainty, allowing the construction of complex hierarchical models for real-world inference tasks. Unfortunately, exact inference in probabilistic models is often computationally expensive or even intractable. A close inspection in such situations often reveals that computational bottlenecks are confined to certain aspects of the model, which can be circumvented by approximations without having to sacrifice the model's interesting aspects. The conceptual framework of graphical models provides an elegant means of representing probabilistic models and deriving both exact and approximate inference algorithms in terms of local computations. This makes graphical models an ideal aid in the development of generalizable approximations. This thesis contains a brief introduction to approximate inference in graphical models (Chapter 2), followed by three extensive case studies in which approximate inference algorithms are developed for challenging applied inference problems. Chapter 3 derives the first probabilistic game tree search algorithm. Chapter 4 provides a novel expressive model for inference in psychometric questionnaires. Chapter 5 develops a model for the topics of large corpora of text documents, conditional on document metadata, with a focus on computational speed. In each case, graphical models help in two important ways: They first provide important structural insight into the problem; and then suggest practical approximations to the exact probabilistic solution.This work was supported by a scholarship from Microsoft Research, Ltd

A Bayesian nonparametric approach to dynamic item-response modeling: An application to the GUSTO cohort study

Statistical analysis of questionnaire data is often performed employing techniques from item-response theory. In this framework, it is possible to differentiate respondent profiles and characterize the questions (items) included in the questionnaire via interpretable parameters. These models are often crosssectional and aim at evaluating the performance of the respondents. The motivating application of this work is the analysis of psychometric questionnaires taken by a group of mothers at different time points and by their children at one later time point. The data are available through the GUSTO cohort study. To this end, we propose a Bayesian semiparametric model and extend the current literature by: (i) introducing temporal dependence among questionnaires taken at different time points; (ii) jointly modeling the responses to questionnaires taken from different, but related, groups of subjects (in our case mothers and children), introducing a further dependency structure and therefore sharing of information; (iii) allowing clustering of subjects based on their latent response profile. The proposed model is able to identify three main groups of mother/child pairs characterized by their response profiles. Furthermore, we report an interesting maternal reporting bias effect strongly affecting the clustering structure of the mother/child dyads

"What drives commuter behaviour?": A Bayesian clustering approach for understanding opposing behaviours in social surveys

This is the final version. Available on open access from Wiley via the DOI in this recordThe city of Exeter, UK, is experiencing unprecedented growth, putting pressure on traffic infrastructure. As well as traffic network management, understanding and influencing commuter behaviour is important for reducing congestion. Information about current commuter behaviour has been gathered through a large online survey, and similar individuals have been grouped to explore distinct behaviour profiles to inform intervention design to reduce commuter congestion. Statistical analysis within societal applications benefit from incorporating available social scientist expert knowledge. Current clustering approaches for the analysis of social surveys assume the number of groups and the within group narratives to be unknown a priori. Here, however, informed by valuable expert knowledge, we develop a novel Bayesian approach for creating a clear opposing transport mode group narrative within survey respondents, simplifying communication with project partners and the general public. Our methodology establishes groups characterising opposing behaviours based on a key multinomial survey question by constraining parts of our prior judgement within a Abbreviations: EST, Engaged Smart Transport; GI, Group Identifier; BI, Behavioural Influencer; MV, Motor Vehicle; PT, Public Transport; Markov chain Monte Carlo, MCMC. 1 2 DAWKINS ET AL. Bayesian finite mixture model. Drivers of group membership and within-group behavioural differences are modelled hierarchically using further information from the survey. In applying the methodology we demonstrate how it can be used to understand the key drivers of opposing behaviours in any wider application

An improved stochastic EM algorithm for large-scale full-information item factor analysis

In this paper, we explore the use of the stochastic EM algorithm (Celeux & Diebolt, 1985) for large-scale full-information item factor analysis. Innovations have been made on its implementation, including (1) an adaptive-rejection-based Gibbs sampler for the stochastic E step, (2) a proximal gradient descent algorithm for the optimization in the M step, and (3) diagnostic procedures for determining the burn-in size and the stopping of the algorithm. These developments are based on the theoretical results of Nielsen (2000), as well as advanced sampling and optimization techniques. The proposed algorithm is computationally efficient and virtually tuning-free, making it scalable to large-scale data with many latent traits (e.g. more than five latent traits) and easy to use for practitioners. Standard errors of parameter estimation are also obtained based on the missing information identity (Louis, 1982). The performance of the algorithm is evaluated through simulation studies and an application to the analysis of the IPIP-NEO personality inventory. Extensions of the proposed algorithm to other latent variable models are discussed

Para além de pressupostos psicométricos : como desenvolver novas medidas psicológicas

Tese (doutorado)—Universidade de Brasília, Instituto de Psicologia, Programa de Pós-graduação em Psicologia Social, do Trabalho e das Organizações, 2019.O que define uma boa medida? Na presente tese, argumentamos e mostramos que definir uma boa medida pode ser muito mais complexo do que simplesmente executar uma análise fatorial ou uma análise usando a teoria da resposta ao item. O objetivo geral desta dissertação é apresentar três principais pressupostos da medida psicométrica e desenvolver alternativas para a medida psicológica tradicional. A tese está dividida em quatro estudos. O primeiro é um estudo teórico no qual são apresentados três pressupostos centrais comuns à teoria psicométrica e à prática psicométrica, e no qual é mostrado como alternativas às abordagens psicométricas tradicionais podem ser usadas para melhorar a medição psicológica. Essas alternativas foram desenvolvidas adaptando cada um desses três pressupostos: (1) o pressuposto de validade estrutural; (2) o pressuposto do processo; e (3) o pressuposto de construto. O pressuposto de validade estrutural refere-se à implementação de modelos matemáticos. O pressuposto de processo implica que um processo subjacente específico está gerando os dados observados. O pressuposto de construto infere que os dados observados por si só não constituem uma medida, mas que as medidas são as variáveis latentes que originam os dados observados. Vários exemplos de abordagens psicométricas alternativas já existentes são apresentados no primeiro estudo. O segundo estudo se refere ao pressuposto de validade estrutural e teve como objetivo desenvolver dois novos modelos de resposta aos itens para itens politômicos e binários que não assumem uma distribuição normal dos escores verdadeiros. O primeiro modelo desenvolvido, o Modelo de resposta ao item condicional (CIRM), assume uma distribuição beta- binomial. O segundo novo modelo é uma implementação Bayesiana do procedimento de escore ótimo (OS-IRM). Ambos os novos modelos foram comparados com o modelo tradicional de Rasch: os resultados indicam que os dois modelos desenvolvidos melhoram vários aspectos do modelo de Rasch. O terceiro estudo foi derivado do pressuposto do processo e tinha três objetivos. Primeiro, desenvolver uma implementação Bayesiana do framework de análise da função de otimização situacional (SOFA). Segundo, comparar essa implementação Bayesiana do SOFA com outros três modelos baseados em Máxima Verossimilhança, usados para estimar escores verdadeiros. O terceiro objetivo foi mostrar como a modelagem conjunta pode ser usada para pesquisas de validade. Uma das principais vantagens do framework SOFA em comparação com a abordagem psicométrica tradicional é que o SOFA depende de dados experimentais, melhorando a validade das medidas. O quarto e último estudo foi derivado do pressuposto de construto e seu principal objetivo era desenvolver um procedimento de aprendizado de estrutura de gráficos de cadeia de potência (PCGs). Um PCG é um tipo de gráfico que representa relações causais entre grupos de variáveis. Pode ser pensado como uma versão exploratória completa da modelagem de equações estruturais, bem como um modelo psicométrico que não depende de variáveis latentes. Esses quatro estudos pretendem mostrar que a modelagem psicométrica não deve se restringir ao uso de modelos tradicionais de mensuração, mas também deve considerar a adaptação desses modelos tradicionais de acordo com o uso pretendido e os processos teóricos que originam as medidas observadas.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).What defines a good measurement? In the present dissertation we argue, and show, that defining a good measurement can be much more complex than simply performing a factor analysis or an analysis using item response theory. The overall objective of this dissertation is to present three principal assumptions of psychometric measurement, and to develop alternatives for traditional psychological measurement. The dissertation is divided in four studies. The first one is a theoretical study in which three central assumptions common to psychometric theory and psychometric practice are presented, and in which is shown how alternatives to traditional psychometric approaches can be used to improve psychological measurement. These alternatives were developed by adapting each of these three assumptions: (1) the assumption of structural validity; (2) the process assumption; and, (3) the construct assumption. The structural validity assumption relates to the implementation of mathematical models. The process assumption implies that a specific underlying process is generating the observed data. The construct assumption infers that the observed data on its own do not constitute a measurement, but the measure are the latent variables that originate the observed data. Several examples of already existing alternative psychometric approaches are presented in the first study. The second study relates to the structural validity assumption and aimed to develop two new item response models for polytomous and binary items that do not assume a normal distribution of the true scores. The first model that was developed, the Conditional Item Response Model (CIRM), assumes a beta-binomial distribution. The second new model is a Bayesian implementation of the optimal score procedure (OS-IRM). Both new models were compared with the traditional Rasch model: the results indicate that the two developed models improve various aspects of the Rasch model. The third study was derived from the process assumption and had three objectives. First, to develop a Bayesian implementation of the situational optimization function analysis (SOFA) framework. Second, to compare this Bayesian implementation of SOFA with three other Maximum Likelihood-based models that are used to estimate true scores. The third objective was to show how joint modeling can be used for validity research. One of the main advantages of the SOFA framework compared to the traditional psychometric approach is that SOFA relies on experimental data, improving the validity of the measures. The fourth and final study was derived from the construct assumption and its main objective was to develop a procedure of structure learning of power chain graphs (PCGs). A PCG is a type of graph that represents causal relations between groups of variables. It can be thought as a full exploratory version of structural equation modeling, as well as a psychometric model that is not dependent on latent variables. These four studies intend to show that psychometric modeling should not be restricted to the use of traditional measurement models, but should also consider adapting these traditional models in accordance with the intended use and theoretical processes that originate the observed measures

When do people prefer dominant over prestigious political leaders?

This is the final version. Available on open access from Cambridge University Press via the DOI in this recordData availability: Datasets and data analyses are available from https://github.com/AngelVJimenez/Preferences_Dominant_Prestigious_LeadersPrevious research has sought to explain the rise of right-wing populist leaders in terms of the evolutionary framework of dominance and prestige. In this framework, dominance is defined as high social rank acquired via coercion and fear, and prestige is defined as high social rank acquired via competence and admiration. Previous studies have shown that right-wing populist leaders are rated as more dominant than non-populist leaders, and right-wing populist/dominant leaders are favoured in times of economic uncertainty and intergroup conflict. In this paper, we explore and critique this application of dominance–prestige to politics. First, we argue that the dominance–prestige framework, originally developed to explain inter-personal relationships within small-scale societies characterised by face-to-face interaction, does not straightforwardly extend to large-scale democratic societies which have frequent anonymous interaction and complex ingroup–outgroup dynamics. Second, we show that economic uncertainty and intergroup conflict predict a preference not only for dominant leaders, but also for prestigious leaders. Third, we show that perceptions of leaders as dominant or prestigious are not fixed, and depend on the political ideology of the perceiver: people view leaders who share their ideology as prestigious, and those who oppose their ideology as dominant, whether that ideology is liberal or conservative. Fourth, we show that political ideology is a stronger predictor than economic uncertainty of preference for Donald Trump vs Hillary Clinton in the 2016 US Presidential Election, contradicting previous findings that link Trump's success to economic uncertainty. We conclude by suggesting that, if economic uncertainty does not directly affect preferences for right-wing populist leaders, other features of their discourse such as higher emotionality might explain their success.Leverhulme Trus

Modeling random and non-random decision uncertainty in ratings data: A fuzzy beta model

Modeling human ratings data subject to raters' decision uncertainty is an attractive problem in applied statistics. In view of the complex interplay between emotion and decision making in rating processes, final raters' choices seldom reflect the true underlying raters' responses. Rather, they are imprecisely observed in the sense that they are subject to a non-random component of uncertainty, namely the decision uncertainty. The purpose of this article is to illustrate a statistical approach to analyse ratings data which integrates both random and non-random components of the rating process. In particular, beta fuzzy numbers are used to model raters' non-random decision uncertainty and a variable dispersion beta linear model is instead adopted to model the random counterpart of rating responses. The main idea is to quantify characteristics of latent and non-fuzzy rating responses by means of random observations subject to fuzziness. To do so, a fuzzy version of the Expectation-Maximization algorithm is adopted to both estimate model's parameters and compute their standard errors. Finally, the characteristics of the proposed fuzzy beta model are investigated by means of a simulation study as well as two case studies from behavioral and social contexts.Comment: 24 pages, 0 figures, 5 table

Lidské vnímání v situaci nejistoty: Vizuální, auditivní a vtělené reakce na nejednoznačné stimuly.

Naše smysly se vyvinuly tak, abychom z okolního prostředí získávat optimální množství informací. Tato optimalizace ovšem znamená, že je třeba počítat s chybami. Proto, abychom předešli těm s významným dopadem, vyvinula se u člověka tendence k nadhodnocování významu vzájemných souvislostí (i ve smyslu vnímání vzorů a posloupností). Ve své práci jsem testovala schopnost vyhodnocování vizuálních a akustických stimulů. Za použití počítačové grafiky byl vyvinut soubor testovacích stimulů, kde bylo rozložení prvků určeno sofistikovaným generátorem pseudo-náhodných čísel. Tyto výsledné masky s různou mírou průhlednosti byly užity k překrytí geometrických tvarů. Podobného postupu bylo užito k vytvoření černobílých stimulů s vysokým kontrastem. Za použití metod bayesovské statistiky jsem nalezla vzájemnou provázanost schopnosti určit přítomnost vzoru (a její absenci) a stylu myšlení, specificky racionálního a na intuici založeného. Dále jsem pak použila nejednoznačné výrazy tváře a vokalizace vysoce intenzivních afektivních stavů (bolest a slast) a stavů nízké intenzity (neutrální výraz/promluva, úsměv/smích). Výsledkem je zjištění, že vysoká intenzita projevu je spojena s nízkou schopností respondentů správně vyhodnotit valenci vizuálních i akustických stimulů. Díky použitému statistickému přístupu jsem...In order to orient ourselves in the environment our senses have evolved so as to acquire optimal information. The optimization, however, incurs mistakes. To avoid costly ones, the over-perception of patterns (in humans) augments the decision making. I tested the decision- making in two modalities, acoustic and visual. A set of stimuli (using computer-generated graphics, based on output from a very good pseudo random generator) was produced: masks with a random pattern with varying degree of transparency over geometrical figures were used, followed by similar task that involved black and white high-contrast patterns. In both cases, I was able to find, using a Bayesian statistical approach, that the ability to detect the correct pattern presence (or lack thereof) was related to respondents' thinking styles, specifically Rationality and Intuition. Furthermore, I used ambiguous facial expressions, and accompanying vocalizations, of high-intensity affects (pain, pleasure and fear) and low- intensity (neutral and smile/laughter). My findings evidenced that the high-intensity facial expressions and vocalizations were rated with a low probability of correct response. Differences in the consistency of the ratings were detected and also the range of probabilities of being due to chance (guessing). When...Katedra filosofie a dějin přírodních vědDepartment of Philosophy and History of SciencePřírodovědecká fakultaFaculty of Scienc