You Take the High Road, and I’ll Take the Low Road:Evaluating the Topographical Consistency of Cognitive Models

We present a novel framework for assessing the fit of cogni-tive models. Using this framework, we highlight limitationsof existing methods of model evaluation, and derive new ap-proaches to validating cognitive models. Tests of topographi-cal consistency emphasize how a model’s structure constrainsbehavior on pairs of coupled stimuli, even when point predic-tions on individual stimuli depend on estimates of the model’sfree parameters. By carefully selecting these coupled stim-uli such that they follow the distinct topography of the model,researchers can overcome some limitations of existing meth-ods. Finally, we provide a proof-of-concept example of how touse our approach to assess a model of multi-alternative, multi-attribute choice

The Fundamental Dilemma of Bayesian Active Meta-learning

Many applications involve estimation of parameters that generalize across multiple diverse, but related, data-scarce task environments. Bayesian active meta-learning, a form of sequential optimal experimental design, provides a framework for solving such problems. The active meta-learner's goal is to gain transferable knowledge (estimate the transferable parameters) in the presence of idiosyncratic characteristics of the current task (task-specific parameters). We show that in such a setting, greedy pursuit of this goal can actually hurt estimation of the transferable parameters (induce so-called negative transfer). The learner faces a dilemma akin to but distinct from the exploration--exploitation dilemma: should they spend their acquisition budget pursuing transferable knowledge, or identifying the current task-specific parameters? We show theoretically that some tasks pose an inevitable and arbitrarily large threat of negative transfer, and that task identification is critical to reducing this threat. Our results generalize to analysis of prior misspecification over nuisance parameters. Finally, we empirically illustrate circumstances that lead to negative transfer

One system for learning and remembering episodes and rules

Humans can learn individual episodes and generalizable rules and also successfully retain both kinds of acquired knowledge over time. In the cognitive science literature, (1) learning individual episodes and rules and (2) learning and remembering are often both conceptualized as competing processes that necessitate separate, complementary learning systems. Inspired by recent research in statistical learning, we challenge these trade-offs, hypothesizing that they arise from capacity limitations rather than from the inherent incompatibility of the underlying cognitive processes. Using an associative learning task, we show that one system with excess representational capacity can learn and remember both episodes and rules

A Social Interpolation model of group problem-solving

Code and materials to replicate analyses reported in the manuscript "A Social Interpolation model of group problem-solving

A Comparison of Methods for Adaptive Experimentation

We use a simulation study to compare three methods for adaptive experimentation: Thompson sampling, Tempered Thompson sampling, and Exploration sampling. We gauge the performance of each in terms of social welfare and estimation accuracy, and as a function of the number of experimental waves. We further construct a set of novel "hybrid" loss measures to identify which methods are optimal for researchers pursuing a combination of experimental aims. Our main results are: 1) the relative performance of Thompson sampling depends on the number of experimental waves, 2) Tempered Thompson sampling uniquely distributes losses across multiple experimental aims, and 3) in most cases, Exploration sampling performs similarly to random assignment

Think of the Consequences: A Decade of Discourse about Same-Sex Marriage

Approaching issues through the lens of non-negotiable values increases the perceived intractability of debate (Baron & Spranca, 1997), while focusing on concrete consequences of policies instead results in the moderation of extreme opinions (Fernbach et al., 2013) and greater likelihood of conflict resolution (Baron & Leshner, 2000). Using comments on the popular social media platform Reddit from January 2006 until September 2017, we show how changes in the framing of same-sex marriage in public discourse relate to changes in public opinion. We use a topic model to show that the contribution of certain protected-values-based topics to the debate (religious arguments and freedom of opinion) increased prior to the emergence of a public consensus in support of same-sex marriage (Gallup, 2017), and declined afterwards. In contrast, discussion of certain consequentialist topics (the impact of politicians’ stance and same-sex marriage as a matter of policy) showed the opposite pattern. Our results reinforce the meaningfulness of protected values and consequentialism as relevant dimensions for describing public discourse and highlight the usefulness of unsupervised machine learning methods in tackling questions about social attitude change