Justify your alpha

Benjamin et al. proposed changing the conventional “statistical significance” threshold (i.e.,the alpha level) from p ≤ .05 to p ≤ .005 for all novel claims with relatively low prior odds. They provided two arguments for why lowering the significance threshold would “immediately improve the reproducibility of scientific research.” First, a p-value near .05provides weak evidence for the alternative hypothesis. Second, under certain assumptions, an alpha of .05 leads to high false positive report probabilities (FPRP2 ; the probability that a significant finding is a false positive

Justify your alpha

In response to recommendations to redefine statistical significance to p ≤ .005, we propose that researchers should transparently report and justify all choices they make when designing a study, including the alpha level

Complexity-Theoretic Limits on the Promises of Artificial Neural Network Reverse-Engineering

Emerging folklore in the cognitive sciences suggests that interpretability techniques to reverse-engineer artificial neural networks (ANNs) could speed up discovery and theory-building. For many researchers in psychology, linguistics, neuroscience, and artificial intelligence (AI), the full observability and perturbability of ANNs trained on complex tasks affords a shortcut to domain insights, cognitive theories, neurocognitive models, application improvement, and user safety. Folklore intuitions, however, are typically disconnected from other relevant knowledge. Here we examine these intuitions formally by drawing relevant connections to computational complexity theory. We model interpretability queries computationally and analyze their resource demands for biological/artificial high-level cognition. We prove mathematically that, contrary to folklore, basic circuit-finding queries in classic ANNs are already infeasibly demanding to answer even approximately. We discuss how interdisciplinary integration can mitigate this disconnect and situate the broader implications for the cognitive sciences, the philosophy of AI-fueled discovery, and AI ethics

Functional shifts in L2

Data for "Reading Shakespearean tropes in a foreign tongue: Age of L2 acquisition modulates neural responses to functional shifts

Clarifying status of DNNs as models of human vision

On several key issues we agree with the commentators. Perhaps most importantly, everyone seems to agree that psychology has an important role to play in building better models of human vision, and (most) everyone agrees (including us) that DNNs will play an important role in modelling human vision going forward. But there are also disagreements about what models are for, how DNN-human correspondences should be evaluated, the value of alternative modelling approaches, and impact of marketing hype in the literature. In our view, these latter issues are contributing to many unjustified claims regarding DNN-human correspondences in vision and other domains of cognition. We explore all these issues in this response

Reading Shakespearean tropes in a foreign tongue: Age of L2 acquisition modulates neural responses to functional shifts

Functional shifts (FSs) – morphosyntactically marked words evoking coherent but novel meanings – are ubiquitous in English and, specially, in Shakespearean literature. While their neural signatures have been explored in native speakers, no study has targeted foreign-language users, let alone comparing early and late bilinguals. Here, we administered a validated FS paradigm to subjects from both populations and evaluated time-frequency modulations evoked by FS and control sentences. Early bilinguals exhibited greater sensitivity towards FSs, indexed by reduced fronto-posterior theta-band oscillations across semantic- and structural-integration windows. Such oscillatory modulations may represent a key marker of age-of-acquisition effects during foreign-language wordplay processing.Fil: Vilas, Martina G.. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; ArgentinaFil: Santilli, Micaela. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; ArgentinaFil: Mikulan, Ezequiel Pablo. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; ArgentinaFil: Gonzalez Adolfi, Federico. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; ArgentinaFil: Martorell Caro, Miguel Angel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; ArgentinaFil: Manes, Facundo Francisco. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; ArgentinaFil: Herrera, Eduar. Universidad ICESI, ; ColombiaFil: Sedeño, Lucas. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; ArgentinaFil: Ibañez, Agustin Mariano. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; Argentina. Universidad Autónoma del Caribe; Colombia. Universidad Adolfo Ibañez; ChileFil: García, Adolfo Martín. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Instituto de Neurología Cognitiva. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt | Fundación Favaloro. Instituto de Neurociencia Cognitiva y Traslacional. Fundación Ineco Rosario Sede del Incyt; Argentin

Deep Problems with Neural Network Models of Human Vision

Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images of objects and are often described as the best models of biological vision. This conclusion is largely based on three sets of findings: (1) DNNs are more accurate than any other model in classifying images taken from various datasets, (2) DNNs do the best job in predicting the pattern of human errors in classifying objects taken from various behavioral benchmark datasets, and (3) DNNs do the best job in predicting brain signals in response to images taken from various brain benchmark datasets (e.g., single cell responses or fMRI data). However, most behavioral and brain benchmarks report the outcomes of observational experiments that do not manipulate any independent variables, and we show that the good prediction on these datasets may be mediated by DNNs that share little overlap with biological vision. More problematically, we show that DNNs account for almost no results from psychological research. This contradicts the common claim that DNNs are good, let alone the best, models of human object recognition. We argue that theorists interested in developing biologically plausible models of human vision need to direct their attention to explaining psychological findings. More generally, theorists need to build models that explain the results of experiments that manipulate independent variables designed to test hypotheses rather than compete on predicting observational data. We conclude by briefly summarizing various promising modelling approaches that focus on psychological data

COBIDAS checklist

Using a checklist to help with more thorough and transparent methods and result reporting in neuroimaging

Justify your alpha

No description supplie