Metacognitive awareness of associative learning: What underlies delayed judgments-of-learning?

Cognitive processes, such as memory, are accompanied by metacognitive states of awareness that allow for evaluation of their function. Across seven experiments we employed the delayed judgment-of-learning (JOL) paradigm with healthy young adults to examine metacognitive monitoring of learning. After studying cue-target word-pairs, participants were presented with the studied cues and predicted their ability to retrieve the target on a subsequent memory test. The key question of interest was the nature of the underlying processes guiding such judgments with a focus on how they relate to memory. The delayed JOL literature has assumed that it is an absolute judgment, based on the ease of access to the target item. Chapters 2 and 3 manipulated target- and cue-related variables and investigated their influence on memory and metamemory. The results showed delayed JOLs are also sensitive to memory for contextual information about the target (Chapter 2) and the level of familiarity with the cue term (Chapter 3). This is strengthened by results from Chapter 4 in which participants provided written justifications of their JOL responses without any experimental manipulations of the learned material. Analysis of these responses confirmed that both cue- and target-related information influences delayed JOLs. Lastly, we showed that delayed JOLs are not sensitive to whether they are predicting recognition or recall (so called theory-based influences) unless participants make a different prediction on each trial (i.e. trial-level design, Chapter 5). Overall, delayed JOLs are shown to vary with variables that fluctuate on a trial level, which can but do not necessarily need to map onto memory. The results suggest that delayed JOLs are primarily comparative judgments, involving the evaluation of the quantity and quality of evidence on any given trial in the context of the task at hand (e.g. by comparison to preceding trials). This is contrary to how it is often treated in the delayed JOL literature but is consistent with other metacognitive paradigms

Investigating the role of assessment method on reports of déjà vu and tip-of-the-tongue states during standard recognition tests

Déjà vu and tip-of-the-tongue (TOT) are retrieval-related subjective experiences whose study relies on participant self-report. In four experiments (ns = 224, 273, 123 and 154), we explored the effect of questioning method on reported occurrence of déjà vu and TOT in experimental settings. All participants carried out a continuous recognition task, which was not expected to induce déjà vu or TOT, but were asked about their experiences of these subjective states. When presented with contemporary definitions, between 32% and 58% of participants nonetheless reported experiencing déjà vu or TOT. Changing the definition of déjà vu or asking participants to bring to mind a real-life instance of déjà vu or TOT before completing the recognition task had no impact on reporting rates. However, there was an indication that changing the method of requesting subjective reports impacted reporting of both experiences. More specifically, moving from the commonly used retrospective questioning (e.g. “Have you experienced déjà vu?”) to free report instructions (e.g. “Indicate whenever you experience déjà vu.”) reduced the total number of reported déjà vu and TOT occurrences. We suggest that research on subjective experiences should move toward free report assessments. Such a shift would potentially reduce the presence of false alarms in experimental work, thereby reducing the overestimation of subjective experiences prevalent in this area of research.Publisher PDFPeer reviewe

What’s new in déjà vu?

Publisher PD

Bayesian imputation of COVID-19 positive test counts for nowcasting under reporting lag

Obtaining up to date information on the number of UK COVID-19 regional infections is hampered by the reporting lag in positive test results for people with COVID-19 symptoms. In the UK, for ‘Pillar 2’ swab tests for those showing symptoms, it can take up to five days for results to be collated. We make use of the stability of the under reporting process over time to motivate a statistical temporal model that infers the final total count given the partial count information as it arrives. We adopt a Bayesian approach that provides for subjective priors on parameters and a hierarchical structure for an underlying latent intensity process for the infection counts. This results in a smoothed time-series representation nowcasting the expected number of daily counts of positive tests with uncertainty bands that can be used to aid decision making. Inference is performed using sequential Monte Carlo

Bayesian imputation of COVID-19 positive test counts for nowcasting under reporting lag

Obtaining up to date information on the number of UK COVID-19 regional infections is hampered by the reporting lag in positive test results for people with COVID-19 symptoms. In the UK, for "Pillar 2" swab tests for those showing symptoms, it can take up to five days for results to be collated. We make use of the stability of the under reporting process over time to motivate a statistical temporal model that infers the final total count given the partial count information as it arrives. We adopt a Bayesian approach that provides for subjective priors on parameters and a hierarchical structure for an underlying latent intensity process for the infection counts. This results in a smoothed time-series representation now-casting the expected number of daily counts of positive tests with uncertainty bands that can be used to aid decision making. Inference is performed using sequential Monte Carlo

Understanding metacognitive confidence : insights from judgment-of-learning justifications

This research was supported by the Economic and Social Research Council Studentship awarded to Radka Jersakova [ES/J500215/1].This study employed the delayed judgment-of-learning (JOL) paradigm to investigate the content of metacognitive judgments; after studying cue-target word-pairs, participants predicted their ability to remember targets on a future memory test (cued recognition in Experiments 1 and 2 and cued recall in Experiment 3). In Experiment 1 and the confidence JOL group of Experiment 3, participants used a commonly employed 6-point numeric confidence JOL scale (0–20–40–60–80–100%). In Experiment 2 and the binary JOL group of Experiment 3 participants first made a binary yes/no JOL prediction followed by a 3-point verbal confidence judgment (sure-maybe-guess). In all experiments, on a subset of trials, participants gave a written justification of why they gave that specific JOL response. We used natural language processing techniques (latent semantic analysis and word frequency [n-gram] analysis) to characterize the content of the written justifications and to capture what types of evidence evaluation uniquely separate one JOL response type from others. We also used a machine learning classification algorithm (support vector machine [SVM]) to quantify the extent to which any two JOL responses differed from each other. We found that: (i) participants can justify and explain their JOLs; (ii) these justifications reference cue familiarity and target accessibility and so are particularly consistent with the two-stage metacognitive model; and (iii) JOL confidence judgements do not correspond to yes/no responses in the manner typically assumed within the literature (i.e. 0–40% interpreted as no predictions).PostprintPeer reviewe

Improving local prevalence estimates of SARS-CoV-2 infections using a causal debiasing framework.

Funder: Oxford University | Jesus College, University of OxfordFunder: Joint Biosecurity CentreGlobal and national surveillance of SARS-CoV-2 epidemiology is mostly based on targeted schemes focused on testing individuals with symptoms. These tested groups are often unrepresentative of the wider population and exhibit test positivity rates that are biased upwards compared with the true population prevalence. Such data are routinely used to infer infection prevalence and the effective reproduction number, Rt, which affects public health policy. Here, we describe a causal framework that provides debiased fine-scale spatiotemporal estimates by combining targeted test counts with data from a randomized surveillance study in the United Kingdom called REACT. Our probabilistic model includes a bias parameter that captures the increased probability of an infected individual being tested, relative to a non-infected individual, and transforms observed test counts to debiased estimates of the true underlying local prevalence and Rt. We validated our approach on held-out REACT data over a 7-month period. Furthermore, our local estimates of Rt are indicative of 1-week- and 2-week-ahead changes in SARS-CoV-2-positive case numbers. We also observed increases in estimated local prevalence and Rt that reflect the spread of the Alpha and Delta variants. Our results illustrate how randomized surveys can augment targeted testing to improve statistical accuracy in monitoring the spread of emerging and ongoing infectious disease

Improving local prevalence estimates of SARS-CoV-2 infections using a causal debiasing framework.

Funder: Oxford University | Jesus College, University of OxfordFunder: Joint Biosecurity CentreGlobal and national surveillance of SARS-CoV-2 epidemiology is mostly based on targeted schemes focused on testing individuals with symptoms. These tested groups are often unrepresentative of the wider population and exhibit test positivity rates that are biased upwards compared with the true population prevalence. Such data are routinely used to infer infection prevalence and the effective reproduction number, Rt, which affects public health policy. Here, we describe a causal framework that provides debiased fine-scale spatiotemporal estimates by combining targeted test counts with data from a randomized surveillance study in the United Kingdom called REACT. Our probabilistic model includes a bias parameter that captures the increased probability of an infected individual being tested, relative to a non-infected individual, and transforms observed test counts to debiased estimates of the true underlying local prevalence and Rt. We validated our approach on held-out REACT data over a 7-month period. Furthermore, our local estimates of Rt are indicative of 1-week- and 2-week-ahead changes in SARS-CoV-2-positive case numbers. We also observed increases in estimated local prevalence and Rt that reflect the spread of the Alpha and Delta variants. Our results illustrate how randomized surveys can augment targeted testing to improve statistical accuracy in monitoring the spread of emerging and ongoing infectious disease

Interoperability of Statistical Models in Pandemic Preparedness: Principles and Reality

We present "interoperability" as a guiding framework for statistical modelling to assist policy makers asking multiple questions using diverse datasets in the face of an evolving pandemic response. Interoperability provides an important set of principles for future pandemic preparedness, through the joint design and deployment of adaptable systems of statistical models for disease surveillance using probabilistic reasoning. We illustrate this through case studies for inferring spatial-temporal coronavirus disease 2019 (COVID-19) prevalence and reproduction numbers in England