Habitual prospective memory in schizophrenia

Background Prospective memory (PM), the act of remembering that something has to be done in the future without any explicit prompting to recall, provides a useful framework with which to examine problems in internal-source monitoring. This is because it requires distinguishing between two internally-generated processes, namely the intention to perform an action versus actual performance of the action. In habitual tasks, such as taking medicine every few hours, the same PM task is performed regularly and thus it is essential that the individual is able to distinguish thoughts (i.e., thinking about taking the medicine) from actions (i.e., actually taking the medicine). Methods We assessed habitual PM in patients with schizophrenia by employing a laboratory analogue of a habitual PM task in which, concurrently with maneuvering a ball around an obstacle course (ongoing activity), participants were to turn over a counter once during each trial (PM task). After each trial, participants were asked whether they had remembered to turn the counter over. Results Patients with schizophrenia made a disproportionate number of errors compared to controls of reporting that a PM response had been made (i.e., the counter turned over) after an omission error (i.e., the counter was not turned over). There was no group difference in terms of reporting that an omission error occurred (i.e., forgetting to turn over the counter) when in fact a PM response had been made. Conclusion Patients with schizophrenia displayed a specific deficit distinguishing between two internally-generated sources, attributable to either poor source monitoring or temporal discrimination

Reflections on measuring disordered thoughts as expressed via language

Thought disorder, as inferred from disorganized and incoherent speech, is an important part of the clinical presentation in schizophrenia. Traditional measurement approaches essentially count occurrences of certain speech events which may have restricted their usefulness. Applying speech technologies in assessment can help automate traditional clinical rating tasks and thereby complement the process. Adopting these computational approaches affords clinical translational opportunities to enhance the traditional assessment by applying such methods remotely and scoring various parts of the assessment automatically. Further, digital measures of language may help detect subtle clinically significant signs and thus potentially disrupt the usual manner by which things are conducted. If proven beneficial to patient care, methods where patients’ voice are the primary data source could become core components of future clinical decision support systems that improve risk assessment. However, even if it is possible to measure thought disorder in a sensitive, reliable and efficient manner, there remain many challenges to then translate into a clinically implementable tool that can contribute towards providing better care. Indeed, embracing technology - notably artificial intelligence - requires vigorous standards for reporting underlying assumptions so as to ensure a trustworthy and ethical clinical science

The mental health consequences on children of the war in Ukraine: A commentary

The news from Ukraine is currently full of heart-wrenching stories accompanied by graphic images of civilian casualties and massacres that are telecast world-wide on a daily basis. It is hard to fathom the magnitude of the devastation and disruption to regular lives and everyday routines that war brings with it, the witnessing of countless deaths, the associated trauma of living in perpetual fear, and the daily experience of many families and orphans who are crowded into basement bomb shelters now for months on end. These issues make us contemplate the mental health consequences, among other lasting effects, of this costly war in Ukraine, and wars in other countries not so widely featured in Western news. Despite people of all ages being affected by war, children are especially vulnerable. This commentary outlines some of the epidemiology of the consequences of war, the mental health sequelae specifically, and the complexity of providing culturally and contextually relevant interventions that meet the needs of children

Improving the Applicability of AI for Psychiatric Applications through Human-in-the-loop Methodologies

Objectives: Machine learning (ML) and natural language processing have great potential to improve effciency and accuracy in diagnosis, treatment recommendations, predictive interventions, and scarce resource allocation within psychiatry. Researchers often conceptualize such an approach as operating in isolation without much need for human involvement, yet it remains crucial to harness human-inthe-loop practices when developing and implementing such techniques as their absence may be catastrophic. We advocate for building ML-based technologies that collaborate with experts within psychiatry in all stages of implementation and use to increase model performance while simultaneously increasing the practicality, robustness, and reliability of the process. Methods: We showcase pitfalls of the traditional ML framework and explain how it can be improved with human-inthe-loop techniques. Specifcally, we applied active learning strategies to the automatic scoring of a story recall task and compared the results to a traditional approach. Results: Human-in-the-loop methodologies supplied a greater understanding of where the model was least confdent or had knowledge gaps during training. As compared to the traditional framework, less than half of the training data were needed to reach a given accuracy. Conclusions: Human-in-the-loop ML is an approach to data collection and model creation that harnesses active learning to select the most critical data needed to increase a model’s accuracy and generalizability more effciently than classic random sampling would otherwise allow. Such techniques may additionally operate as safeguards from spurious predictions and can aid in decreasing disparities that artifcial intelligence systems otherwise propagate

Random Texts Do Not Exhibit the Real Zipf's Law-Like Rank Distribution

Zipf's law states that the relationship between the frequency of a word in a text and its rank (the most frequent word has rank , the 2nd most frequent word has rank ,...) is approximately linear when plotted on a double logarithmic scale. It has been argued that the law is not a relevant or useful property of language because simple random texts - constructed by concatenating random characters including blanks behaving as word delimiters - exhibit a Zipf's law-like word rank distribution.In this article, we examine the flaws of such putative good fits of random texts. We demonstrate - by means of three different statistical tests - that ranks derived from random texts and ranks derived from real texts are statistically inconsistent with the parameters employed to argue for such a good fit, even when the parameters are inferred from the target real text. Our findings are valid for both the simplest random texts composed of equally likely characters as well as more elaborate and realistic versions where character probabilities are borrowed from a real text.The good fit of random texts to real Zipf's law-like rank distributions has not yet been established. Therefore, we suggest that Zipf's law might in fact be a fundamental law in natural languages

Detecting order-disorder transitions in discourse : implications for schizophrenia

Abstract Several psychiatric and neurological conditions affect the semantic organization and content of a patient's speech. Specifically, the discourse of patients with schizophrenia is frequently characterized as lacking coherence. The evaluation of disturbances in discourse is often used in diagnosis and in assessing treatment efficacy, and is an important factor in prognosis. Measuring these deviations, such as “loss of meaning” and incoherence, is difficult and requires substantial human effort. Computational procedures can be employed to characterize the nature of the anomalies in discourse. We present a set of new tools derived from network theory and information science that may assist in empirical and clinical studies of communication patterns in patients, and provide the foundation for future automatic procedures. First we review information science and complex network approaches to measuring semantic coherence, and then we introduce a representation of discourse that allows for the computation of measures of disorganization. Finally we apply these tools to speech transcriptions from patients and a healthy participant, illustrating the implications and potential of this novel framework

COMT Val158Met polymorphism, cognitive stability and cognitive flexibility: an experimental examination

<p>Abstract</p> <p>Background</p> <p>Dopamine in prefrontal cortex (PFC) modulates core cognitive processes, notably working memory and executive control. Dopamine regulating genes and polymorphisms affecting PFC - including Catechol-O-Methyltransferase (COMT) Val158Met - are crucial to understanding the molecular genetics of cognitive function and dysfunction. A mechanistic account of the COMT Val158Met effect associates the Met allele with increased tonic dopamine transmission underlying maintenance of relevant information, and the Val allele with increased phasic dopamine transmission underlying the flexibility of updating new information. Thus, consistent with some earlier work, we predicted that Val carriers would display poorer performance when the maintenance component was taxed, while Met carriers would be less efficient when rapid updating was required.</p> <p>Methods</p> <p>Using a Stroop task that manipulated level of required cognitive stability and flexibility, we examined reaction time performance of patients with schizophrenia (n = 67) and healthy controls (n = 186) genotyped for the Val/Met variation.</p> <p>Results</p> <p>In both groups we found a Met advantage for tasks requiring cognitive stability, but no COMT effect when a moderate level of cognitive flexibility was required, or when a conflict cost measure was calculated.</p> <p>Conclusions</p> <p>Our results do not support a simple stability/flexibility model of dopamine COMT Val/Met effects and suggest a somewhat different conceptualization and experimental operationalization of these cognitive components.</p

Reflections on the nature of measurement in language-based automated assessments of patients' mental state and cognitive function

Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determining which computational analyses align best with the targeted neurocognitive/psychological functions that we want to assess. In this paper we reflect on two decades of experience with the application of language-based assessment to patients' mental state and cognitive function by addressing the questions of what we are measuring, how it should be measured and why we are measuring the phenomena. We address the questions by advocating for a principled framework for aligning computational models to the constructs being assessed and the tasks being used, as well as defining how those constructs relate to patient clinical states. We further examine the assumptions that go into the computational models and the effects that model design decisions may have on the accuracy, bias and generalizability of models for assessing clinical states. Finally, we describe how this principled approach can further the goal of transitioning language-based computational assessments to part of clinical practice while gaining the trust of critical stakeholders

Extending the usefulness of the verbal memory test: The promise of machine learning

The evaluation of verbal memory is a core component of neuropsychological assessment in a wide range of clinical and research settings. Leveraging story recall to assay neurocognitive function could be made more useful if it were possible to administer frequently (i.e., would allow for the collection of more patient data over time) and automatically assess the recalls with machine learning methods. In the present study, we evaluated a novel story recall test with 24 parallel forms that was deployed using smart devices in 94 psychiatric inpatients and 80 nonpatient adults. Machine learning and vector-based natural language processing methods were employed to automate test scoring, and performance using these methods was evaluated in their incremental validity, criterion validity (i.e., convergence with trained human raters), and parallel forms reliability. Our results suggest moderate to high consistency across the parallel forms, high convergence with human raters (r values ~ 0.89), and high incremental validity for discriminating between groups. While much work remains, the present findings are critical for implementing an automated, neuropsychological test deployable using remote technologies across multiple and frequent administrations

Assessing dimensions of thought disorder with large language models: The tradeoff of accuracy and consistency

Natural Language Processing (NLP) methods have shown promise for the assessment of formal thought disorder, a hallmark feature of schizophrenia in which disturbances to the structure, organization, or coherence of thought can manifest as disordered or incoherent speech. We investigated the suitability of modern Large Language Models (LLMs - e.g., GPT-3.5, GPT-4, and Llama 3) to predict expert-generated ratings for three dimensions of thought disorder (coherence, content, and tangentiality) assigned to speech samples collected from both patients with a diagnosis of schizophrenia (n = 26) and healthy control participants (n = 25). In addition to (1) evaluating the accuracy of LLM-generated ratings relative to human experts, we also (2) investigated the degree to which the LLMs produced consistent ratings across multiple trials, and we (3) sought to understand the factors that impacted the consistency of LLM-generated output. We found that machine-generated ratings of the level of thought disorder in speech matched favorably those of expert humans, and we identified a tradeoff between accuracy and consistency in LLM ratings. Unlike traditional NLP methods, LLMs were not always consistent in their predictions, but these inconsistencies could be mitigated with careful parameter selection and ensemble methods. We discuss implications for NLP-based assessment of thought disorder and provide recommendations of best practices for integrating these methods in the field of psychiatry