Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis

Background: Impairments in speech production are a core symptom of non-affective psychosis (NAP). While traditional clinical ratings of patients’ speech involve a subjective human factor, modern methods of natural language processing (NLP) promise an automatic and objective way of analyzing patients’ speech. This study aimed to validate NLP methods for analyzing speech production in NAP patients. Methods: Speech samples from patients with a diagnosis of schizophrenia or schizoaffective disorder were obtained at two measurement points, 6 months apart. Out of N = 71 patients at T1, speech samples were also available for N = 54 patients at T2. Global and local models of semantic coherence as well as different word embeddings (word2vec vs. GloVe) were applied to the transcribed speech samples. They were tested and compared regarding their correlation with clinical ratings and external criteria from cross-sectional and longitudinal measurements. Results: Results did not show differences for global vs. local coherence models and found more significant correlations between word2vec models and clinically relevant outcome variables than for GloVe models. Exploratory analysis of longitudinal data did not yield significant correlation with coherence scores. Conclusion: These results indicate that natural language processing methods need to be critically validated in more studies and carefully selected before clinical application

Modeling Incoherent Discourse in Non-Affective Psychosis

Background: Computational linguistic methodology allows quantification of speech abnormalities in non-affective psychosis. For this patient group, incoherent speech has long been described as a symptom of formal thought disorder. Our study is an interdisciplinary attempt at developing a model of incoherence in non-affective psychosis, informed by computational linguistic methodology as well as psychiatric research, which both conceptualize incoherence as associative loosening. The primary aim of this pilot study was methodological: to validate the model against clinical data and reduce bias in automated coherence analysis. Methods: Speech samples were obtained from patients with a diagnosis of schizophrenia or schizoaffective disorder, who were divided into two groups of n = 20 subjects each, based on different clinical ratings of positive formal thought disorder, and n = 20 healthy control subjects. Results: Coherence metrics that were automatically derived from interview transcripts significantly predicted clinical ratings of thought disorder. Significant results from multinomial regression analysis revealed that group membership (controls vs. patients with vs. without formal thought disorder) could be predicted based on automated coherence analysis when bias was considered. Further improvement of the regression model was reached by including variables that psychiatric research has shown to inform clinical diagnostics of positive formal thought disorder. Conclusions: Automated coherence analysis may capture different features of incoherent speech than clinical ratings of formal thought disorder. Models of incoherence in non-affective psychosis should include automatically derived coherence metrics as well as lexical and syntactic features that influence the comprehensibility of speech

Whodunnit? Electrophysiological correlates of agency judgements.

Sense of agency refers to the feeling that "I" am responsible for those external events that are directly produced by one's own voluntary actions. Recent theories distinguish between a non-conceptual "feeling" of agency linked to changes in the processing of self-generated sensory events, and a higher-order judgement of agency, which attributes sensory events to the self. In the current study we explore the neural correlates of the judgement of agency by means of electrophysiology. We measured event-related potentials to tones that were either perceived or not perceived as triggered by participants' voluntary actions and related these potentials to later judgements of agency over the tones. Replicating earlier findings on predictive sensory attenuation, we found that the N1 component was attenuated for congruent tones that corresponded to the learned action-effect mapping as opposed to incongruent tones that did not correspond to the previously acquired associations between actions and tones. The P3a component, but not the N1, directly reflected the judgement of agency: deflections in this component were greater for tones judged as self-generated than for tones judged as externally produced. The fact that the outcome of the later agency judgement was predictable based on the P3a component demonstrates that agency judgements incorporate early information processing components and are not purely reconstructive, post-hoc evaluations generated at time of judgement

The sense of agency for actions. Results from an electrophysiological experiment

“Sense of agency” refers to the feeling that oneself is responsible for those external events that are directly produced by one’s own voluntary actions. Recent theories distinguish between a non-conceptual ‘‘feeling’’ of agency linked to changes in the processing of self-generated sensory events, and a higher-order “judgment” of agency, which attributes sensory events to the self. In the current study (Kühn et al. 2011), we explore the neural correlates of the judgment of agency by means of electrophysiology. We measured event-related potentials to tones that were either perceived or not perceived as triggered by participants’ voluntary actions, and related these potentials to later judgments of agency over the tones. We investigated whether N1, P3a and the movement-related cortical potentials, as potential electrophysiological markers of first-step feeling of agency processing predict the outcome of the later agency judgment. The experimental design was based on Sato and Yasuda. Participants learned that certain actions resulted in certain consequences in the environment (tones). They were then introduced to an ambiguous context in which tone congruency and delay were manipulated. The participants had to judge whether presented tones where self-generated or externally produced. Replicating earlier findings on predictive sensory attenuation, we found that the N1 component was attenuated for congruent tones that corresponded to the learned action-effect mapping as opposed to incongruent tones that did not correspond to the previously acquired associations between actions and tones. In addition, it was found that the N1 attenuation depends on learning. We then focused our analysis on the most ambiguous condition, namely, congruent tones presented with 300 ms delay. We divided these identical trials according to the judgments of whether each individual tone was self- or externally-produced. The P3a component, but not the N1, directly reflected the judgment of agency: deflections in this component were greater for tones judged as self-generated than for tones judged as externally produced. No significant correlation could be found between the agency judgments and the movement-related cortical potentials. The fact that the outcome of the later agency judgment was predictable based on the P3a component demonstrates that agency judgments incorporate early information processing components and are not purely reconstructive, post-hoc evaluations generated at the time of judgment.Das Gefühl der Urheberschaft von Handlungen („sense of agency“) beschreibt das Gefühl, selbst verantwortlich für Konsequenzen eigener Handlungen in der Umwelt zu sein. In theoretischen Modellen wird zwischen einem impliziten Urheberschaftsgefühl, das mit der Verarbeitung selbst-generierter sensorischer Reize in Zusammenhang steht, und einem expliziten Urheberschaftsurteil, bei welchem ein sensorischer Reiz dem Selbst zugeschrieben wird, unterschieden. In der vorliegenden Studie (Kühn et al. 2011) untersuchten wir die neuronalen Korrelate expliziter Urheberschaftsurteile mittels Elektroenzephalographie (EEG). Mit einem akustischen Reiz assoziierte ereigniskorrelierte Potentiale (ERP), die entweder als durch einen eigenen Tastendruck ausgelöst oder als fremdproduziert erlebt wurden, korrelierten wir mit den darauffolgenden expliziten Urheberschaftsurteilen. So konnten wir untersuchen, ob N1-, P3a- und die bewegungsabhängigen kortikalen Potentiale als potenzielle elektrophysiologische Marker für das implizite Urheberschaftsgefühl spätere explizite Urheberschaftsurteile prädizieren können. Das experimentelle Design basierte auf einem von Sato und Yasuda entwickelten Experiment. Zu Beginn des Experiments (Lernphase, mapping) lernten die Probanden eine Handlung-Effekt-Sequenz (linker bzw. rechter Tastendruck – hochfrequenter bzw. niederfrequenter Ton). Im eigentlichen Experiment wurde dann eine Ambiguität bezüglich der Urheberschaft hergestellt. Die Töne wurden kongruent oder inkongruent zu der Lernphase und mit drei zeitlichen Intervallen zum Tastendruck (100, 300 und 600 ms) präsentiert. Die Probanden sollten beurteilen (Urheberschaftsurteil), ob die gehörten Töne aus den eigenen Handlungen resultierten oder unabhängig davon, durch den Computer generiert auftraten. Wir replizierten einen bekannten Effekt prädiktiver sensorischer Attenuierung, bei dem die Amplitude der N1-Komponente bei Tönen, die der gelernten Handlungs- Effekt-Sequenz entsprachen (Kongruenz) im Gegensatz zu solchen, bei denen die gelernte Sequenz nicht beibehalten wurde, vermindert war. Zusätzlich konnten wir zeigen, dass dieser N1-Effekt von Lernprozessen abhängig ist. Bei Analyse der experimentellen Bedingung mit der höchsten Ambiguität – kongruenten Tönen mit 300 ms Intervall zwischen Tastendruck und Ton – wurden dann die identischen Einzeldurchgänge ausschließlich nach den subjektiven, expliziten Urteilen über die Urheberschaft (Selbst- oder Computer-generiert) unterteilt. Die P3a-, jedoch nicht die N1-Komponente, schien hier im Zusammenhang mit den Urheberschaftsurteilen zu stehen: Die P3a-Komponente war für Töne attenuiert, die als selbstproduziert empfunden wurden. Keine signifikante Korrelation konnte zwischen den bewegungsabhängigen kortikalen Potentialen und den Urheberschaftsurteilen gefunden werden. Die Tatsache, dass die späteren expliziten Urheberschaftsurteile durch die Amplitude der P3a Komponente des ERP prädizierbar war, spricht dafür, dass Urheberschaftsurteile unter anderem auch durch neurale Ereignisse, die zeitlich sehr nah am Stimulus liegen, beeinflusst werden und somit nicht ausschließlich einen rekonstruktiven, post-hoc Prozess darstellen

Emoji Norming

We introduce a novel dataset of affective, semantic, and descriptive norms for all facial emojis. We gathered and examined subjective ratings from 138 German speakers along five essential dimensions: valence, arousal, familiarity, clarity, and visual complexity. Additionally, we provide absolute frequency counts of emoji use, drawn from an extensive Twitter corpus. Our results replicate the well-established quadratic relationship between arousal and valence of lexical items, also known for words. We also report associations among the variables: for example, subjective familiarity is strongly correlated with usage frequency, and positively associated with valence and clarity. To establish the meanings associated with face emojis, we asked for up to three descriptions for each emoji. Using this linguistic data, we computed vector embeddings for each emoji, enabling an exploration of their distribution within the semantic space. Our description based emoji vector embeddings not only capture typical meaning components of emojis, such as their valence, but also surpass simple definitions and direct emoji2vec models

Emoji Homophones

Code and experimental data for our self-paced reading study on the processing of emojis in sentences

Schematic Drawing of the experimental design.

<p>Schematic Drawing of the experimental design.</p

Plots displaying mean signal averaged over electrodes F3, Fz, F4, FC1, FCz, FC2.

<p>(A) Interaction plot of component (N1 vs. P3a) and condition (congruent vs. incongruent tones), (B) Interaction plot of component (N1 vs. P3a) and condition (“me” vs. ”somebody else” agency judgement, median split in trials with congruent tones and delay 300 ms). * indicates a significant post-hoc t-test.</p

Behavioral effects of agency judgement.

<p>Error bars depict standard error of the mean.</p