3 research outputs found
Cognitive network science reveals bias in GPT-3, ChatGPT, and GPT-4 mirroring math anxiety in high-school students
Large language models are becoming increasingly integrated into our lives.
Hence, it is important to understand the biases present in their outputs in
order to avoid perpetuating harmful stereotypes, which originate in our own
flawed ways of thinking. This challenge requires developing new benchmarks and
methods for quantifying affective and semantic bias, keeping in mind that LLMs
act as psycho-social mirrors that reflect the views and tendencies that are
prevalent in society. One such tendency that has harmful negative effects is
the global phenomenon of anxiety toward math and STEM subjects. Here, we
investigate perceptions of math and STEM fields provided by cutting-edge
language models, namely GPT-3, Chat-GPT, and GPT-4, by applying an approach
from network science and cognitive psychology. Specifically, we use behavioral
forma mentis networks (BFMNs) to understand how these LLMs frame math and STEM
disciplines in relation to other concepts. We use data obtained by probing the
three LLMs in a language generation task that has previously been applied to
humans. Our findings indicate that LLMs have an overall negative perception of
math and STEM fields, with math being perceived most negatively. We observe
significant differences across the three LLMs. We observe that newer versions
(i.e. GPT-4) produce richer, more complex perceptions as well as less negative
perceptions compared to older versions and N=159 high-school students. These
findings suggest that advances in the architecture of LLMs may lead to
increasingly less biased models that could even perhaps someday aid in reducing
harmful stereotypes in society rather than perpetuating them.Comment: 23 pages, 8 figure
Voices of Rape: cognitive networks link passive voice usage to psychological distress in online narratives
Past studies of sexual assault have found that passive voice descriptions of rape elicit an increased perception of victim responsibility in readers. Building on this, we investigate rape survivors' self-perceptions and perspectives, as disclosed online, in relation to passive voice vs. active voice usage. From roughly 65,000 posts written by over 17,000 users on Reddit’s r/sexualassault board, we group texts into a passive voice group and an active voice group, respectively. We then use textual forma mentis networks, a cognitive network science approach, to create network representations from text such that nodes represent words/concepts while links represent syntactic and semantic relationships between them. These network representations make it possible to investigate the structural linguistic, cognitive, and emotional patterns that are prevalent in passive voice and active voice narratives, respectively. Ranking nodes by closeness centrality and comparing across networks, we identify nodes that are significantly more central to one network compared to the other, thus identifying characteristic concepts that semantically differentiate the two groups of narratives. We then investigate the contexts of these concepts through an in-depth semantic frame analysis. We find that concepts related to psychological distress (e.g. PTSD, flashback) are significantly more central to the passive voice network, providing quantitative evidence of a link between passive voice usage and an increased focus on psychological distress. We also find that family members (e.g. parent, brother) are more central to the active voice network, suggesting a connection between active voice usage and an increased focus on others' roles in rape survivors’ experiences. Our quantitative results have important implications for rape survivors as well as mental health professionals regarding the link between language and mental health