A Human-centric Approach to NLP in Healthcare Applications

The abundance of personal health information available to healthcare professionals can be a facilitator to better care. However, it can also be a barrier, as the relevant information is often buried in the sheer amount of personal data, and healthcare professionals already lack time to take care of both patients and their data. This dissertation focuses on the role of natural language processing (NLP) in healthcare and how it can surface information relevant to healthcare professionals by modeling the extensive collections of documents that describe those whom they serve. In this dissertation, the extensive natural language data about a person is modeled as a set of documents, where the model inference is at the level of the individual, but evidence supporting that inference is found in a subset of their documents. The effectiveness of this modeling approach is demonstrated in the context of three healthcare applications. In the first application, clinical coding, document-level attention is used to model the hierarchy between a clinical encounter and its documents, jointly learning the encounter labels and the assignment of credits to specific documents. The second application, suicidality assessment using social media, further investigates how document-level attention can surface "high-signal" posts from the document set representing a potentially at-risk individual. Finally, the third application aims to help healthcare professionals write discharge summaries using an extract-then-abstract multidocument summarization pipeline to surface relevant information. As in many healthcare applications, these three applications seek to assist, not replace, clinicians. Evaluation and model design thus centers around healthcare professionals' needs. In clinical coding, document-level attention is shown to align well with professional clinical coders' expectations of evidence. In suicidality assessment, document-level attention leads to better and more time-efficient assessment by surfacing document-level evidence, shown empirically using a theoretically grounded time-aware evaluation measure and a dataset annotated by suicidality experts. Finally, extract-then-abstract summarization pipelines that assist healthcare professionals in writing discharge summaries are evaluated by their ability to surface faithful and relevant evidence

A Dual-Prompting for Interpretable Mental Health Language Models

Despite the increasing demand for AI-based mental health monitoring tools, their practical utility for clinicians is limited by the lack of interpretability.The CLPsych 2024 Shared Task (Chim et al., 2024) aims to enhance the interpretability of Large Language Models (LLMs), particularly in mental health analysis, by providing evidence of suicidality through linguistic content. We propose a dual-prompting approach: (i) Knowledge-aware evidence extraction by leveraging the expert identity and a suicide dictionary with a mental health-specific LLM; and (ii) Evidence summarization by employing an LLM-based consistency evaluator. Comprehensive experiments demonstrate the effectiveness of combining domain-specific information, revealing performance improvements and the approach's potential to aid clinicians in assessing mental state progression

Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networks

Mental health is a critical issue in modern society, and mental disorders could sometimes turn to suicidal ideation without effective treatment. Early detection of mental disorders and suicidal ideation from social content provides a potential way for effective social intervention. However, classifying suicidal ideation and other mental disorders is challenging as they share similar patterns in language usage and sentimental polarity. This paper enhances text representation with lexicon-based sentiment scores and latent topics and proposes using relation networks to detect suicidal ideation and mental disorders with related risk indicators. The relation module is further equipped with the attention mechanism to prioritize more critical relational features. Through experiments on three real-world datasets, our model outperforms most of its counterparts

Analyzing Suicide Risk From Linguistic Features in Social Media: Evaluation Study

Background: Effective suicide risk assessments and interventions are vital for suicide prevention. Although assessing such risks is best done by health care professionals, people experiencing suicidal ideation may not seek help. Hence, machine learning (ML) and computational linguistics can provide analytical tools for understanding and analyzing risks. This, therefore, facilitates suicide intervention and prevention.Objective: This study aims to explore, using statistical analyses and ML, whether computerized language analysis could be applied to assess and better understand a person's suicide risk on social media.Methods: We used the University of Maryland Suicidality Dataset comprising text posts written by users (N=866) of mental health-related forums on Reddit. Each user was classified with a suicide risk rating (no, low, moderate, or severe) by either medical experts or crowdsourced annotators, denoting their estimated likelihood of dying by suicide. In language analysis, the Linguistic Inquiry and Word Count lexicon assessed sentiment, thinking styles, and part of speech, whereas readability was explored using the TextStat library. The Mann-Whitney U test identified differences between at-risk (low, moderate, and severe risk) and no-risk users. Meanwhile, the Kruskal-Wallis test and Spearman correlation coefficient were used for granular analysis between risk levels and to identify redundancy, respectively. In the ML experiments, gradient boost, random forest, and support vector machine models were trained using 10-fold cross validation. The area under the receiver operator curve and F1-score were the primary measures. Finally, permutation importance uncovered the features that contributed the most to each model's decision-making.Results: Statistically significant differences (P0.84) was present between the part of speech variables, which implied redundancy and demonstrated the utility of aggregate features. All ML models performed similarly in their area under the curve (0.66-0.68); however, the random forest and gradient boost models were noticeably better in their F1-score (0.65 and 0.62) than the support vector machine (0.52). The features that contributed the most to the ML models were authenticity, clout, and negative emotions.Conclusions: In summary, our statistical analyses found linguistic features associated with suicide risk, such as social posturing (eg, authenticity and clout), first-person singular pronouns, and negation. This increased our understanding of the behavioral and thought patterns of social media users and provided insights into the mechanisms behind ML models. We also demonstrated the applicative potential of ML in assisting health care professionals to assess and manage individuals experiencing suicide risk

Overview of the CLPsych 2022 Shared Task: Capturing Moments of Change in Longitudinal User Posts

We provide an overview of the CLPsych 2022 Shared Task, which focusses on the automatic identification of Moments of Change in longitudinal posts by individuals on social media and its connection with information regarding mental health . This year\u27s task introduced the notion of longitudinal modelling of the text generated by an individual online over time, along with appropriate temporally sensitive evaluation metrics. The Shared Task consisted of two subtasks: (a) the main task of capturing changes in an individual\u27s mood (drastic changes-`Switches\u27- and gradual changes -`Escalations\u27- on the basis of textual content shared online; and subsequently (b) the sub-task of identifying the suicide risk level of an individual -- a continuation of the CLPsych 2019 Shared Task-- where participants were encouraged to explore how the identification of changes in mood in task (a) can help with assessing suicidality risk in task (b)

The Grievance Dictionary: Understanding Threatening Language Use

This paper introduces the Grievance Dictionary, a psycholinguistic dictionary which can be used to automatically understand language use in the context of grievance-fuelled violence threat assessment. We describe the development the dictionary, which was informed by suggestions from experienced threat assessment practitioners. These suggestions and subsequent human and computational word list generation resulted in a dictionary of 20,502 words annotated by 2,318 participants. The dictionary was validated by applying it to texts written by violent and non-violent individuals, showing strong evidence for a difference between populations in several dictionary categories. Further classification tasks showed promising performance, but future improvements are still needed. Finally, we provide instructions and suggestions for the use of the Grievance Dictionary by security professionals and (violence) researchers.Comment: pre-prin

Chronological detection of depression in social media threads by means of natural language processing

Detecting depression in social media has become an increasingly important research area in recent years. With the widespread use of social media platforms, individuals at risk of suicide often express their thoughts and emotions online, providing an opportunity for early detection and intervention. Artificial Intelligence and, particularly, Natural Language Processing open pathways towards the processing of massive amount of messages and the detection of depression traits and other risks related to mental health. Our main thesis question rests on the early prediction of depression detection in social media messages. We explore the accuracy gained by a system as more and more information (in terms of more social messages over time) from a user are available. Is the system becoming more and more accurate given subsequent information or is there a limit? How many messages do we need to train a simple model capable to attain an accuracy above a threshold? Do recent messages add much information to older ones? These research questions have arisen in our work. A key cornerstone in artificial intelligence-based approaches rests, needless to say, on to the available data-sets. The data available bounds the ability of the system to gain knowledge. Thus, an important part of this work consists on an overview of the data-sets used to detect depression in social media, also mentioning various extra data-sets along the way. In our study we found that there are international challenges devoted to this task, among others, CLPsych. We explore simple though efficient inference algorithms able to classify messages; next, we test the ability of the models to classify a user as with or without risk, just given social messages written by the user. In an attempt to put the focus on our main research question (i.e. assessing the impact of getting more and more information across time to gain accuracy in the task of message classification in the frame of early detection of depression signs) we opted for simple classifiers, that is, linear approaches, and left out of the scope exploring the behaviour of different classification approaches. Our experimental framework is developed using the practice data-set made available at CLPSych 2021. To make use of the data more intelligently, the chronological factor is added. Using a specific technique that progressively takes into account new data (chronologically) at each time, we can observe promising changes in the classification accuracy. These values might provide key ideas about the evolution of depression signs for detection. In other words, the results in a time-line might help to gain evidences that a user might be showing traces of or towards depression. At the end, some comparisons and discussion are made regarding past research work related to this field, to do a critical analysis of the results. Hizkuntza: Ingelesa