Problem-solving recognition in scientific text

As far back as Aristotle, problems and solutions have been recognised as a core pattern of thought, and in particular of the scientific method. Therefore, they play a significant role in the understanding of academic texts from the scientific domain. Capturing knowledge of such problem-solving utterances would provide a deep insight into text understanding. In this dissertation, I present the task of problem-solving recognition in scientific text. To date, work on problem-solving recognition has received both theoretical and computational treatment. However, theories of problem-solving put forward by applied linguists lack practical adaptation to the domain of scientific text, and computational analyses have been narrow in scope. This dissertation provides a new model of problem-solving. It is an adaptation of Hoey's (2001) model, tailored to the scientific domain. As far as modelling problems is concerned, I divided the text string expressing the statement of a problem into sub-components; this is one of my main contributions. I have mapped these sub-components to functional roles, and thus operationalised the model in such a way that it can be annotated by humans reliably. As far as the problem-solving relationship between problems and solutions is concerned, my model takes into account the local network of relationships existing between problems. In order to validate this new model, a large-scale annotation study was conducted. The annotation study shows significant agreement amongst the annotators. The model is automated in two stages using a blend of classical machine learning and state-of-the-art deep learning methods. The first stage involves the implementation of problem and solution recognisers which operate at the sentence level. The second stage is more complex in that it recognises problems and solutions jointly at the token-level, and also establishes whether there is a problem-solving relationship between each of them. One of the best performers at this stage was a Neural Relational Topic Model. The results from automation show that the model is able to recognise problem-solving utterances in text to a high degree of accuracy. My work has already shown a positive impact in both industry and academia. One start-up is currently using the model for representing academic articles, and a Japanese collaborator has received a grant to adapt my model to Japanese text

Measuring Service Encounter Satisfaction with Customer Service Chatbots using Sentiment Analysis

Chatbots are software-based systems designed to interact with humans using text-based natural language and have attracted considerable interest in online service encounters. In this context, service providers face the challenge of measuring chatbot service encounter satisfaction (CSES), as most approaches are limited to post-interaction surveys that are rarely answered and often biased. Asa result, service providers cannot react quickly to service failures and dissatisfied customers. To address this challenge, we investigate the application of automated sentiment analysis methods as a proxy to measure CSES. Therefore, we first compare different sentiment analysis methods. Second, we investigate the relationship between objectively computed sentiment scores of dialogs and subjectively measured CSES values. Third, we evaluate whether this relationship also exists for utterance sequences throughout the dialog. The paper contributes by proposing and applying an automatic and objective approach to use sentiment scores as a proxy to measure CSES

Psy-LLM: Scaling up Global Mental Health Psychological Services with AI-based Large Language Models

The demand for psychological counseling has grown significantly in recent years, particularly with the global outbreak of COVID-19, which has heightened the need for timely and professional mental health support. Online psychological counseling has emerged as the predominant mode of providing services in response to this demand. In this study, we propose the Psy-LLM framework, an AI-based system leveraging Large Language Models (LLMs) for question-answering in online psychological consultation. Our framework combines pre-trained LLMs with real-world professional Q&A from psychologists and extensively crawled psychological articles. The Psy-LLM framework serves as a front-end tool for healthcare professionals, allowing them to provide immediate responses and mindfulness activities to alleviate patient stress. Additionally, it functions as a screening tool to identify urgent cases requiring further assistance. We evaluated the framework using intrinsic metrics, such as perplexity, and extrinsic evaluation metrics, with human participant assessments of response helpfulness, fluency, relevance, and logic. The results demonstrate the effectiveness of the Psy-LLM framework in generating coherent and relevant answers to psychological questions. This article concludes by discussing the potential of large language models to enhance mental health support through AI technologies in online psychological consultation

Are We There Yet?: The Development of a Corpus Annotated for Social Acts in Multilingual Online Discourse

We present the AAWD and AACD corpora, a collection of discussions drawn from Wikipedia talk pages and small group IRC discussions in English, Russian and Mandarin. Our datasets are annotated with labels capturing two kinds of social acts: alignment moves and authority claims. We describe these social acts, describe our annotation process, highlight challenges we encountered and strategies we employed during annotation, and present some analyses of resulting data set which illustrate the utility of our corpus and identify interactions among social acts and between participant status and social acts and in online discourse

Current Challenges and Visions in Music Recommender Systems Research

Music recommender systems (MRS) have experienced a boom in recent years, thanks to the emergence and success of online streaming services, which nowadays make available almost all music in the world at the user's fingertip. While today's MRS considerably help users to find interesting music in these huge catalogs, MRS research is still facing substantial challenges. In particular when it comes to build, incorporate, and evaluate recommendation strategies that integrate information beyond simple user--item interactions or content-based descriptors, but dig deep into the very essence of listener needs, preferences, and intentions, MRS research becomes a big endeavor and related publications quite sparse. The purpose of this trends and survey article is twofold. We first identify and shed light on what we believe are the most pressing challenges MRS research is facing, from both academic and industry perspectives. We review the state of the art towards solving these challenges and discuss its limitations. Second, we detail possible future directions and visions we contemplate for the further evolution of the field. The article should therefore serve two purposes: giving the interested reader an overview of current challenges in MRS research and providing guidance for young researchers by identifying interesting, yet under-researched, directions in the field

Development and Evaluation of Three Chatbots for Postpartum Mood and Anxiety Disorders

In collaboration with Postpartum Support International (PSI), a non-profit organization dedicated to supporting caregivers with postpartum mood and anxiety disorders, we developed three chatbots to provide context-specific empathetic support to postpartum caregivers, leveraging both rule-based and generative models. We present and evaluate the performance of our chatbots using both machine-based metrics and human-based questionnaires. Overall, our rule-based model achieves the best performance, with outputs that are close to ground truth reference and contain the highest levels of empathy. Human users prefer the rule-based chatbot over the generative chatbot for its context-specific and human-like replies. Our generative chatbot also produced empathetic responses and was described by human users as engaging. However, limitations in the training dataset often result in confusing or nonsensical responses. We conclude by discussing practical benefits of rule-based vs. generative models for supporting individuals with mental health challenges. In light of the recent surge of ChatGPT and BARD, we also discuss the possibilities and pitfalls of large language models for digital mental healthcare

Development of English as a Second Language in the Context of Massively Multiplayer Online Role-playing Games

This dissertation examined the affordances of commercially developed massively multiplayer online (role-playing) games (MMOGs) for second language (L2) development. It comprises three self-contained but related studies. The first study, as a scoping review, synthesized 32 empirical papers, which investigated different aspects of L2 development in the context of these games. It sought to find out what aspects of L2 learning have been examined and how, and what the findings suggest regarding L2 learning opportunities and outcomes. This study highlighted that empirical research in this area is mainly qualitative and that L2-related affective factors, vocabulary, and communicative competence have been the most widely investigated topics. It concluded that MMOGs afford socially supportive and emotionally safe environments, which encourage L2 learners to use multiple opportunities for enriching their L2 vocabulary and enhancing their communicative competence in the target language. The second study was an exploratory research. It adopted an interactionist approach to characterize the nature of the negotiations of meaning that occurred in the conversational exchanges between native (NES) and non-native English speakers (NNESs) playing World of Warcraft. The data consisted of 63 hours of audio-recorded, in-game conversations over a 5-month period. The participants consisted of an NES and 6 NNESs who were divided into two groups (low and high intermediate) according to their English language proficiency. This study identified and characterized the most frequently occurred triggers, indicators, responses and reaction to the responses in three types of dyadic conversational exchanges. The third study examined L2 development through ―usage-based‖ theories of language learning. It was a time-series (longitudinal) research that examined the trend of changes in the linguistic complexity of the NNESs‘ spoken discourse during a 5-month period of gameplay. This examination involved repeated (in three equally-distributed time intervals) calculations of fourteen syntactic complexity indices and the indices associated with three components of lexical complexity (diversity, sophistication, and density). Overall, the results turned out to be more promising for the low intermediate than the high intermediate group of the NNESs. More detailed findings are presented and discussed in light of the current literature

Development of English as a Second Language in the Context of Massively Multiplayer Online Role-playing Games

This dissertation examined the affordances of commercially developed massively multiplayer online (role-playing) games (MMOGs) for second language (L2) development. It comprises three self-contained but related studies. The first study, as a scoping review, synthesized 32 empirical papers, which investigated different aspects of L2 development in the context of these games. It sought to find out what aspects of L2 learning have been examined and how, and what the findings suggest regarding L2 learning opportunities and outcomes. This study highlighted that empirical research in this area is mainly qualitative and that L2-related affective factors, vocabulary, and communicative competence have been the most widely investigated topics. It concluded that MMOGs afford socially supportive and emotionally safe environments, which encourage L2 learners to use multiple opportunities for enriching their L2 vocabulary and enhancing their communicative competence in the target language. The second study was an exploratory research. It adopted an interactionist approach to characterize the nature of the negotiations of meaning that occurred in the conversational exchanges between native (NES) and non-native English speakers (NNESs) playing World of Warcraft. The data consisted of 63 hours of audio-recorded, in-game conversations over a 5-month period. The participants consisted of an NES and 6 NNESs who were divided into two groups (low and high intermediate) according to their English language proficiency. This study identified and characterized the most frequently occurred triggers, indicators, responses and reaction to the responses in three types of dyadic conversational exchanges. The third study examined L2 development through ―usage-based‖ theories of language learning. It was a time-series (longitudinal) research that examined the trend of changes in the linguistic complexity of the NNESs‘ spoken discourse during a 5-month period of gameplay. This examination involved repeated (in three equally-distributed time intervals) calculations of fourteen syntactic complexity indices and the indices associated with three components of lexical complexity (diversity, sophistication, and density). Overall, the results turned out to be more promising for the low intermediate than the high intermediate group of the NNESs. More detailed findings are presented and discussed in light of the current literature