Focused Crawling and Model Evaluation in the field of Conversational Agents and Motivational Interviewing

The exploitation of Motivational Interviewing concepts when analysing individuals’ speech contributes to gaining valuable insights into their perspectives and attitudes towards behaviour change. The scarcity of labelled user data poses a persistent challenge and impedes technical advancements in research in non-English language scenarios. To address the limitations of manual data labelling, we propose a semisupervised learning method as a means to augment an existing training corpus. Our approach leverages machine-translated user-generated data sourced from social media communities and employs self-training techniques for annotation. We conduct an evaluation of multiple classifiers trained on various augmented datasets. To that end, we consider diverse source contexts and employ different effectiveness metrics. The results indicate that this weak labelling approach does not yield significant improvements in the overall classification capabilities of the models. However, notable enhancements were observed for the minority classes. As part of future work, we propose to enlarge the datasets only with new examples from the minority classes. We conclude that several factors, including the quality of machine translation, can potentially bias the pseudo-labelling models. The imbalanced nature of the data and the impact of a strict pre-filtering threshold are other important aspects that need to be taken into account.Universidade de Santiago de Compostela. Escola Técnica Superior de Enxeñarí

Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review

Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field. This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots. Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated. Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content). The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies

Digital Human Representations for Health Behavior Change: A Structured Literature Review

Organizations have increasingly begun using digital human representations (DHRs), such as avatars and embodied agents, to deliver health behavior change interventions (BCIs) that target modifiable risk factors in the smoking, nutrition, alcohol overconsumption, and physical inactivity (SNAP) domain. We conducted a structured literature review of 60 papers from the computing, health, and psychology literatures to investigate how DHRs’ social design affects whether BCIs succeed. Specifically, we analyzed how differences in social cues that DHRs use affect user psychology and how this can support or hinder different intervention functions. Building on established frameworks from the human-computer interaction and BCI literatures, we structure extant knowledge that can guide efforts to design future DHR-delivered BCIs. We conclude that we need more field studies to better understand the temporal dynamics and the mid-term and long-term effects of DHR social design on user perception and intervention outcomes

Investigating Predictors of Preferences for Deliberative Qualities of Political Conversations Using the Analytic Hierarchy Process

This thesis presents a conceptual and methodological approach to researching preferences for political conversation. The thesis contends that although real-world political discussion is not deliberative, insofar as it fails to satisfy the rigorous requirements deliberative theorists have laid out, the lack of empirical evidence is not cause to reject deliberation as a viable political theory. To connect the theoretical and empirical, this thesis presents a “quasi-deliberation” framework. Quasi-deliberation, for the purpose of this thesis, is the state of political discourse shaped by the choices made when ideal deliberative qualities conflict in the real world. Quasi-deliberation suggests that the differences between the real world and the theoretical are described by preferences regarding different qualities of political conversation. These qualities, drawn from the deliberation literature (Moy & Gastil, 2006) are “dominance during political conversation,” “clarity” of opinion expression, use of “reason, logic, and evidence,” and “understanding of other conversants\u27 views” (p. 448). The thesis tests the exploratory supposition that these choices are predictable outcomes of antecedent political characteristics of respondents, using a nationwide online survey instrument distributed to the non-random membership of a website. The characteristics measured in this thesis are culturally-informed worldview (Kahan, Slovic, Braman, & Gastil, 2006), attributes of personal discursive networks (Moy & Gastil, 2006), and political information efficacy (Kaid, McKinney, & Tedesco, 2007). These three sets of measures are used as independent variables to describe the unique, discursively relevant characteristics of the respondent. Each is then tested as a predictor of the relative priorities ascribed to each deliberative quality. Saaty’s (1980) Analytic Hierarchy Process is used to create the dependent priority ranking variables. Respondents provide a preference for each quality vis-à-vis each other, producing a preference matrix, from which a single priority vector is derived

Transformational Leadership and Moral Discourse in the Workplace and Civil Society

This study was grounded in the theory and practice of transformational leadership, where leaders function as moral agents of change as they facilitate values talk (moral discourse) among their constituents. The study took its cue from Rost\u27s call for a new paradigm for leadership ethics that calls for methods of group moral decision making to assess organizational and social ends. The inquiry sought to better understand how leaders engage others in moral conversation and how such processes influence organizational culture and democratic civil society. The methodology was qualitative and phenomenological as it was centered on leaders\u27 perceptions of their experiences in diverse organizational settings across public, private, and social sectors. Data was collected through focus groups and individual interviews and analyzed through the constant comparative method. Data was also interpreted within the socio-political context of a communitarian worldview that postures moral discourse as a means to identify shared values that build social capital and sustain the common good. Other theoretical contexts draw from discourse ethics, adult critical pedagogy, and moral development. The findings of the study put forth a typology of moral discourse framed in categories that include: conversational venues, individual and social impediments to the conversation, communicative dynamics that stimulate the conversation, speech actions, speech styles, functions of moral discourse, and specific leader practices that advance the conversation. Implications for practice in the workplace are framed in areas of organizational development and business ethics. Other implications are considered for the practice of democratic deliberation

The effects of user assistance systems on user perception and behavior

The rapid development of information technology (IT) is changing how people approach and interact with IT systems (Maedche et al. 2016). IT systems can increasingly support people in performing ever more complex tasks (Vtyurina and Fourney 2018). However, people's cognitive abilities have not evolved as quickly as technology (Maedche et al. 2016). Thus, different external factors (e.g., complexity or uncertainty) and internal conditions (e.g., cognitive load or stress) reduce decision quality (Acciarini et al. 2021; Caputo 2013; Hilbert 2012). User-assistance systems (UASs) can help to compensate for human weaknesses and cope with new challenges. UASs aim to improve the user's cognition and capabilities, benefiting individuals, organizations, and society. To achieve this goal, UASs collect, prepare, aggregate, analyze information, and communicate results according to user preferences (Maedche et al. 2019). This support can relieve users and improve the quality of decision-making. Using UASs offers many benefits but requires successful interaction between the user and the UAS. However, this interaction introduces social and technical challenges, such as loss of control or reduced explainability, which can affect user trust and willingness to use the UAS (Maedche et al. 2019). To realize the benefits, UASs must be developed based on an understanding and incorporation of users' needs. Users and UASs are part of a socio-technical system to complete a specific task (Maedche et al. 2019). To create a benefit from the interaction, it is necessary to understand the interaction within the socio-technical system, i.e., the interaction between the user, UAS, and task, and to align the different components. For this reason, this dissertation aims to extend the existing knowledge on UAS design by better understanding the effects and mechanisms during the interaction between UASs and users in different application contexts. Therefore, theory and findings from different disciplines are combined and new theoretical knowledge is derived. In addition, data is collected and analyzed to validate the new theoretical knowledge empirically. The findings can be used to reduce adaptation barriers and realize a positive outcome. Overall this dissertation addresses the four classes of UASs presented by Maedche et al. (2016): basic UASs, interactive UASs, intelligent UASs, and anticipating UASs. First, this dissertation contributes to understanding how users interact with basic UASs. Basic UASs do not process contextual information and interact little with the user (Maedche et al. 2016). This behavior makes basic UASs suitable for application contexts, such as social media, where little interaction is desired. Social media is primarily used for entertainment and focuses on content consumption (Moravec et al. 2018). As a result, social media has become an essential source of news but also a target for fake news, with negative consequences for individuals and society (Clarke et al. 2021; Laato et al. 2020). Thus, this thesis presents two approaches to how basic UASs can be used to reduce the negative influence of fake news. Firstly, basic UASs can provide interventions by warning users of questionable content and providing verified information but the order in which the intervention elements are displayed influences the fake news perception. The intervention elements should be displayed after the fake news story to achieve an efficient intervention. Secondly, basic UASs can provide social norms to motivate users to report fake news and thereby stop the spread of fake news. However, social norms should be used carefully, as they can backfire and reduce the willingness to report fake news. Second, this dissertation contributes to understanding how users interact with interactive UASs. Interactive UASs incorporate limited information from the application context but focus on close interaction with the user to achieve a specific goal or behavior (Maedche et al. 2016). Typical goals include more physical activity, a healthier diet, and less tobacco and alcohol consumption to prevent disease and premature death (World Health Organization 2020). To increase goal achievement, previous researchers often utilize digital human representations (DHRs) such as avatars and embodied agents to form a socio-technical relationship between the user and the interactive UAS (Kim and Sundar 2012a; Pfeuffer et al. 2019). However, understanding how the design features of an interactive UAS affect the interaction with the user is crucial, as each design feature has a distinct impact on the user's perception. Based on existing knowledge, this thesis highlights the most widely used design features and analyzes their effects on behavior. The findings reveal important implications for future interactive UAS design. Third, this dissertation contributes to understanding how users interact with intelligent UASs. Intelligent UASs prioritize processing user and contextual information to adapt to the user's needs rather than focusing on an intensive interaction with the user (Maedche et al. 2016). Thus, intelligent UASs with emotional intelligence can provide people with task-oriented and emotional support, making them ideal for situations where interpersonal relationships are neglected, such as crowd working. Crowd workers frequently work independently without any significant interactions with other people (Jäger et al. 2019). In crowd work environments, traditional leader-employee relationships are usually not established, which can have a negative impact on employee motivation and performance (Cavazotte et al. 2012). Thus, this thesis examines the impact of an intelligent UAS with leadership and emotional capabilities on employee performance and enjoyment. The leadership capabilities of the intelligent UAS lead to an increase in enjoyment but a decrease in performance. The emotional capabilities of the intelligent UAS reduce the stimulating effect of leadership characteristics. Fourth, this dissertation contributes to understanding how users interact with anticipating UASs. Anticipating UASs are intelligent and interactive, providing users with task-related and emotional stimuli (Maedche et al. 2016). They also have advanced communication interfaces and can adapt to current situations and predict future events (Knote et al. 2018). Because of these advanced capabilities anticipating UASs enable collaborative work settings and often use anthropomorphic design cues to make the interaction more intuitive and comfortable (André et al. 2019). However, these anthropomorphic design cues can also raise expectations too high, leading to disappointment and rejection if they are not met (Bartneck et al. 2009; Mori 1970). To create a successful collaborative relationship between anticipating UASs and users, it is important to understand the impact of anthropomorphic design cues on the interaction and decision-making processes. This dissertation presents a theoretical model that explains the interaction between anthropomorphic anticipating UASs and users and an experimental procedure for empirical evaluation. The experiment design lays the groundwork for empirically testing the theoretical model in future research. To sum up, this dissertation contributes to information systems knowledge by improving understanding of the interaction between UASs and users in different application contexts. It develops new theoretical knowledge based on previous research and empirically evaluates user behavior to explain and predict it. In addition, this dissertation generates new knowledge by prototypically developing UASs and provides new insights for different classes of UASs. These insights can be used by researchers and practitioners to design more user-centric UASs and realize their potential benefits