A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion

Towards a crowdsourced solution for the authoring bottleneck in interactive narratives

Interactive Storytelling research has produced a wealth of technologies that can be employed to create personalised narrative experiences, in which the audience takes a participating rather than observing role. But so far this technology has not led to the production of large scale playable interactive story experiences that realise the ambitions of the field. One main reason for this state of affairs is the difficulty of authoring interactive stories, a task that requires describing a huge amount of story building blocks in a machine friendly fashion. This is not only technically and conceptually more challenging than traditional narrative authoring but also a scalability problem. This thesis examines the authoring bottleneck through a case study and a literature survey and advocates a solution based on crowdsourcing. Prior work has already shown that combining a large number of example stories collected from crowd workers with a system that merges these contributions into a single interactive story can be an effective way to reduce the authorial burden. As a refinement of such an approach, this thesis introduces the novel concept of Crowd Task Adaptation. It argues that in order to maximise the usefulness of the collected stories, a system should dynamically and intelligently analyse the corpus of collected stories and based on this analysis modify the tasks handed out to crowd workers. Two authoring systems, ENIGMA and CROSCAT, which show two radically different approaches of using the Crowd Task Adaptation paradigm have been implemented and are described in this thesis. While ENIGMA adapts tasks through a realtime dialog between crowd workers and the system that is based on what has been learned from previously collected stories, CROSCAT modifies the backstory given to crowd workers in order to optimise the distribution of branching points in the tree structure that combines all collected stories. Two experimental studies of crowdsourced authoring are also presented. They lead to guidelines on how to employ crowdsourced authoring effectively, but more importantly the results of one of the studies demonstrate the effectiveness of the Crowd Task Adaptation approach

Designing Human-Centered Collective Intelligence

Human-Centered Collective Intelligence (HCCI) is an emergent research area that seeks to bring together major research areas like machine learning, statistical modeling, information retrieval, market research, and software engineering to address challenges pertaining to deriving intelligent insights and solutions through the collaboration of several intelligent sensors, devices and data sources. An archetypal contextual CI scenario might be concerned with deriving affect-driven intelligence through multimodal emotion detection sources in a bid to determine the likability of one movie trailer over another. On the other hand, the key tenets to designing robust and evolutionary software and infrastructure architecture models to address cross-cutting quality concerns is of keen interest in the “Cloud” age of today. Some of the key quality concerns of interest in CI scenarios span the gamut of security and privacy, scalability, performance, fault-tolerance, and reliability. I present recent advances in CI system design with a focus on highlighting optimal solutions for the aforementioned cross-cutting concerns. I also describe a number of design challenges and a framework that I have determined to be critical to designing CI systems. With inspiration from machine learning, computational advertising, ubiquitous computing, and sociable robotics, this literature incorporates theories and concepts from various viewpoints to empower the collective intelligence engine, ZOEI, to discover affective state and emotional intent across multiple mediums. The discerned affective state is used in recommender systems among others to support content personalization. I dive into the design of optimal architectures that allow humans and intelligent systems to work collectively to solve complex problems. I present an evaluation of various studies that leverage the ZOEI framework to design collective intelligence

Sketching the vision of the Web of Debates

The exchange of comments, opinions, and arguments in blogs, forums, social media, wikis, and review websites has transformed the Web into a modern agora, a virtual place where all types of debates take place. This wealth of information remains mostly unexploited: due to its textual form, such information is difficult to automatically process and analyse in order to validate, evaluate, compare, combine with other types of information and make it actionable. Recent research in Machine Learning, Natural Language Processing, and Computational Argumentation has provided some solutions, which still cannot fully capture important aspects of online debates, such as various forms of unsound reasoning, arguments that do not follow a standard structure, information that is not explicitly expressed, and non-logical argumentation methods. Tackling these challenges would give immense added-value, as it would allow searching for, navigating through and analyzing online opinions and arguments, obtaining a better picture of the various debates for a well-intentioned user. Ultimately, it may lead to increased participation of Web users in democratic, dialogical interchange of arguments, more informed decisions by professionals and decision-makers, as well as to an easier identification of biased, misleading, or deceptive arguments. This paper presents the vision of the Web of Debates, a more human-centered version of the Web, which aims to unlock the potential of the abundance of argumentative information that currently exists online, offering its users a new generation of argument-based web services and tools that are tailored to their real needs

헬스케어를 위한 대화형 인공지능의 모사된 페르소나 디자인 및 사용자 경험 연구

학위논문(박사) -- 서울대학교대학원 : 인문대학 협동과정 인지과학전공, 2021.8. 이준환.디지털 헬스케어(Digital Healthcare) 기술의 발전은 일상 헬스케어 영역에서의 혁신을 주도 하고 있다. 이는 의학 전문가들의 정확한 진단, 질병의 치료를 도울 뿐만 아니라 사용자가 스스로 일상에서 자기관리를 할 수 있도록 돕는다. 디지털 헬스케어 기술의 대표적인 목표 중 하나는 효과적으로 헬스케어 서비스를 개인화 시키는 것인데, 이러한 측면에서 대화형 인공지능(Conversational AI)은 사용하기 쉽고 효율적인 비용으로 개인화된 서비스를 제공할 수 있기에 주목받고 있다. 기존의 개인화된 케어 서비스들의 경우는 대부분 의료기관의 질병치료 서비스들에 포함되었는데, 대화형 인공지능은 이러한 개인화된 케어 서비스의 영역을 일상에서의 질병 예방을 위한 관리로 확장하는데 기여한다. 일대일 대화를 통해 맞춤형 교육, 테라피, 그외의 관련 정보 등을 제공할 수 있다는 측면에서 일상 헬스케어에 적합한 디지털 헬스케어 기술로의 활용도가 높다. 이러한 이점으로 인해 다양한 역할을 가진 대화형 인공지능들의 개발이 이루어지고 있다. 그러나, 이러한 대화형 인공지능들에게 사용자의 선호도에 적합한 페르소나를 부여하는 연구는 드물게 이루어 지고 있다. 대화형 인공지능의 주요 기능인 자연어 기반 상호작용은 CASA 패러다임(CASA Paradigm)에서 제기하는 사용자가 시스템을 의인화하는 경향을 높인다. 때문에 페르소나에 대한 사용자의 선호도가 지속적인 대화형 인공지능의 사용과 몰입에 영향을 미친다. 또한 대화형 인공지능의 장기적인 사용을 위해서 적절한 사용자와의 사회적, 감정적 상호작용을 디자인 해 주어야 하는데, 인지된 페르소나에 대한 사용자의 선호도가 이 과정에도 유의미한 영향을 미친다. 때문에 지속적인 참여가 결과에 큰 영향을 미치는 일상 헬스케어 영역에서 대화형 인공지능을 활용하는데 개인화된 페르소나 디자인이 긍정적인 사용자 경험 및 사용자 건강 증진의 가능성을 높일 것으로 본 연구는 가정한다. 개인화된 페르소나 디자인을 위해 사용자와 현실에서 친밀한 관계에 있는 실존인물(호스트)의 페르소나를 대화형 인공지능에 적용하고 평가하는 것이 본 연구의 핵심적인 아아디어이다. 이를 검증하기 위해서 해당 학위 논문은 총 세 가지의 세부 연구를 포함한다. 첫째는 실존인물의 페르소나 중에서도 일상 건강관리에 적합한 호스트의 페르소나를 탐색하는 연구이다. 둘째는 호스트의 페르소나를 대화형 인공지능에 적용하기 위해 고려해야 할 언어적 요소들을 정의하는 연구이다. 마지막으로는 위의 과정을 통해 개발된 실존하는 인물의 페르소나를 가진 대화형 인공지능이 일상 헬스케어 영역에서 실제 효과를 보이는지를 평가하는 연구이다. 또한 해당 학위논문은 일련의 연구들에서 발견한 결과들을 바탕으로 사용자와 친밀한 관계에 있는 페르소나를 일상 헬스케어를 위한 대화형 인공지능에 적용할 때 고려해야할 디자인 함의점들을 도출하고 가이드라인을 제시한다.Advance in digital healthcare technologies has been leading a revolution in healthcare. It has been showing the enormous potential to improve medical professionals’ ability for accurate diagnosis, disease treatment, and the users’ daily self-care. Since the recent transformation of digital healthcare aims to provide effective personalized health services, Conversational AI (CA) is being highlighted as an easy-to-use and cost-effective means to deliver personalized services. Particularly, CA is gaining attention as a mean for personalized care by ingraining positive self-care behavior in a daily manner while previous methods for personalized care are focusing on the medical context. CA expands the boundary of personalized care by enabling one-to-one tailored conversation to deliver health education and healthcare therapies. Due to CA's opportunities as a method for personalized care, it has been implemented with various types of roles including CA for diagnosis, CA for prevention, and CA for therapy. However, there lacks study on the personalization of healthcare CA to meet user's preferences on the CA's persona. Even though the CASA paradigm has been applied to previous studies designing and evaluating the human-likeness of CA, few healthcare CAs personalize its human-like persona except some CAs for mental health therapy. Moreover, there exists the need to improve user experience by increasing social and emotional interaction between the user and the CA. Therefore, designing an acceptable and personalized persona of CA should be also considered to make users to be engaged in the healthcare task with the CA. In this manner, the thesis suggests an idea of applying the persona of the person who is in a close relationship with the user to the conversational CA for daily healthcare as a strategy for persona personalization. The main hypothesis is the idea of applying a close person's persona would improve user engagement. To investigate the hypothesis, the thesis explores if dynamics derived from the social relationship in the real world can be implemented to the relationship between the user and the CA with the persona of a close person. To explore opportunities and challenges of the research idea, series of studies were conducted to (1) explore appropriate host whose persona would be implemented to healthcare CA, (2) define linguistic characteristics to consider when applying the persona of a close person to the CA, and (3)implement CA with the persona of a close person to major lifestyle domains. Based on findings, the thesis provides design guidelines for healthcare CA with the persona of the real person who is in a close relationship with the user.Abstract 1 1 Introduction 12 2 Literature Review 18 2.1 Roles of CA in Healthcare 18 2.2 Personalization in Healthcare CA 23 2.3 Persona Design CA 25 2.4 Methods for Designing Chatbot’s Dialogue Style 30 2.4.1 Wizard of Oz Method 32 2.4.2 Analyzing Dialogue Data with NLP 33 2.4.3 Participatory Design 35 2.4.4 Crowdsourcing 37 3 Goal of the Study 39 4 Study 1. Exploring Candidate Persona for CA 43 4.1 Related Work 44 4.1.1 Need for Support in Daily Healthcare 44 4.1.2 Applying Persona to Text-based CA 45 4.2 Research Questions 47 4.3 Method 48 4.3.1 Wizard of Oz Study 49 4.3.2 Survey Measurement 52 4.3.3 Post Interview 54 4.3.4 Analysis 54 4.4 Results 55 4.4.1 System Acceptance 56 4.4.2 Perceived Trustfulness and Perceived Intimacy 57 4.4.3 Predictive Power of Corresponding Variables 58 4.4.4 Linguistic Factors Affecting User Perception 58 4.5 Implications 60 5 Study 2. Linguistic Characteristics to Consider When Applying Close Person’s Persona to a Text-based Agent 63 5.1 Related Work 64 5.1.1 Linguistic Characteristics and Persona Perception 64 5.1.2 Language Component 66 5.2 Research Questions 68 5.3 Method 69 5.3.1 Modified Wizard of Oz Study 69 5.3.2 Survey 72 5.4 Results 73 5.4.1 Linguistic Characteristics 73 5.4.2 Priority of Linguistic Characteristics 80 5.4.3 Differences between language Component 82 5.5 Implications 82 6 Study3.Implementation on Lifestyle Domains 85 6.1 Related Work 86 6.1.1 Family as Effective Healthcare Provider 86 6.1.2 Chatbots Promoting Healthy Lifestyle 87 6.2 Research questions 94 6.3 Implementing Persona of Family Member 95 6.3.1 Domains of Implementation 96 6.3.2 Measurements Used in the Study 97 6.4 Experiment 1: Food Journaling Chatbot 100 6.4.1 Method 100 6.4.2 Results 111 6.5 Experiment 2: Physical Activity Intervention 128 6.5.1 Method 131 6.5.2 Results 140 6.6 Experiment 3: Chatbot for Coping Stress 149 6.6.1 Method 151 6.6.2 Results 158 6.7 Implications from Domain Experiments 169 6.7.1 Comparing User Experience 170 6.7.2 Comparing User Perception 174 6.7.3 Implications from Study 3 183 7 Discussion 192 7.1 Design Guidelines 193 7.2 Ethical Considerations 200 7.3 Limitations 206 8 Conclusion 210 References 212 Appendix 252 국문초록 262박

The effects of user assistance systems on user perception and behavior

The rapid development of information technology (IT) is changing how people approach and interact with IT systems (Maedche et al. 2016). IT systems can increasingly support people in performing ever more complex tasks (Vtyurina and Fourney 2018). However, people's cognitive abilities have not evolved as quickly as technology (Maedche et al. 2016). Thus, different external factors (e.g., complexity or uncertainty) and internal conditions (e.g., cognitive load or stress) reduce decision quality (Acciarini et al. 2021; Caputo 2013; Hilbert 2012). User-assistance systems (UASs) can help to compensate for human weaknesses and cope with new challenges. UASs aim to improve the user's cognition and capabilities, benefiting individuals, organizations, and society. To achieve this goal, UASs collect, prepare, aggregate, analyze information, and communicate results according to user preferences (Maedche et al. 2019). This support can relieve users and improve the quality of decision-making. Using UASs offers many benefits but requires successful interaction between the user and the UAS. However, this interaction introduces social and technical challenges, such as loss of control or reduced explainability, which can affect user trust and willingness to use the UAS (Maedche et al. 2019). To realize the benefits, UASs must be developed based on an understanding and incorporation of users' needs. Users and UASs are part of a socio-technical system to complete a specific task (Maedche et al. 2019). To create a benefit from the interaction, it is necessary to understand the interaction within the socio-technical system, i.e., the interaction between the user, UAS, and task, and to align the different components. For this reason, this dissertation aims to extend the existing knowledge on UAS design by better understanding the effects and mechanisms during the interaction between UASs and users in different application contexts. Therefore, theory and findings from different disciplines are combined and new theoretical knowledge is derived. In addition, data is collected and analyzed to validate the new theoretical knowledge empirically. The findings can be used to reduce adaptation barriers and realize a positive outcome. Overall this dissertation addresses the four classes of UASs presented by Maedche et al. (2016): basic UASs, interactive UASs, intelligent UASs, and anticipating UASs. First, this dissertation contributes to understanding how users interact with basic UASs. Basic UASs do not process contextual information and interact little with the user (Maedche et al. 2016). This behavior makes basic UASs suitable for application contexts, such as social media, where little interaction is desired. Social media is primarily used for entertainment and focuses on content consumption (Moravec et al. 2018). As a result, social media has become an essential source of news but also a target for fake news, with negative consequences for individuals and society (Clarke et al. 2021; Laato et al. 2020). Thus, this thesis presents two approaches to how basic UASs can be used to reduce the negative influence of fake news. Firstly, basic UASs can provide interventions by warning users of questionable content and providing verified information but the order in which the intervention elements are displayed influences the fake news perception. The intervention elements should be displayed after the fake news story to achieve an efficient intervention. Secondly, basic UASs can provide social norms to motivate users to report fake news and thereby stop the spread of fake news. However, social norms should be used carefully, as they can backfire and reduce the willingness to report fake news. Second, this dissertation contributes to understanding how users interact with interactive UASs. Interactive UASs incorporate limited information from the application context but focus on close interaction with the user to achieve a specific goal or behavior (Maedche et al. 2016). Typical goals include more physical activity, a healthier diet, and less tobacco and alcohol consumption to prevent disease and premature death (World Health Organization 2020). To increase goal achievement, previous researchers often utilize digital human representations (DHRs) such as avatars and embodied agents to form a socio-technical relationship between the user and the interactive UAS (Kim and Sundar 2012a; Pfeuffer et al. 2019). However, understanding how the design features of an interactive UAS affect the interaction with the user is crucial, as each design feature has a distinct impact on the user's perception. Based on existing knowledge, this thesis highlights the most widely used design features and analyzes their effects on behavior. The findings reveal important implications for future interactive UAS design. Third, this dissertation contributes to understanding how users interact with intelligent UASs. Intelligent UASs prioritize processing user and contextual information to adapt to the user's needs rather than focusing on an intensive interaction with the user (Maedche et al. 2016). Thus, intelligent UASs with emotional intelligence can provide people with task-oriented and emotional support, making them ideal for situations where interpersonal relationships are neglected, such as crowd working. Crowd workers frequently work independently without any significant interactions with other people (Jäger et al. 2019). In crowd work environments, traditional leader-employee relationships are usually not established, which can have a negative impact on employee motivation and performance (Cavazotte et al. 2012). Thus, this thesis examines the impact of an intelligent UAS with leadership and emotional capabilities on employee performance and enjoyment. The leadership capabilities of the intelligent UAS lead to an increase in enjoyment but a decrease in performance. The emotional capabilities of the intelligent UAS reduce the stimulating effect of leadership characteristics. Fourth, this dissertation contributes to understanding how users interact with anticipating UASs. Anticipating UASs are intelligent and interactive, providing users with task-related and emotional stimuli (Maedche et al. 2016). They also have advanced communication interfaces and can adapt to current situations and predict future events (Knote et al. 2018). Because of these advanced capabilities anticipating UASs enable collaborative work settings and often use anthropomorphic design cues to make the interaction more intuitive and comfortable (André et al. 2019). However, these anthropomorphic design cues can also raise expectations too high, leading to disappointment and rejection if they are not met (Bartneck et al. 2009; Mori 1970). To create a successful collaborative relationship between anticipating UASs and users, it is important to understand the impact of anthropomorphic design cues on the interaction and decision-making processes. This dissertation presents a theoretical model that explains the interaction between anthropomorphic anticipating UASs and users and an experimental procedure for empirical evaluation. The experiment design lays the groundwork for empirically testing the theoretical model in future research. To sum up, this dissertation contributes to information systems knowledge by improving understanding of the interaction between UASs and users in different application contexts. It develops new theoretical knowledge based on previous research and empirically evaluates user behavior to explain and predict it. In addition, this dissertation generates new knowledge by prototypically developing UASs and provides new insights for different classes of UASs. These insights can be used by researchers and practitioners to design more user-centric UASs and realize their potential benefits

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl

Digital Food Marketing to Children and Adolescents: Problematic Practices and Policy Interventions

Examines trends in digital marketing to youth that uses "immersive" techniques, social media, behavioral profiling, location targeting and mobile marketing, and neuroscience methods. Recommends principles for regulating inappropriate advertising to youth

Countering Misinformation via Emotional Response Generation

The proliferation of misinformation on social media platforms (SMPs) poses a significant danger to public health, social cohesion and ultimately democracy. Previous research has shown how social correction can be an effective way to curb misinformation, by engaging directly in a constructive dialogue with users who spread -- often in good faith -- misleading messages. Although professional fact-checkers are crucial to debunking viral claims, they usually do not engage in conversations on social media. Thereby, significant effort has been made to automate the use of fact-checker material in social correction; however, no previous work has tried to integrate it with the style and pragmatics that are commonly employed in social media communication. To fill this gap, we present VerMouth, the first large-scale dataset comprising roughly 12 thousand claim-response pairs (linked to debunking articles), accounting for both SMP-style and basic emotions, two factors which have a significant role in misinformation credibility and spreading. To collect this dataset we used a technique based on an author-reviewer pipeline, which efficiently combines LLMs and human annotators to obtain high-quality data. We also provide comprehensive experiments showing how models trained on our proposed dataset have significant improvements in terms of output quality and generalization capabilities.Comment: Accepted to EMNLP 2023 main conferenc