Measuring perceived empathy in dialogue systems

Dialogue systems, from Virtual Personal Assistants such as Siri, Cortana, and Alexa to state-of-the-art systems such as BlenderBot3 and ChatGPT, are already widely available, used in a variety of applications, and are increasingly part of many people’s lives. However, the task of enabling them to use empathetic language more convincingly is still an emerging research topic. Such systems generally make use of complex neural networks to learn the patterns of typical human language use, and the interactions in which the systems participate are usually mediated either via interactive text-based or speech-based interfaces. In human–human interaction, empathy has been shown to promote prosocial behaviour and improve interaction. In the context of dialogue systems, to advance the understanding of how perceptions of empathy affect interactions, it is necessary to bring greater clarity to how empathy is measured and assessed. Assessing the way dialogue systems create perceptions of empathy brings together a range of technological, psychological, and ethical considerations that merit greater scrutiny than they have received so far. However, there is currently no widely accepted evaluation method for determining the degree of empathy that any given system possesses (or, at least, appears to possess). Currently, different research teams use a variety of automated metrics, alongside different forms of subjective human assessment such as questionnaires, self-assessment measures and narrative engagement scales. This diversity of evaluation practice means that, given two DSs, it is usually impossible to determine which of them conveys the greater degree of empathy in its dialogic exchanges with human users. Acknowledging this problem, the present article provides an overview of how empathy is measured in human–human interactions and considers some of the ways it is currently measured in human–DS interactions. Finally, it introduces a novel third-person analytical framework, called the Empathy Scale for Human–Computer Communication (ESHCC), to support greater uniformity in how perceived empathy is measured during interactions with state-of-the-art DSs

Taking a stance: experimenting with deliberation in dialogue

PhDAbstract How do people manage disagreements in conversation? Previous studies of dialogue have shown that the interactional consequences of disagreement are not straightforward. Although often interpreted as face-threatening when performed in an unmitigated manner, disagreement can also encourage novel contributions. This thesis explores how systematically altering the presentation of someone’s stance influences the deliberative potential of a dialogue. A corpus analysis of ordinary conversations shows that exposed disagreement occurs rarely, but that speakers can signal a potentially adversarial position in a variety of other ways. One of the most interesting among these is the way people mark their rights to speak about something. Resources such as reported speech and prefacing incongruent content with discourse markers (e.g. ‘well’) can be important to the management of interpersonal factors. The idea that disagreement is problematic but also useful for deliberation is examined. Using a method that allows fine-grained manipulations of text based dialogues in real-time, agreement and disagreement fragments are inserted into a discussion dialogue. The findings show that inserting exposed disagreement violates the conventions of polite dialogue leading participants to put more effort into the production of their replies, and does not improve levels of deliberation. This raises the question of whether manipulating apparent degrees of speaker commitment might be more important for influencing the quality of deliberation. An experiment was devised which presented oppositional content with differing degrees of ‘knowingness’. The findings indicate that marking stance as knowing leads to less guarded exchanges, but does not increase deliberation. Conversely, framing statements as less knowing increases the likelihood that participants consider more alternative viewpoints, thus increasing the deliberative quality of a dialogue. Potential applications include training guidelines for professionals developing tools to support considered debate. Implications for computational argumentation studies include the importance of interpersonal dynamics and stance construction for formulating polite arguments

Scaffolding Young People's Participation in Public Service Evaluation through Designing a Digital Feedback Process

Young people facing marginalisation often rely on publicly funded services for support. Such services must include users in improving their provision, but often lack the processes and tools to facilitate this. The civic turn within HCI means that we are still tackling the complexities of community-based design research required to provide digital tools of relevance to public services. To address this, we worked with groups of young people to explore the design of a service evaluation process, supported by digital resources, intended to support marginalised youths to influence service delivery. Our findings demonstrate how the groups of young people participating in processes of service evaluation using our digital tools embraced the opportunity to express themselves. We also identify tensions from the social values underpinning the youth voluntary sector that impede their participation. We close by discussing challenges for community-based design and implications for digital technologies that facilitate the participation of marginalised young people in civic processes

Inferring Cultural Preference of Arts Audiences Through Twitter Data

CreativeWorks London, funded by the Arts and Humanities Research Council (AHRC)

Information sharing practices during the COVID-19 pandemic: A case study about face masks

This article contributes an empirical analysis of information sharing practices on Twitter relating to the use of face masks in the context of COVID-19. Behavioural changes, such as the use of face masks, are often influenced by people’s knowledge and perceptions, which in turn can be affected by the information available to them. Face masks were not recommended for use by the UK public at the beginning of the COVID-19 pandemic. Due to developments in scientific understanding, the guidance changed and by the end of 2020 they were mandatory on public transport and in shops. This research examines tweets in this longitudinal context and, therefore, provides novel insights into the dynamics of crisis communication in an ongoing crisis event with emerging scientific evidence. Specifically, analysis of the content of tweets, external resources most frequently shared, and users sharing information are considered. The conclusions contribute to developing understanding of the digital information ecology and provide practical insights for crisis communicators. Firstly, the analysis shows changes in the frequency of tweets about the topic correspond with key guidance and policy changes. These are, therefore, points in time official channels of information need to utilise the public’s information seeking and sharing practices. Secondly, due to changes in face mask guidance and policy, the current literature on digital information ecology is insufficient for capturing the dynamic nature of a long-term ongoing crisis event. Challenges can arise due to the prolonged circulation of out-of-date information, i.e. not strategic misinformation, nor “mis”-information at all, which can have serious ramifications for crisis communication practitioners. Thirdly, the role of traditional media and other journalism/broadcasting platforms in shaping conversations is evident, as is the potential for scientific organisations’ and individual people’s Twitter user accounts. This plurality of contributors needs to be acknowledged and understood to inform crisis communication strategies