Measuring Service Encounter Satisfaction with Customer Service Chatbots using Sentiment Analysis

Chatbots are software-based systems designed to interact with humans using text-based natural language and have attracted considerable interest in online service encounters. In this context, service providers face the challenge of measuring chatbot service encounter satisfaction (CSES), as most approaches are limited to post-interaction surveys that are rarely answered and often biased. Asa result, service providers cannot react quickly to service failures and dissatisfied customers. To address this challenge, we investigate the application of automated sentiment analysis methods as a proxy to measure CSES. Therefore, we first compare different sentiment analysis methods. Second, we investigate the relationship between objectively computed sentiment scores of dialogs and subjectively measured CSES values. Third, we evaluate whether this relationship also exists for utterance sequences throughout the dialog. The paper contributes by proposing and applying an automatic and objective approach to use sentiment scores as a proxy to measure CSES

Affect and Social Processes in Online Communication- Experiments with an Affective Dialog System

Abstract—This paper presents an integrated view on a series of experiments conducted with an affective dialog system, applied as a tool in studies of emotions and social processes in online communication. The different realizations of the system are evaluated in three experimental setups in order to verify effects of affective profiles, as well as of fine-grained communication scenarios on users’ expressions of affective states, experienced emotional changes, and interaction patterns. Results demonstrate that the system applied in virtual reality settings matches a Wizard-of-Oz in terms of chatting enjoyment, dialog coherence and realism. Variants of the system’s affective profile significantly influence the rating of chatting enjoyment and an emotional connection. Self-reported emotional changes experienced by participants during an interaction with the system are in line with the type of applied profile. Analysis of interaction patterns, i.e., usage of particular dialog act classes, word categories, and textual expressions of affective states for different scenarios, demonstrates that a communication scenario for social sharing of emotions was successfully established. The experimental evidence provides valuable input for applications of affective dialog systems and strengthens them as valid tools for studying affect and social aspects in online communication. Index Terms—Affective dialog system, human-computer interaction, affect sensing and analysis, structuring affective interactions.

"How May I Help You?": Modeling Twitter Customer Service Conversations Using Fine-Grained Dialogue Acts

Given the increasing popularity of customer service dialogue on Twitter, analysis of conversation data is essential to understand trends in customer and agent behavior for the purpose of automating customer service interactions. In this work, we develop a novel taxonomy of fine-grained "dialogue acts" frequently observed in customer service, showcasing acts that are more suited to the domain than the more generic existing taxonomies. Using a sequential SVM-HMM model, we model conversation flow, predicting the dialogue act of a given turn in real-time. We characterize differences between customer and agent behavior in Twitter customer service conversations, and investigate the effect of testing our system on different customer service industries. Finally, we use a data-driven approach to predict important conversation outcomes: customer satisfaction, customer frustration, and overall problem resolution. We show that the type and location of certain dialogue acts in a conversation have a significant effect on the probability of desirable and undesirable outcomes, and present actionable rules based on our findings. The patterns and rules we derive can be used as guidelines for outcome-driven automated customer service platforms.Comment: 13 pages, 6 figures, IUI 201