Measuring Service Encounter Satisfaction with Customer Service Chatbots using Sentiment Analysis

Chatbots are software-based systems designed to interact with humans using text-based natural language and have attracted considerable interest in online service encounters. In this context, service providers face the challenge of measuring chatbot service encounter satisfaction (CSES), as most approaches are limited to post-interaction surveys that are rarely answered and often biased. Asa result, service providers cannot react quickly to service failures and dissatisfied customers. To address this challenge, we investigate the application of automated sentiment analysis methods as a proxy to measure CSES. Therefore, we first compare different sentiment analysis methods. Second, we investigate the relationship between objectively computed sentiment scores of dialogs and subjectively measured CSES values. Third, we evaluate whether this relationship also exists for utterance sequences throughout the dialog. The paper contributes by proposing and applying an automatic and objective approach to use sentiment scores as a proxy to measure CSES

Is Making Mistakes Human? On the Perception of Typing Errors in Chatbot Communication

The increasing application of Conversational Agents (CAs) changes the way customers and businesses interact during a service encounter. Research has shown that CA equipped with social cues (e.g., having a name, greeting users) stimulates the user to perceive the interaction as human-like, which can positively influence the overall experience. Specifically, social cues have shown to lead to increased customer satisfaction, perceived service quality, and trustworthiness in service encounters. However, many CAs are discontinued because of their limited conversational ability, which can lead to customer dissatisfaction. Nevertheless, making errors and mistakes can also be seen as a human characteristic (e.g., typing errors). Existing research on human-computer interfaces lacks in the area of CAs producing human-like errors and their perception in a service encounter situation. Therefore, we conducted a 2x2 online experiment with 228 participants on how CAs typing errors and CAs human-like behavior treatments influence user’s perception, including perceived service quality

Understanding chatbot service encounters:consumers’ satisfactory and dissatisfactory experiences

Abstract. The service industry keeps growing these years. Artificial intelligence (AI) has started to be used in the service industry gradually, and the service chatbot is an excellent example of this phenomenon. Many giants have applied chatbots to handle their consumer services, such as LATTJO from IKEA, Stylebot from Nike, and Siri from Apple. Understanding the advanced chatbot service experiences can help companies to optimize their chatbot services and improve their consumers’ satisfaction, which can bring them positive word-of-mouth, customer loyalty, re-purchase behavior, etc. However, chatbot services is an edge research area with limited studies about it. Thus, having the most advanced understanding of chatbot service experiences becomes particularly important. This study intends to fill this gap from chatbot service encounters’ perspective by understanding consumers’ satisfactory and unsatisfactory experiences with chatbots. Due to this study focuses on chatbot service encounters and online customer service experiences, a qualitative research method be applied because it enables data to be explainable and justifiable. Data collection methods consist of the critical incident technique (CIT) and the online focus group. In the end, 22 validity incidents were collected. Through data analysis, the author developed an incident sorting process and concluded eight types of chatbot service encounters within three groups by this process. The three groups are chatbot response to after-sales services, chatbot response to consumers’ needs, and unprompted chatbot actions. Moreover, 16 sources of different types of chatbot service encounters were found. Based on all the findings stated above, this study created an integrated framework for chatbot service encounters in online customer service experiences. In conclusion, this study develops theoretical contributions by developing the integrated framework, creating an incident sorting process, and finding the sources for different service encounters. Based on these findings, this study also provides some managerial implications that companies could use to manage their chatbot services

Towards Designing a Conversation Mining System for Customer Service Chatbots

Chatbots are increasingly used to provide customer service. However, despite technological advances, customer service chatbots frequently reach their limits in customer interactions. This is not immediately apparent to both chatbot operators (e.g., customer service managers) and chatbot developers because analyzing conversational data is difficult and labor-intensive. To address this problem, our ongoing design science research project aims to develop a conversation mining system for the automated analysis of customer-chatbot conversations. Based on the exploration of large dataset (N= 91,678 conversations) and six interviews with industry experts, we developed the backend of the system. Specifically, we identified and operationalized important criteria for evalu-ating conversations. Our next step will be the evaluation with industry experts. Ultimately, we aim to contribute to research and practice by providing design knowledge for conversation mining systems that leverage the treasure trove of data from customer-chatbot conversations to generate valuable insights for managers and developers

The effect of chatbot introduction on user satisfaction

Chatbots are becoming better and better, and their skills to perfectly imitate a human-being are almost impeccable. By means of a survey experiment this paper researches the effect of the introductory message of a chatbot on the end-user satisfaction. The experiment conditions involved a chatbot introducing itself as a chatbot, a chatbot not introducing itself, and a human being as control group. Satisfaction was measured using the constructs social presence, perceived humanness and service encounter satisfaction. Findings show that an undisclosed chatbot introduction yields a higher satisfaction with both the conversation itself and the conversational partner, while for the given advice and outcome of the conversation, users are indifferent between chatbots and real human beings

PREDICTIVE MODEL FOR CFPB CONSUMER COMPLAINTS

Within the dynamic and highly competitive financial industry, the timely and efficient resolution of customer complaints stands as a central challenge, particularly in the intricate domain of mortgage services. The traditional processes for handling these complaints have long been recognized as laborious and resource-intensive, a situation that financial institutions, including the esteemed Wells Fargo, are keen to improve. Currently, the industry largely relies on basic data analytics for identifying trends in customer complaints. However, this approach has its limitations, especially when dealing with complaints within the mortgage services domain. In response to this challenge, this research advocates the adoption of advanced predictive models as a groundbreaking solution. These models, powered by Random Forests hold the promise of transforming the management of mortgage-related complaints fundamentally. The Random Forests model, known for its capacity to analyze complex, non- linear relationships within data, is poised to revolutionize the prediction of customer complaint resolution outcomes. By analyzing a vast dataset from the Consumer Complaint Database, comprising 3,211,591 complaints spanning a decade, the model aspires whether a mortgage-related complaint will be swiftly resolved or require an extended resolution time. The anticipated outcomes of this endeavor encompass a transformative impact on the mortgage-related complaint resolution landscape. While this research is a pivotal step forward, broader complaint categories, and further refined predictive models could enhance the efficacy of complaint management and resolution processes

Uncovering lost potential : the shortcomings of DNBs chatbot

In 2018 DNB Bank ASA (DNB) launched their chatbot, Aino, an advanced virtual banking agent. Aino handles 55% of all the incoming chat traffic for DNBs Customer Center and is continuously being trained by AI trainers to increase the percentage of messages it can respond to. The former CEO of DNB, Rune Bjerke, stated in 2017 that by 2020, 80% of all incoming chat traffic would be handled by chatbots. However, to get closer to this target, DNBs AI trainers will have to make some priorities in the development process. The purpose of this study is to contribute to the decision-making process of which types of problems, and intents the AI trainers should prioritize to reduce DNBs costs. The data basis is conversational logs from conversations between customers of DNB and Aino, in addition to structural interviews with four DNB employees with significant knowledge of Aino. This thesis is a mixed-methods study that consists of both statistical analyses to determine group effect, structured interviews, quantitative content analysis, statistical analyses of chatlogs, as well as analysis of economical impact.I 2018 lanserte DNB Bank ASA (DNB) sin chatbot, Aino, en avansert virtuell bankagent. Aino håndterer 55% av all innkommende chat-trafikk for DNBs kundesenter og blir kontinuerlig opplært av AI-trenere for å øke prosentandelen av meldinger den kan svare på. Den tidligere konsernsjefen i DNB, Rune Bjerke, uttalte i 2017 at innen 2020 ville 80% av all innkommende chat-trafikk bli håndtert av chatbots. For å komme nærmere dette målet, vil DNBs AI-trenere imidlertid måtte gjøre noen prioriteringer i utviklingsprosessen. Hensikten med denne studien er å bidra til beslutningsprosessen for hvilke typer problemer, og intensjoner AI-trenerne bør prioritere for å redusere DNBs kostnader. Datagrunnlaget er samtalelogger fra samtaler mellom kunder av DNB og Aino, i tillegg til strukturerte intervjuer med fire DNB-ansatte med betydelig kunnskap om Aino. Denne oppgaven er et kombinasjonsstudie som består av både statistiske analyser for å bestemme gruppeeffekt, strukturerte intervjuer, kvantitativ innholdsanalyse, statistisk analyse av chatlogger, i tillegg til analyse av finansiell påvirkning.submittedVersionM-Ø

ExtraBot vs IntroBot: The Influence of Linguistic Cues on Communication Satisfaction

Conversational agents (CA) have emerged as a new type of dialogue systems, able to simulate human conversation. However, research suggests that current CAs fail to provide convincing interactions due to a lack of satisficing communication with users. To address this problem, we propose the idea of a personality adaptive CA that could enhance communication satisfaction during a user\u27s interaction experience. As personality differences manifest themselves in language cues, we investigate in an experiment, whether linguistic styles have an influence regarding a user\u27s communication satisfaction, when interacting with a CA. The results show that users perceive greater satisfaction when communicating with an extraverted CA (ExtraBot) than with an introverted CA (IntroBot). The outcomes of our study highlight that different linguistic styles can influence the course of the conversation and determine whether the user is satisfied with the communication and sees any value in the interaction with the CA