Emotion Analysis and Dialogue Breakdown Detection in Dialogue of Chat Systems Based on Deep Neural Networks

In dialogues between robots or computers and humans, dialogue breakdown analysis is an important tool for achieving better chat dialogues. Conventional dialogue breakdown detection methods focus on semantic variance. Although these methods can detect dialogue breakdowns based on semantic gaps, they cannot always detect emotional breakdowns in dialogues. In chat dialogue systems, emotions are sometimes included in the utterances of the system when responding to the speaker. In this study, we detect emotions from utterances, analyze emotional changes, and use them as the dialogue breakdown feature. The proposed method estimates emotions by utterance unit and generates features by calculating the similarity of the emotions of the utterance and the emotions that have appeared in prior utterances. We employ deep neural networks using sentence distributed representation vectors as the feature. In an evaluation of experimental results, the proposed method achieved a higher dialogue breakdown detection rate when compared to the method using a sentence distributed representation vectors

Underreporting of errors in NLG output, and what to do about it

We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make. This is a problem, because mistakes are an important indicator of where systems should still be improved. If authors only report overall performance metrics, the research community is left in the dark about the specific weaknesses that are exhibited by `state-of-the-art' research. Next to quantifying the extent of error under-reporting, this position paper provides recommendations for error identification, analysis and reporting.Peer reviewe

Detecting and Classifying Malevolent Dialogue Responses: Taxonomy, Data and Methodology

Conversational interfaces are increasingly popular as a way of connecting people to information. Corpus-based conversational interfaces are able to generate more diverse and natural responses than template-based or retrieval-based agents. With their increased generative capacity of corpusbased conversational agents comes the need to classify and filter out malevolent responses that are inappropriate in terms of content and dialogue acts. Previous studies on the topic of recognizing and classifying inappropriate content are mostly focused on a certain category of malevolence or on single sentences instead of an entire dialogue. In this paper, we define the task of Malevolent Dialogue Response Detection and Classification (MDRDC). We make three contributions to advance research on this task. First, we present a Hierarchical Malevolent Dialogue Taxonomy (HMDT). Second, we create a labelled multi-turn dialogue dataset and formulate the MDRDC task as a hierarchical classification task over this taxonomy. Third, we apply stateof-the-art text classification methods to the MDRDC task and report on extensive experiments aimed at assessing the performance of these approaches.Comment: under review at JASIS

Confusion Modelling - An Estimation by Semantic Embeddings

Approaching the task of coherence assessment of a conversation from its negative perspective ‘confusion’ rather than coherence itself, has been attempted by very few research works. Influencing Embeddings to learn from similarity/dissimilarity measures such as distance, cosine similarity between two utterances will equip them with the semantics to differentiate a coherent and an incoherent conversation through the detection of negative entity, ‘confusion’. This research attempts to measure coherence of conversation between a human and a conversational agent by means of such semantic embeddings trained from scratch by an architecture centralising the learning from the distance between the embeddings. State of the art performance of general BERT’s embeddings and state of the art performance of ConveRT’s conversation specific embeddings in addition to the GLOVE embeddings are also tested upon the laid architecture. Confusion, being a more sensible entity, real human labelling performance is set as the baseline to evaluate the models. The base design resulted in not such a good performance against the human score but the pre-trained embeddings when plugged into the base architecture had performance boosts in a particular order from lowest to highest, through BERT, GLOVE and ConveRT. The intuition and the efficiency of the base conceptual design is proved of its success when the variant having the ConveRT embeddings plugged into the base design, outperformed the original ConveRT’s state of art performance on generating similarity scores. Though a performance comparable to real human performance was not achieved by the models, there witnessed a considerable overlapping between the ConveRT variant and the human scores which is really a great positive inference to be enjoyed as achieving human performance is always the state of art in any research domain. Also, from the results, this research joins the group of works claiming BERT to be unsuitable for conversation specific modelling and embedding works

Treat him as a normal baby: paediatrician's framing of parental responsibility as advice in the management of a genetic condition

Oral Presentation - Parallel Session 2: 2E Risk and Uncertainty/Ethics: no. 2E.4Parental responsibility in the management of genetic conditions has been the focus of both family-oriented interview-based research (e.g. Arribas-Ayllon et al. 2008; 2011) as well as real-life face-to-face genetic counselling research (Sarangi fc; Thomassen and Sarangi 2012). The current paper is an attempt to contribute to the latter tradition involving paediatricians and parents where parental responsibility is constitutive of professional advice. The genetic condition in question is G6PD deficiency (commonly known as favism), a mild hereditary disorder prevalent in Asia (Zayts and Sarangi 2013). We draw on 18 consultations in a maternal unit in Hong Kong (recruitment ongoing) where paediatricians communicate with mothers of newborns diagnosed with G6PD. We employ theme-oriented discourse analysis – comprising activity analysis and accounts analysis (Sarangi 2010) – to examine how the paediatricians frame their advice-giving trajectories – on to which elements of parental responsibility (in terms of future actions and moral selves) can be mapped. We show how 'causal responsibility' (Sarangi, fc) that concerns potential consequences of the mothers' actions in managing the condition emerges as a dominant thread in our data corpus. 'Causal [parental] responsibility' is embedded in the paediatrician's advice-giving trajectories which include, among other things, how to 'treat' these children, ranging from safeguards against certain medications and food to prevention of negative physiological scenarios (such as an acute hemolytic reaction). We examine closely the attendant discourse devices through which parental responsibility is framed, e.g., modalisation, contrast, character/event work. We conclude that, in terms of temporality, 'causal [parental] responsibility' is 'forward-looking' as the mothers' responsible actions can normalise the child’s immediate and future wellbeing.postprin