Helping, I Mean Assessing Psychiatric Communication: An Applicaton of Incremental Self-Repair Detection

18th SemDial Workshop on the Semantics and Pragmatics of Dialogue (DialWatt), 1-3 September 2014, Edinburgh, ScotlandSelf-repair is pervasive in dialogue, and models thereof have long been a focus of research, particularly for disfluency detection in speech recognition and spoken dialogue systems. However, the generality of such models across domains has received little attention. In this paper we investigate the application of an automatic incremental self-repair detection system, STIR, developed on the Switchboard corpus of telephone speech, to a new domain – psychiatric consultations. We find that word-level accuracy is reduced markedly by the differences in annotation schemes and transcription conventions between corpora, which has implications for the generalisability of all repair detection systems. However, overall rates of repair are detected accurately, promising a useful resource for clinical dialogue studies

Алгоритмы и программные средства автоматического определения речевых сбоев в звуковом сигнале

During automatic speech processing a number of problems appear, and among them are such as speech variation and different kinds of speech disfluences. In this article different types of speech disfluencies and their causes are presented, as well as the algorithm for their automatic detection based on the analysis of acoustical parameters. The method of cross-correlation was used to deteсt voiced hesitation phenomena and a method of band-filtering was used to detect unvoiced hesitation phenomena and artefacts. The experiments were performed on a specially collected corpus of spontaneous Russian map-task and appointment-task dialogs. Experiments showed that voiced hesitation phenomena are detected with 80% accuracy and devoiced hesitation phenomena and artefacts – with 66% accuracy.При автоматической обработке спонтанной речи возникает ряд трудностей, таких как вариативность речи или присутствие речевых сбоев различной природы. В статье рассматриваются различные виды речевых сбоев и причины их возникновения, а также представлен алгоритм их автоматического определения, основанный на анализе акустических параметров. Для выделения звонких хезитационных явлений использовался кросскорреляционный метод, а для выделения глухих хезитационных явлений – метод полосовой спектральной фильтрации. Эксперименты проводились на специально собранном корпусе спонтанной русской речи, состоящем из диалогов по описанию маршрута по карте и нахождению общего свободного времени по расписанию. Проведенные эксперименты показали, что звонкие хезитационные явления выделяются с точностью 80%, глухие хезитационные явления и дыхание - с точностью 66%

Computational Models of Miscommunication Phenomena

Miscommunication phenomena such as repair in dialogue are important indicators of the quality of communication. Automatic detection is therefore a key step toward tools that can characterize communication quality and thus help in applications from call center management to mental health monitoring. However, most existing computational linguistic approaches to these phenomena are unsuitable for general use in this way, and particularly for analyzing human–human dialogue: Although models of other-repair are common in human-computer dialogue systems, they tend to focus on specific phenomena (e.g., repair initiation by systems), missing the range of repair and repair initiation forms used by humans; and while self-repair models for speech recognition and understanding are advanced, they tend to focus on removal of “disfluent” material important for full understanding of the discourse contribution, and/or rely on domain-specific knowledge. We explain the requirements for more satisfactory models, including incrementality of processing and robustness to sparsity. We then describe models for self- and other-repair detection that meet these requirements (for the former, an adaptation of an existing repair model; for the latter, an adaptation of standard techniques) and investigate how they perform on datasets from a range of dialogue genres and domains, with promising results.EPSRC. Grant Number: EP/10383/1; Future and Emerging Technologies (FET). Grant Number: 611733; German Research Foundation (DFG). Grant Number: SCHL 845/5-1; Swedish Research Council (VR). Grant Numbers: 2016-0116, 2014-3

Mapping Nonverbal Communication into Social Status: Automatic Recognition of Journalists and Non-journalists in Radio News

This work shows how features accounting for nonverbal speaking characteristics can be used to map people into predefined categories. In particular, the results of this paper show that the speakers participating in radio broadcast news can be classified into journalists and non-journalists with an accuracy higher than 80 percent. The results of the approach proposed for this task is compared with the effectiveness of 16 human assessors performing the same task. The assessors do not understand the language of the data and are thus forced to use mostly nonverbal features. The results of the comparison suggest that the assessors and the automatic system have a similar performance

Strongly Incremental Repair Detection

Hough J, Purver M. Strongly Incremental Repair Detection. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: ACL; 2014: 78-89.We present STIR (STrongly Incremental Repair detection), a system that detects speech repairs and edit terms on transcripts incrementally with minimal latency. STIR uses information-theoretic measures from n-gram models as its principal decision features in a pipeline of classifiers detecting the different stages of repairs. Results on the Switchboard disfluency tagged corpus show utterance-final accuracy on a par with state-of-the-art incremental repair detection methods, but with better incremental accuracy, faster time-to-detection and less computational overhead. We evaluate its performance using incremental metrics and propose new repair processing evaluation standards

Analyse, modélisation, et détection automatique des disfluences dans le dialogue oral spontané contraint : le cas du Contrôle Aérien

The disfluencies are a frequently occurring phenomenon in any spontaneous speech production; it consists of the interruption of the normal flow of speech. They have given rise to numerous studies in Natural Language Processing. Indeed, their study and precise identification are essential, both from a theoretical and applicative perspective.However, most of the researches about the subject relate to everyday uses of language: “small talk” dialogs, requests for schedule, speeches, etc. But what about spontaneous speech production made in a restrained framework? To our knowledge, no study has ever been carried out in this context. However, we know that using a “language specialty” in the framework of a given task leads to specific behaviours.Our thesis work is devoted to the linguistic and computational study of disfluencies within such a framework. These dialogs concern air traffic control, which entails both pragmatic and linguistic constraints. We carry out an exhaustive study of disfluencies phenomena in this context. At first we conduct a subtle analysis of these phenomena. Then we model them to a level of abstraction, which allows us to obtain the patterns corresponding to the different configurations observed. Finally we propose a methodology for automatic processing. It consists of several algorithms to identify the different phenomena, even in the absence of explicit markers. It is integrated into a system of automatic processing of speech. Eventually, the methodology is validated on a corpus of 400 sentences.Les disfluences sont un phénomène apparaissant fréquemment dans toute production orale spontanée ; elles consistent en l'interruption du cours normal du discours. Elles ont donné lieu à de nombreuses études en Traitement Automatique du Langage Naturel. En effet, leur étude et leur identification précise sont primordiales, sur les plans théorique et applicatif.Cependant, la majorité des travaux de recherche sur le sujet portent sur des usages de langage quotidien : dialogues « à bâtons rompus », demandes d'horaire, discours, etc. Mais qu'en est-il des productions orales spontanées produites dans un cadre contraint ? Aucune étude n'a à notre connaissance été menée dans ce contexte. Or, on sait que l'utilisation d'une « langue de spécialité » dans le cadre d'une tâche donnée entraîne des comportements spécifiques.Notre travail de thèse est consacré à l'étude linguistique et informatique des disfluences dans un tel cadre. Il s'agit de dialogues de contrôle de trafic aérien, aux contraintes pragmatiques et linguistiques. Nous effectuons une étude exhaustive des phénomènes de disfluences dans ce contexte. Dans un premier temps nous procédons à l'analyse fine de ces phénomènes. Ensuite, nous les modélisons à un niveau de représentation abstrait, ce qui nous permet d'obtenir les patrons correspondant aux différentes configurations observées. Enfin nous proposons une méthodologie de traitement automatique. Celle-ci consiste en plusieurs algorithmes pour identifier les différents phénomènes, même en l'absence de marqueurs explicites. Elle est intégrée dans un système de traitement automatique de la parole. Enfin, la méthodologie est validée sur un corpus de 400 énoncés

Recognizing disfluencies in conversational speech

We present a system for modeling disfluency in conversational speech: repairs, fillers, and self-interruption points (IPs). For each sentence, candidate repair analyses are generated by a stochastic tree adjoining grammar (TAG) noisy-channel model. A probabilistic syntactic language model scores the fluency of each analysis, and a maximum-entropy model selects the most likely analysis given the language model score and other features. Fillers are detected independently via a small set of deterministic rules, and IPs are detected by combining the output of repair and filler detection modules. In the recent Rich Transcription Fall 2004 (RT-04F) blind evaluation, systems competed to detect these three forms of disfluency under two input conditions: a best-case scenario of manually transcribed words and a fully automatic case of automatic speech recognition (ASR) output. For all three tasks and on both types of input, our system was the top performer in the evaluation.8 page(s