The role of semantics in spoken dialogue translation systems

In this paper, we consider the role of semantics in the spoken dialogue translation systems. We begin by looking at some of the key properties of an existing spoken dialogue system, namely the sundial system which provides flight and train information over the telephone, and how these properties affect the design methodology and functionality of spoken translation systems. These properties include the effects of speech processing, designing the system to meet the needs of users, and an analysis model which clearly separates the linguistic, conceptual, pragmatic and task levels. In this model many task functionalities are dependent upon, and sometimes realizable by, the semantic and pragmatic analysis components. Central to this approach, is the use of underspecified semantic representations which are further specified as and when required by domain and/or task analysis. This model can be applied in the development of spoken translation systems with two important effects: monolingual semantic and pragmatic analysis can be carried out by processes independent of, but correlated with, the (translation) function of the system; and the main functions of the transfer processes is to further specifying the representations for the target language and to deal with mismatches between source and target language representations. We illustrate this approach with semantic analyses of German utterances required for translation in the verbmobil spoken dialogue translation system

The Verbmobil semantic formalism (Version 1.3)

This report describes the semantic formalism developed at Saarbrücken University as part of the Verbmobil project. The formalism is based upon DRT with additional functionality to meet the requirements on semantic construction arising from spoken dialogue translation. We define the syntax of the formalism and illustrate the semantic composition process in detail

A Study of Accomodation of Prosodic and Temporal Features in Spoken Dialogues in View of Speech Technology Applications

Inter-speaker accommodation is a well-known property of human speech and human interaction in general. Broadly it refers to the behavioural patterns of two (or more) interactants and the effect of the (verbal and non-verbal) behaviour of each to that of the other(s). Implementation of thisbehavior in spoken dialogue systems is desirable as an improvement on the naturalness of humanmachine interaction. However, traditional qualitative descriptions of accommodation phenomena do not provide sufficient information for such an implementation. Therefore, a quantitativedescription of inter-speaker accommodation is required. This thesis proposes a methodology of monitoring accommodation during a human or humancomputer dialogue, which utilizes a moving average filter over sequential frames for each speaker. These frames are time-aligned across the speakers, hence the name Time Aligned Moving Average (TAMA). Analysis of spontaneous human dialogue recordings by means of the TAMA methodology reveals ubiquitous accommodation of prosodic features (pitch, intensity and speech rate) across interlocutors, and allows for statistical (time series) modeling of the behaviour, in a way which is meaningful for implementation in spoken dialogue system (SDS) environments.In addition, a novel dialogue representation is proposed that provides an additional point of view to that of TAMA in monitoring accommodation of temporal features (inter-speaker pause length and overlap frequency). This representation is a percentage turn distribution of individual speakercontributions in a dialogue frame which circumvents strict attribution of speaker-turns, by considering both interlocutors as synchronously active. Both TAMA and turn distribution metrics indicate that correlation of average pause length and overlap frequency between speakers can be attributed to accommodation (a debated issue), and point to possible improvements in SDS “turntaking” behaviour. Although the findings of the prosodic and temporal analyses can directly inform SDS implementations, further work is required in order to describe inter-speaker accommodation sufficiently, as well as to develop an adequate testing platform for evaluating the magnitude ofperceived improvement in human-machine interaction. Therefore, this thesis constitutes a first step towards a convincingly useful implementation of accommodation in spoken dialogue systems

Evaluating Innovative Multimedia Customer Handling Systems

The subject of this thesis is tripartite communication between two human participants and a computer, where one human has access to the computer but the other does not. Its goal is to show that there is an intricate, triadic, relationship between the three. Previous investigations of such communication which have indicated that there are triadic relationships have concentrated on in depth study of the dialogues between the two human participants but have been informal and typically ethnomethodological. Formal or controlled experimental investigations have not taken account of the human-human dialogue, and as a consequence have not recognised the communications' triadic nature. This thesis redresses this dichotomy through extensive analysis of controlled experimental studies of simulated tripartite customer service situations. These analyses cover the three measures of perception, performance and process which have been used in the study of mediated human-human communication to build a complete model of the interactions. Particular attention was paid to the process of the human-human dialogues, using an adaptation of the Conversational Games Analysis (Kowtko, Isard and Doherty-Sneddon, 1992) coding scheme to gather in depth information about the ways in which participants used their utterances. The analyses of these measures provided empirical evidence that the informal studies were correct in proposing the relationship between the three participants in these communications were triadic. It was shown that not only broad differences in the computer's interaction style but also apparently small variations could have major impact on the human-human dialogue, and that the conditions of visibility between the three participants interacted strongly with these variations effects

Ubiquitous Technologies for Emotion Recognition

Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now, with the advent of wearable, mobile, and ubiquitous technologies, that we can aim to sense and recognize emotions, continuously and in real time. This book brings together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and the recognition of human emotions

"How do you know?":Everyday negotiations of expert authority

News headlines often argue that science has become just another opinion, that truth has died or that we have given up on facts. I employed Conversation Analysis in order to investigate this claim, basing my analysis on a corpus of health-related radio phone-in conversations, online conversations and coaching sessions. I studied interactions about Attention Deficit Hyperactivity Disorder as example of a controversial field to which both experiential and scientific knowledge claims are highly relevant. I compared the negotiations of expertise in interactions about ADHD with those in conversations about vaccination, aspartame, diabetes and ALS. I found that lay interactants collectively protect the symbols of science, even if their interpretation of what constitutes a scientific method is not necessarily correct. Speakers take care so as not to be seen as gullible or naïve. Both claiming scientific or factual expertise and disputing its authority when offered by others are an essential part of building a rational identity. What seems to indicate pure disregard for science often turns out to be an attempt to take a more scientific attitude. What is more, speakers' treatment of scientific expertise cannot be understood without taking the identity and accountability concerns into account that are relevant to the knowledge field in question. Because of this, one should be careful when speaking about the waxing or waning authority of scientific expertise in general. If it is important to interactants to demonstrate their critical attitude so as to show themselves not gullible or naive, this is likely to have consequences for the way in which they receive and respond to information. Whereas science communicators typically see distrust in science as the problem, interactants themselves treat being (seen as) trusting as the main interactional problem. An increased awareness that it is often an important identity concern for interactants to position themselves as epistemologically vigilant might contribute to the quality of science-society interactions. Based on this research, I argue that science has not become just another opinion. On the contrary; it seems as if we lack a language to discuss what matters, other than by disputing offered facts and claims to expertise. I suggest we stop deploring science’s alleged demotion to merely an opinion, and start wondering how we can solve - or even just discuss - the pressing problems of our time in a way that leaves room for other languages than science as we know it. Rather than lamenting the death of truth, we should (re)consider the consequences of our necessarily incomplete grasp of it