Individual and Domain Adaptation in Sentence Planning for Dialogue

One of the biggest challenges in the development and deployment of spoken dialogue systems is the design of the spoken language generation module. This challenge arises from the need for the generator to adapt to many features of the dialogue domain, user population, and dialogue context. A promising approach is trainable generation, which uses general-purpose linguistic knowledge that is automatically adapted to the features of interest, such as the application domain, individual user, or user group. In this paper we present and evaluate a trainable sentence planner for providing restaurant information in the MATCH dialogue system. We show that trainable sentence planning can produce complex information presentations whose quality is comparable to the output of a template-based generator tuned to this domain. We also show that our method easily supports adapting the sentence planner to individuals, and that the individualized sentence planners generally perform better than models trained and tested on a population of individuals. Previous work has documented and utilized individual preferences for content selection, but to our knowledge, these results provide the first demonstration of individual preferences for sentence planning operations, affecting the content order, discourse structure and sentence structure of system responses. Finally, we evaluate the contribution of different feature sets, and show that, in our application, n-gram features often do as well as features based on higher-level linguistic representations

An information assistant system for the prevention of tunnel vision in crisis management

In the crisis management environment, tunnel vision is a set of bias in decision makers’ cognitive process which often leads to incorrect understanding of the real crisis situation, biased perception of information, and improper decisions. The tunnel vision phenomenon is a consequence of both the challenges in the task and the natural limitation in a human being’s cognitive process. An information assistant system is proposed with the purpose of preventing tunnel vision. The system serves as a platform for monitoring the on-going crisis event. All information goes through the system before arrives at the user. The system enhances the data quality, reduces the data quantity and presents the crisis information in a manner that prevents or repairs the user’s cognitive overload. While working with such a system, the users (crisis managers) are expected to be more likely to stay aware of the actual situation, stay open minded to possibilities, and make proper decisions

Fillers in Spoken Language Understanding: Computational and Psycholinguistic Perspectives

Disfluencies (i.e. interruptions in the regular flow of speech), are ubiquitous to spoken discourse. Fillers ("uh", "um") are disfluencies that occur the most frequently compared to other kinds of disfluencies. Yet, to the best of our knowledge, there isn't a resource that brings together the research perspectives influencing Spoken Language Understanding (SLU) on these speech events. This aim of this article is to synthesise a breadth of perspectives in a holistic way; i.e. from considering underlying (psycho)linguistic theory, to their annotation and consideration in Automatic Speech Recognition (ASR) and SLU systems, to lastly, their study from a generation standpoint. This article aims to present the perspectives in an approachable way to the SLU and Conversational AI community, and discuss moving forward, what we believe are the trends and challenges in each area.Comment: To appear in TAL Journa

The Role of Formulaic Language in the Creation of Grammar

Research in the field of Formulaic Language has shown it to be a very diverse phenomenon in both the form it takes and the functions it performs (e.g., Erma and Warren, 2000; Wray, 2002). The proposal made by Sinclair (1991) states that language as a system is organized according to two principles, the idiom principle\u27, which includes the use of all multi-word prefabricated sequences, and \u27the open choice principle,\u27 which covers word-for-word operations. Formulaic language is the embodiment of the idiom principle and constitutes the core of linguistic structure. Therefore, it must be subjected to scientific scrutiny from the variety of perspectives \u2013 typological, psycholinguistic, socio-pragmatic, and language acquisition. This dissertation reports on the percentage of formulaic sequences - prefabs - in spoken and written Russian; the distribution of prefab types across two spoken and four written genres, and their interaction with non-prefabricated language and the impact that prefabs have on the structure of a particular language type. Russian is the language typologically and structurally different from English. The main structural difference between English and Russian is that the Russian language has a free word order, wide inflectional system to code grammatical relations, and a satellite verb system. I hypothesize that these structural differences influence the quantity and the nature of formulaic sequences used in the language, the nature of alternation of prefabricated and non-prefabricated strings, and the preference of the speakers for one rather than the other aforementioned principles. The method applied in the analysis of Russian prefabs is developed by Erman and Warren (2000) and originally was applied to the analysis of the English texts. This dissertation seeks to address a methodological issue of applying this method to typologically different languages. It has been argued (Garcia and Florimon van Putte, 1989) that the fixedness of the English word order contributes to the co-occurrence of elements and the formation of formulaic sequences in English. In this case, formulaic language becomes a language-specific tendency pertaining to English, and not a universal mechanism for language storage, processing, production and use. The findings support the usage-based approaches driven by forces resulting from the frequency of use, discourse and communicative functions, grounded in the fine balance between the economy principle and the power of language creativity. The results of the study are used to draw implications for language processing and language modeling. As we continue to perfect the methods of identification, classification and analysis of formulaic sequences, we will be in a better position to describe not only the amount but the nature of formulaic language, its interaction with non-formulas, and the impact this alternation has on the linguistic structure as a whole. The current study investigates the nature of formulaic language in a free word order language. We seek to apply the method of identification, classification and analysis of prefabs, its interaction with each other and with non-formulaic language, as well as the estimation of choices made in producing spoken and written language. My dissertation results suggest that a free word order language uses at least as many prefabs as a fixed word order language. On average, in a free word order language like Russian 65% of spoken and 58% of written language is composed of multiword formulaic sequences. The results strengthen the hypothesis that the idiom principle is a mechanism of global linguistic organization and processing. The proportion and distribution or prefabs is less affected by language type than by spoken written medium distinction and genre variation. In addition, the results show that prefabs are frozen structures not amicable to standard syntactic transformations even in a free word order language. The results support the dual system of language processing, i.e., holistic and analytic, present in a free word order language

Reference and the facilitation of search in spatial domains

This is a pre-final version of the article, whose official publication is expected in the winter of 2013-14.Peer reviewedPreprin

Online discussion compensates for suboptimal timing of supportive information presentation in a digitally supported learning environment

This study used a sequential set-up to investigate the consecutive effects of timing of supportive information presentation (information before vs. information during the learning task clusters) in interactive digital learning materials (IDLMs) and type of collaboration (personal discussion vs. online discussion) in computer-supported collaborative learning (CSCL) on student knowledge construction. Students (N = 87) were first randomly assigned to the two information presentation conditions to work individually on a case-based assignment in IDLM. Students who received information during learning task clusters tended to show better results on knowledge construction than those who received information only before each cluster. The students within the two separate information presentation conditions were then randomly assigned to pairs to discuss the outcomes of their assignments under either the personal discussion or online discussion condition in CSCL. When supportive information had been presented before each learning task cluster, online discussion led to better results than personal discussion. When supportive information had been presented during the learning task clusters, however, the online and personal discussion conditions had no differential effect on knowledge construction. Online discussion in CSCL appeared to compensate for suboptimal timing of presentation of supportive information before the learning task clusters in IDLM

Towards an Indexical Model of Situated Language Comprehension for Cognitive Agents in Physical Worlds

We propose a computational model of situated language comprehension based on the Indexical Hypothesis that generates meaning representations by translating amodal linguistic symbols to modal representations of beliefs, knowledge, and experience external to the linguistic system. This Indexical Model incorporates multiple information sources, including perceptions, domain knowledge, and short-term and long-term experiences during comprehension. We show that exploiting diverse information sources can alleviate ambiguities that arise from contextual use of underspecific referring expressions and unexpressed argument alternations of verbs. The model is being used to support linguistic interactions in Rosie, an agent implemented in Soar that learns from instruction.Comment: Advances in Cognitive Systems 3 (2014