ANTLIMA -- A Listener Model with Mental Images

AI research concerning the connection between seeing and speaking mainly employs what is often called reference semantics. Applying this approach to the situation of a radio sports reporter, we have to coordinate the demand of referentially anchoring an utterance dealing with the visually perceived, and the demand for coherence of an utterance as part of a verbal interaction with somebody not situated in the same perceptual context. In consequence, we are led to the conception of a speaker anticipating the listeners' understanding by means of mental images which replace the percepts being described, and thus provide the referents for the audience. We present a system realizing this type of partner modeling, emphasizing mainly the reconstruction of the referents, i.e., of a mental image. Starting from the thesis that the audience expects the speaker to mean the most typical case of the described class of events or situations with respect to the communicated context, we explain a mechanism for representing and using typicality distributions of static spatial relations which is related to Herskovits' analytical framework. Extended to restrictions of speed and temporal duration, this mechanism also allows us to construct dynamic mental images corresponding to the referents of objective sports reports

Time-Dependent Generation of Minimal Sets of Spatial Descriptions

Talking about a location within a geometrically represented environment implies the translation of coordinates into suitable natural language expressions. Therefore, a reference semantics anchored in the given geometry has to be developed in order to be able to describe spatial relations between objects. Answering the question "Where is object X?" will lead to a large set of spatial propositions like "X is near Y", "X is to the right of Z" etc. Aiming for a description of the specified location which is both easily understood and highly informative for a human being, the elements of this set must be analysed with respect to the degrees of applicability, compatibility, uniqueness and facility to be memorized. Furthermore, the actual context as well as presumptions about the user's knowledge must be taken into account. Usually, in dialog situations it is a priori unknown how much time is given for the generation of an utterance. We therefore demonstrate a first approach of applying anyti..

Referring Locative Expressions in a Bounded-Optimal Localization Agent

Hitherto, in Artificial Intelligence the relations between visual and verbal space have mostly been examined assuming unlimited computational and natural resources and complete perceptual information. But considering dynamic environments like dialog situations, we find that response times are usually limited. Therefore, we should be able to generate some answer even before a perhaps complex computation has finished with the best possible result. These responses necessarily are suboptimal but by applying intelligent algorithms and architectures their quality will increase with the amount of time, the main resource in this case. Here we present some aspects of a boundedoptimal localization agent describing spatial configurations with referring locative expressions. Introduction In the last few years, the idea of anytime computation became more and more popular. This development was caused by the fact that we cannot assume that all information to an AI system is always complete and corr..

Natural Language Access to Intelligent Robots: Explaining Automatic Error Recovery

The increasing intelligence and autonomy of modern robot systems requires new and powerful man-machine-interfaces. For example, a robot's capability to autonomously recover from error situations corresponds with dynamic adjustments of robot plans during execution. This makes it difficult for an operator to predict and understand the behaviour of the machine. Explanations and descriptions of why and how a certain plan has been changed would impose the acceptance of robots. Descriptions are even more important in error situations which can not be handled autonomously. In this context, natural language is an effective tool of using robots in a more flexible manner. In this article, the joint efforts of the University of Karlsruhe and the University of Saarland to provide a natural language explanation for the error recovery of the autonomous, mobile, two-arm robot, KAMRO are reported

Spatial Information in Instructions and Questions to an Autonomous System

The task of supplying user-friendly access to an autonomous system using natural language is an interesting field of research which can be partitioned into access to autonomous robots and simulated agents. The perceptual information of such a system is to be related to language expressions in order to be able to talk about spatial configurations. The multiplicity of possible utterances is one of the main difficulties with natural language in general. In this paper, we concentrate on spatial expressions in different kinds of instructions and questions using anytime algorithms in order to consider resource limitations. 1 Introduction Autonomous systems are characterized by their sometimes unpredictable behavior. E.g., their error recovery feature can cause understanding problems for the user: he possibly does not know why the system acts in a certain way. Thus, intelligent systems should also have intelligent man-machine interfaces in order to improve cooperativeness [14]. Talking about..