References to graphical objects in interactive multimodel queries

This thesis describes a computational model for interpreting natural language expressions in an interactive multimodal query system integrating both natural language text and graphic displays. The primary concern of the model is to interpret expressions that might involve graphical attributes, and expressions whose referents could be objects on the screen.Graphical objects on the screen are used to visualise entities in the application domain and their attributes (in short, domain entities and domain attributes). This is why graphical objects are treated as descriptions of those domain entities/attributes in the literature. However, graphical objects and their attributes are visible during the interaction, and are thus known by the participants of the interaction. Therefore, they themselves should be part of the mutual knowledge of the interaction.This poses some interesting problems in language processing. As part of the mutual knowledge, graphical attributes could be used in expressions, and graphical objects could be referred to by expressions. In consequence, there could be ambiguities about whether an attribute in an expression belongs to a graphical object or to a domain entity. There could also be ambiguities about whether the referent of an expression is a graphical object or a domain entity.The main contributions of this thesis consist of analysing the above ambiguities, de¬ signing, implementing and testing a computational model and a demonstration system for resolving these ambiguities. Firstly, a structure and corresponding terminology are set up, so these ambiguities can be clarified as ambiguities derived from referring to different databases, the screen or the application domain (source ambiguities). Secondly, a meaning representation language is designed which explicitly represents the information about which database an attribute/entity comes from. Several linguistic regularities inside and among referring expressions are described so that they can be used as heuristics in the ambiguity resolution. Thirdly, a computational model based on constraint satisfaction is constructed to resolve simultaneously some reference ambiguities and source ambiguities. Then, a demonstration system integrating natural language text and graphics is implemented, whose core is the computational model.This thesis ends with an evaluation of the computational model. It provides some concrete evidence about the advantages and disadvantages of the above approach

Dialogue and the machine: an interactional perspective on computer dialogue models, mediation and artifacts

The topic of this thesis is the notion of dialogue and how machines have not only influenced the development of our understanding of this fundamental human social activity but also the possibilities for engaging in mediated dialogue. In particular, the concern is with its adoption and distortion from a computational point of view. An interactional perspective is developed that provides insight into the problems and limitations of computer dialogue models, motivates the investigation of the achievement of dialogue mediated 'through' machines, and informs the conception and design of computer systems (or artifacts) that support the metaphor of dialogue 'with' machines. To motivate a reconstruction of the notion of dialogue and a different understanding of the status of machines in terms of action, a critical analysis of computer models of dialogue, concerning theory, data and implementation, is given. In general, computer models lack a consideration of interaction as a constitutive domain, assume the interchange model of dialogue, promote a sanitised view of data, and are a poor foundation for the design of machines that are to engage in dialogue-like behaviour with a user. An alternative interactional perspective is derived from hermeneutics and ethnomethodology in which it is argued that the machine is an intelligible - not intelligent - artifact, and communicative activity is circumstantial, situated and interactively constituted. Instead of reifying dialogue as the repeated exchange of discrete messages between isolated cognitive processors (the interchange model), dialogue is understood here to be the collection of practices in which parties are mutually engaged in coordinating communicative actions and achieving shared understanding out of the materials at hand. The empirical methodology of the thesis comes from conversation analysis and forms the basis for the investigation of the achievement of dialogue 'through' machines. A detailed audio-visual study of a particular computer-mediated communication modality is presented. Parties engaged in cooperatively constructing mutual orientation in dialogue (in a virtual dialogue space) were recorded and features of their conduct were rendered for analysis with the aid of a notation system specially developed for this study. The findings are that the computer-mediated dialogue activity is a skilled, interactive accomplishment in which dialogic presence, monitoring and participation are contingently created and maintained. An emergent transformation of the dialogue activity demonstrates the situated work of constructing participation, a process that is shaped by the dynamics of that activity. A brief study of copresent collaboration documents two further features: the embodiment of actions and their complementarity. The consequences of the interactional perspective and the empirical study for computer models and dialogue 'with' machines are discussed. Suggestions are also made about an alternative use of computer modelling for dialogue 'between' machines, and about the future of dialogue mediation and artifacts

VALUTARE I SISTEMI FLESSIBILI: UN APPROCCIO GLOBALE ALLA HCI

1995/1996VIII Ciclo1966Versione digitalizzata della tesi di dottorato cartacea