Towards Multi-Modal Interactions in Virtual Environments: A Case Study

We present research on visualization and interaction in a realistic model of an existing theatre. This existing ‘Muziek¬centrum’ offers its visitors information about performances by means of a yearly brochure. In addition, it is possible to get information at an information desk in the theatre (during office hours), to get information by phone (by talking to a human or by using IVR). The database of the theater holds the information that is available at the beginning of the ‘theatre season’. Our aim is to make this information more accessible by using multi-modal accessible multi-media web pages. A more general aim is to do research in the area of web-based services, in particu¬lar interactions in virtual environments

A Study of Interpreting Accreditation Testing Formatsin Australia

Advanced and affordable video conferencing technology has led to an increase in remote interpreting services via video, which has become a significant alternative to telephone and face-to-face interpreting. In keeping with this development, training providers are now incorporating video conference interpreting in their training. Video and audio resources are also increasingly being used as e-learning resources in online learning tools such as Blackboard and other university student learning portals. This has implications for the testing of interpreting skills, and the RMIT University Translating and Interpreting Program in Melbourne, Australia has started phasing in video assessment in examination and accreditation tests. In Australia, three test modes are commonly used: audio, video, and live-simulated tests. This article reports on a small-scale study that compared the three testing modes in terms of their potential impact on student examination results and also in terms of testing efficiency for training providers. Due to a lack of relevant research on the topic in the interpreting discipline, the discussion draws on relevant studies in diverse fields such as applied psychology, behavioral science, and musical performance. This study has implications for interpreter training strategies, for designing and administering interpreter assessment tests, and for resourcing in training programs

The role of perceived proximity in video-mediated communication

As technology for remote communication continues to advance and become more widespread, there is a need for research to attempt to understand the manner in which such technology may most suitably support human communication. This thesis describes a series of experiments which investigated the role of proximity within video-mediated communication. Proximity is one of the most fundamental forms of non-verbal communication used in a face-to-face interaction. Even subtle changes in interpersonal positioning are rich in information which people use to attempt to regulate the behaviour of themselves and others. At present it is unknown whether this type of non-verbal communication is preserved in video-mediated interactions. The aim of the present research project was to investigate whether impressions of proximity could be conveyed across a video link. In addition the research attempts to illuminate the physical parameters which may underpin the perception of proximity and to explore the impact upon users that any changes in perceived proximity may cause. The research uses a wide range of approaches to study the potential impact of proximity including analyses of the structure and content of dialogue, objective and subjective task outcome measures. The research demonstrates that perceptions of proximity can exist in a video-mediated environment and when they do, they can lead to differences in the communication behaviour of individuals communicating across a video link. It is found that when participants interact with a remote interlocutor who appears to be close, they tend to be more interactive. The research goes on to investigate the perceptual basis behind this effect and also considers how this relates to other variables which are known to affect communication, most notably familiarity

Multimodal and ubiquitous computing systems: supporting independent-living older users

We document the rationale and design of a multimodal interface to a pervasive/ubiquitous computing system that supports independent living by older people in their own homes. The Millennium Home system involves fitting a resident’s home with sensors – these sensors can be used to trigger sequences of interaction with the resident to warn them about dangerous events, or to check if they need external help. We draw lessons from the design process and conclude the paper with implications for the design of multimodal interfaces to ubiquitous systems developed for the elderly and in healthcare, as well as for more general ubiquitous computing applications

Quantifying mutual-understanding in dialogue

PhDThere are two components of communication that provide a natural index of mutual-understanding in dialogue. The first is Repair; the ways in which people detect and deal with problems with understanding. The second is Ellipsis/Anaphora; the use of expressions that depend directly on the accessibility of the local context for their interpretation. This thesis explores the use of these two phenomena in systematic comparative analyses of human-human dialogue under different task and media conditions. In order to do this it is necessary to a) develop reliable, valid protocols for coding the different Repair and Ellipsis/Anaphora phenomena b) establish their baseline patterns of distribution in conversation and c) model their basic statistical inter-relationships and their predictive value. Two new protocols for coding Repair and Ellipsis/Anaphora phenomena are presented and applied to two dialogue corpora, one of ordinary 'everyday' conversations and one of task-oriented dialogues. These data illustrate that there are significant differences in how understanding is created and negotiated across conditions. Repair is shown to be a ubiquitous feature in all dialogue. The goals of the speaker directly affect the type of Repair used. Giving instructions leads to a higher rate of self-editing; following instructions increases corrections and requests for clarification. Medium and familiarity also influence Repair; when eye contact is not possible there are a greater number of repeats and clarifications. Anaphora are used less frequently in task-oriented dialogue whereas types of Ellipsis increase. The use of Elliptical phrases that check, confirm or acknowledge is higher when there is no eye contact. Familiar pairs use more elliptical expressions, especially endophora and elliptical questions. Following instructions leads to greater use of elliptical (non-sentential) phrases. Medium, task and social norms all have a measureable effect on the components of dialogue that underpin mutual-understanding