Towards a multimodal interaction space: Categorisation and applications

Based on many experiences of developing interactive systems by the authors, a framework for the description and analysis of interaction has been developed. The dimensions of this multimodal interaction space have been identified as sensory modalities, modes and levels of interaction. To illustrate and validate this framework, development of multimodal interaction styles is carried out and interactions in the real world are studied, going from theory to practice and back again. The paper describes the framework and two recent projects, one in the field of interactive architecture and another in the field of multimodal HCI research. Both projects use multiple modalities for interaction, particularly movement based interaction styles. © Springer-Verlag London Limited 2007

Very-High-Fidelity Prototyping for both Presentation and Dialogue Parts of Multimodal Interactive Systems

Also part of the book sub series: Programming and Software Engineering (LNPSE)International audienceThis paper presents a tool suite (made up of two previously unrelated approaches) for the engineering of multimodal Post-WIMP Interactive Systems. The first element of this integration is ICOM (a data-flow model dedicated to low-level input modelling) and its environment ICON which allows for editing and simulating ICOM models. The other element is ICOs (a formal description technique mainly dedicated to dialogue modelling) and its environment PetShop which allows for editing, simulating and verifying ICOs models. This paper shows how these two approaches have been integrated and how they support multimodal interactive systems engineering. We show on a classical rubber banding case study how these tools can be used for prototyping interactive systems. We also present in details how the changes in the interaction techniques impact the models at various levels of the software architecture

Very-High-Fidelity Prototyping for Both Presentation and Dialogue Parts of Multimodal Interactive Systems

Abstract. This paper presents a tool suite (made up of two previously unrelated approaches) for the engineering of multimodal Post-WIMP Interactive Systems. The first element of this integration is ICOM (a data-flow model dedicated to low-level input modelling) and its environment ICON which allows for editing and simulating ICOM models. The other element is ICOs (a formal description technique mainly dedicated to dialogue modelling) and its environment PetShop which allows for editing, simulating and verifying ICOs models. This paper shows how these two approaches have been integrated and how they support multimodal interactive systems engineering. We show on a classical rubber banding case study how these tools can be used for prototyping interactive systems. We also present in details how the changes in the interaction techniques impact the models at various levels of the software architecture

Improving task modelling to support the co-evolution of information systems and business processes

In business environments, information systems are required to change in response to changes in business processes. We refer to this process as co-evolution: the process of reciprocal change in a software system and the activities and goals of the system's users. This research focuses on improving task modelling techniques to support the co-evolution of information systems and business processes.We propose the Interaction Template approach to improve task modelling to support co-evolution. Interaction Templates make the task modelling process less tedious in both the design phase and the evolution phase of a system's lifecycle. Our approach adds data schemas and presentation components to task models, allowing us to build task models that adapt to data elements and parameters. Binding presentation components to task models allows us to generate user interface prototypes from task models. The generated user interface prototypes improve task model simulation and help make the effects of changes to business processes more clear. This thesis describes a study of the seven year evolution of a real world information system. Through this study, we gain a better understanding of how information systems evolve in response to the evolution of an organization's business processes. This thesis presents the Interaction Template approach, as well as a notation for specifying Interaction Templates. A prototype system supporting the Interaction Template approach is provided, along with examples demonstrating the approach