An MDE-based framework to support the development of Mixed Interactive Systems

International audienceIn the domain of Human Computer Interaction (HCI), recent advances in sensors, communication technologies, miniaturization and computing capabilities have led to new and advanced forms of interaction. Among them, Mixed Interactive Systems (MIS), form a class of interactive systems that comprises augmented reality, tangible interfaces and ambient computing; MIS aim to take advantage of physical and digital worlds to promote a more transparent integration of interactive systems with the user's environment. Due to the constant change of technologies and the multiplicity of these interaction forms, specific development approaches have been developed. As a result, numerous taxonomies, frameworks, API and models have emerged, each one covering a specific and limited aspect of the development of MIS. To support a coherent use of these multiple development resources and contribute to the increasing popularity of MIS, we have developed a framework based on Model-Driven Engineering. The goal is to take advantage of Model-Driven Engineering (MDE) standards, methodology and tools to support the manipulation of complementary Domain Specific Languages (DSL), to organize and link the use of different design and implementation resources, and to ensure a rationalized implementation based on design choices. In this paper, we first summarize existing uses of MDE in HCI before focusing on five major benefits MDE can provide in a MIS development context. We then detail which MDE tools and resources support these benefits and thus form the pillars of the success of an MDE-based MIS development approach. Based on this analysis, we introduce our framework, called Guide-Me, and illustrate its use through a case study. This framework includes two design models. Model transformations are also included to link one model to another; as a result the frameworks coverage extends from the earliest design step to a software component-based prototyping platform. A toolset based on Eclipse Modeling Framework (EMF) that supports the use of the framework is also presented. We finally assess our MDE-based development process for MIS based on the five major MDE benefits for MIS

Interaction Tangible sur Table, définitions et modèles

International audienceIn recent years, tangible user interfaces, which imply interactions performed with one or several objects, gain more and more interest in research in Human-Computer Interaction (HCI). The tangible object represents a subject or an action. It acts on the system, as an action in classical user interfaces (e.g,. GUI). Interaction on a table, which is a common furniture in everyday life and used in multiple activities (desktop, coffee table, kitchen table, etc.), opens a new way for research and development in HCI. In this article, we present definitions, models, and key issues elicited from the literature that enable understanding and reasoning about the couple within an interactive system. Then, we propose a framework that allows to characterize applications supported by the couple in a domain-independent manner.Depuis quelques années les interfaces tangibles impliquant des interactions réalisées via un objet (ou plusieurs) prennent de plus en plus d’importance dans les recherches en interaction homme-machine. L’objet tangible représente un sujet ou une action ; l’objet agit sur le système, telle une action sur une interface « classique ». L’interaction sur table, c’est-à-dire sur un meuble présent dans la vie courante et utilisé à diverses fins (bureau, table à manger, table de salon, table bar, etc.), ouvre un champ nouveau de recherche et de développement. La mise en exergue, issue de l’état de l’art, des définitions, modèles et problématiques, permet d’abord d’appréhender le couple (table, objet tangible) au sein d’un système interactif. Puis, nous proposons un cadre qui permet de positionner des applications mettant en oeuvre le couple (table, objet tangible). Le cadre est décrit de manière à être utilisé pour positionner des applications indépendamment du domaine

Interfaces tangíveis em simuladores veiculares: componentes para um protocolo de avaliação de usabilidade

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro de Comunicação e Expressão, Programa de Pós-Graduação em Design e Expressão Gráfica, Florianópolis, 2016.A partir do final dos anos 1990, inovações tecnológicas relacionadas à interação de usuários e sistemas digitais proporcionaram o advento de um novo campo de estudos em Design. Trata-se das Interfaces Tangíveis, que podem ser definidas como o uso de objetos e ambientes físicos para emissão de comandos e feedback em interações digitais, onde a usabilidade possui parâmetros distintos das interfaces gráficas. Nesse contexto, esta pesquisa teve por objetivo a identificação de componentes para um protocolo de avaliação de usabilidade de Interfaces Tangíveis naturais e de incorporação ambiental, isto é, que consistem no design de todo um ambiente para promover uma interação intuitiva. A abordagem metodológica utilizada é de natureza qualitativa e exploratória, consistindo em: a) Revisão sistemática na literatura relativa a Interfaces Tangíveis e b) Testes de usabilidade de um produto que proporciona interação tangível ambiental (um simulador de motocicleta). Como resultados foram identificados componentes para um protocolo de avaliação: seis dimensões gerais de análise de usabilidade identificadas pela literatura especializada (Usuário, Instrutor, Ambiente Físico, Artefato Real, Cenário Intangível e Relação Real-Virtual), sendo cada dimensão constituída por fatores específicos oriundos dos testes de usabilidade realizados.Abstract : With the emergence, from the late 1990s, of technological innovations related to the interaction of users and systems, it has been the advent of a new field of study in Design: the tangible user interfaces. Such interfaces can be defined as the use of objects and physical environments for issuing commands and feedback in digital interactions where usability has different parameters of the graphical interfaces. This research aims to stablish components for a protocol for assessment of tangible interface, specially those of the environmental type, ie, consisting of the design of an entire physical environment with the purpose digital interaction. The methodological approach for this research is qualitative and exploratory, consisting of: a) Systematic reviewing of the literature on tangible interfaces; b) Usability testing of a product that provides tangible environmental interaction (a motorcycle simulator). As result is had been identified components for a protocol for assessment which includes general criteria for usability described in the specialized literature and specific items identified by tests

Couplage de techniques d'interaction avancées avec des environnements virtuels 3D interactifs

The work of this thesis fit on the boundary between two complementary research areas: the field of 3D Virtual Environment (3DVE) from Computer Graphics (CG) and Virtual Reality (RV) and the field of Human-Computer Interaction (HCI). They rely on three assessments. Firstly, we observe that 3DVE takes more importance in our daily life (video games, serious games, e-commerce, museums, through the web and on mobile devices). Secondly, HCI becomes more complex with the emergence of advance forms of interaction like ambient computing, tangible interaction or spatial and gestural interactions. This evolution goes along with a diversification of devices (3D mouse, the Wiimote, the Kinect or the Leap Motion). Thirdly, the design of interaction techniques with 3DVE brings up some different considerations taken into account by the communities in the field of 3DVE and HCI. Therefore, take advantage of the latest considerations of EV3D communities (metaphors, quality of 3D interaction) and HCI (advance forms of interaction) results in the need to develop the coupling between advance forms of interaction techniques and EV3D. In this context, the objective of this thesis work is to contribute to the development of interactive 3D environments in multiple situations, including large audience situations. The approach we developed aimed to create a bridge between 3D and HCI design considerations. We intend to improve the coupling of advance interaction techniques with interactive 3D virtual environment. After analyzing methods for the design of interaction techniques for 3DVE, a first contribution consists in a design framework of 3D interaction. This framework aggregates design issues stem from 3D and HCI and help the designer to identify several elements involve in the coupling of interaction with a 3DVE. This design framework is based on the analysis of the links between user tasks and elements of the 3DVE impacted by these tasks. In order to precisely characterize each link, we have introduced the 3DIM (3D Interaction Modality) notation that describes the characteristics of the different elements constituting a "3D Interaction Modality" for the accomplishment of a user's interaction task in a 3DVE. We have grouped these elements into six blocks: the user, the physical actions, the physical objects, the input devices, the 3D behaviors and the 3D interactive objects. We complete our framework with analytical properties for guiding the designer and provide descriptive, evaluative and generative power at our conceptual model of advanced interaction techniques for 3DVE. Collaborating with the Museum of "Le Pic du Midi" observatory in France, we used our framework to design and implement tangible interaction and technique based on smartphone usage. Museum visitors can use these techniques in a 3DVE of the "Telescope Bernard Lyot" to explore and understand its functioning. We have conducted three users' studies in order to explore the design space of using a smartphone to interact with 3DVE. We used the smartphone in different ways to navigate, select and manipulate a 3D object displayed on a large remote screen. We explored several design solutions with a smartphone as a touch device, as a tangible object or mid-air interaction around the device.Les travaux de cette thèse s'inscrivent à la frontière entre deux domaines de recherche complémentaires : le domaine des Environnements Virtuels 3D (EV3D) issus de l'Informatique Graphique (IG) et de la Réalité Virtuelle (RV) et le domaine de l'Interaction Homme-Machine (IHM). Ils s'appuient sur trois constats. D'une part, on observe une place grandissante des EV3D dans notre quotidien (jeux vidéo, jeux sérieux, e-commerce, dans les musées, à travers le web et sur les dispositifs mobiles). D'autre part, les IHM se complexifient notamment avec l'apparition de formes avancées d'interaction comme l'informatique ambiante, l'interaction tangible, ou encore l'interaction spatiale et gestuelle, et s'accompagne d'une diversification des dispositifs d'interaction (souris 3D, la Wiimote, la Kinect, le Leap Motion). Enfin, la conception de techniques d'interaction avancées avec des EV3D fait apparaitre des considérations différentes prises en compte par les communautés des domaines EV3D et IHM. Par conséquent, tirer profit des considérations les plus récentes des communautés EV3D (métaphores, qualité de l'interaction 3D) et IHM (formes avancées de technique d'interaction) se traduit par un besoin de développer le couplage entre formes avancées de technique d'interaction et EV3D. Dans ce contexte, l'objectif de ces travaux de thèse est de contribuer à l'essor des environnements 3D interactifs dans de multiples situations, et notamment des situations grand public, en adoptant une approche visant à faire converger les approches 3D et IHM pour mieux établir le couplage de Techniques d'Interaction Avancées avec des Environnements Virtuels 3D Interactifs. Après une analyse des méthodes de conception de techniques d'interaction pour les EV3D, une première contribution de nos travaux consiste en un cadre de conception de l'interaction 3D. En y agrégeant les problématiques issues de la 3D et de l'IHM, ce cadre de conception permet d'identifier les différents éléments de couplages impliqués lors d'une interaction avec un EV3D. Il se base sur l'analyse des liens entre les tâches utilisateurs et les éléments de l'EV3D impactés par ces tâches. Afin de caractériser finement chaque lien, nous avons introduit la notation 3DIM (3D Interaction Modality) qui décrit les caractéristiques des différents éléments constituant une " modalité d'interaction 3D " permettant la réalisation d'une tâche d'interaction de l'utilisateur dans un EV3D. Nous avons regroupé ces éléments en 6 blocs : l'utilisateur, les actions physiques, les objets physiques manipulés, les dispositifs utilisés, les comportements 3D et les éléments 3D. Nous complétons ce cadre conceptuel par des propriétés analytiques qui permettent de guider le concepteur et procurent ainsi un caractère descriptif, évaluatif et génératif à notre modèle conceptuel de techniques d'interaction avancées pour des EV3D. Dans la cadre d'une collaboration avec le musée de l'observatoire du Pic du Midi, une mise en œuvre de ce cadre nous a conduit à concevoir et développer des techniques d'interaction tangibles et basées smartphone. Ces techniques sont utilisées par les visiteurs du musée dans un EV3D représentatif du Télescope Bernard Lyot pour l'explorer et comprendre son fonctionnement. Nous avons mené trois évaluations utilisateur afin d'explorer l'usage d'un smartphone utilisé de trois manière différentes : comme dispositif tactile, comme un objet tangible ou comme support pour une interaction gestuelle autour du dispositif pour naviguer, sélectionner ou manipuler un objet 3D dans un EV3D affiché sur un grand écran distant