MARIA: A universal, declarative, multiple abstraction-level language for service-oriented applications in ubiquitous environments

n/

User Interface Plasticity: Model Driven Engineering to the Limit!

Keynote paper.International audienceTen years ago, I introduced the notion of user interface plasticity to denote the capacity of user interfaces to adapt, or to be adapted, to the context of use while preserving usability. The Model Driven Engineering (MDE) approach, which was used for user interface generation since the early eighties in HCI, has recently been revived to address this complex problem. Although MDE has resulted in interesting and convincing results for conventional WIMP user interfaces, it has not fully demonstrated its theoretical promises yet. In this paper, we discuss how to push MDE to the limit in order to reconcile high-level modeling techniques with low-level programming in order to go beyond WIMP user interfaces

Interacção multimodal : contribuições para simplificar o desenvolvimento de aplicações

Doutoramento em Engenharia InformáticaA forma como interagimos com os dispositivos que nos rodeiam, no nosso diaa- dia, está a mudar constantemente, consequência do aparecimento de novas tecnologias e métodos que proporcionam melhores e mais aliciantes formas de interagir com as aplicações. No entanto, a integração destas tecnologias, para possibilitar a sua utilização alargada, coloca desafios significativos e requer, da parte de quem desenvolve, um conhecimento alargado das tecnologias envolvidas. Apesar de a literatura mais recente apresentar alguns avanços no suporte ao desenho e desenvolvimento de sistemas interactivos multimodais, vários aspectos chave têm ainda de ser resolvidos para que se atinja o seu real potencial. Entre estes aspectos, um exemplo relevante é o da dificuldade em desenvolver e integrar múltiplas modalidades de interacção. Neste trabalho, propomos, desenhamos e implementamos uma framework que permite um mais fácil desenvolvimento de interacção multimodal. A nossa proposta mantém as modalidades de interacção completamente separadas da aplicação, permitindo um desenvolvimento, independente de cada uma das partes. A framework proposta já inclui um conjunto de modalidades genéricas e módulos que podem ser usados em novas aplicações. De entre as modalidades genéricas, a modalidade de voz mereceu particular atenção, tendo em conta a relevância crescente da interacção por voz, por exemplo em cenários como AAL, e a complexidade associada ao seu desenvolvimento. Adicionalmente, a nossa proposta contempla ainda o suporte à gestão de aplicações multi-dispositivo e inclui um método e respectivo módulo para criar fusão entre eventos. O desenvolvimento da arquitectura e da framework ocorreu num contexto de I&D diversificado, incluindo vários projectos, cenários de aplicação e parceiros internacionais. A framework permitiu o desenho e desenvolvimento de um conjunto alargado de aplicações multimodais, sendo um exemplo digno de nota o assistente pessoal AALFred, do projecto PaeLife. Estas aplicações, por sua vez, serviram um contínuo melhoramento da framework, suportando a recolha iterativa de novos requisitos, e permitido demonstrar a sua versatilidade e capacidades.The way we interact with the devices around us, in everyday life, is constantly changing, boosted by emerging technologies and methods, providing better and more engaging ways to interact with applications. Nevertheless, the integration with these technologies, to enable their widespread use in current systems, presents a notable challenge and requires considerable knowhow from developers. While the recent literature has made some advances in supporting the design and development of multimodal interactive systems, several key aspects have yet to be addressed to enable its full potential. Among these, a relevant example is the difficulty to develop and integrate multiple interaction modalities. In this work, we propose, design and implement a framework enabling easier development of multimodal interaction. Our proposal fully decouples the interaction modalities from the application, allowing the separate development of each part. The proposed framework already includes a set of generic modalities and modules ready to be used in novel applications. Among the proposed generic modalities, the speech modality deserved particular attention, attending to the increasing relevance of speech interaction, for example in scenarios such as AAL, and the complexity behind its development. Additionally, our proposal also tackles the support for managing multi-device applications and includes a method and corresponding module to create fusion of events. The development of the architecture and framework profited from a rich R&D context including several projects, scenarios, and international partners. The framework successfully supported the design and development of a wide set of multimodal applications, a notable example being AALFred, the personal assistant of project PaeLife. These applications, in turn, served the continuous improvement of the framework by supporting the iterative collection of novel requirements, enabling the proposed framework to show its versatility and potential

User Interface Abstraction for enabling TV set based Inclusive Access to the Information Society

199 p.The television (TV) set is present in most homes worldwide, and is the most used Information and Communication Technology (ICT). Despite its large implantation in the market, the interactive services consumption on TV set is limited. This thesis focuses on overcoming the following limiting factors: (i) limited Human Computer Interaction and (ii) lack of considering user’s real life context in the digital television (dTV) service integration strategy. Making interactive services accessible to TV set’s large user base, and especially to the most vulnerable ones, is understood as the path to integrate the mankind with the information society. This thesis explores the use of user interface abstraction technologies to reach the introduced goals. The main contributions of this thesis are: (i) an approach to enable the universally accessible remote control of the TV set, (ii) an approach for the provision of universally accessible interactive services through TV sets, and (iii) an approach for the provision of universally accessible services in the TV user’s real life context. We have implemented the contributing approaches for different use cases, and we have evaluated them with real users, achieving good results

USER INTERFACES FOR MOBILE DEVICES: TECHNIQUES AND CASE STUDIES

The interactive capabilities of portable devices that are nowadays increasingly available, enable mobile computing in diverse contexts. However, in order to fully exploit the potentialities of such technologies and to let end users benefit from them, effective and usable techniques are still needed. In general, differences in capabilities, such as computational power and interaction resources, lead to an heterogeneity that is sometimes positively referred to as device diversity but also, negatively, as device fragmentation. When designing applications for mobile devices, besides general rules and principles of usability, developers cope with further constraints. Restricted capabilities, due to display size, input modality and computational power, imply important design and implementation choices in order to guarantee usability. In addition, when the application is likely to be used by subjects affected by some impairment, the system has also to comply with accessibility requirements. The aim of this dissertation is to propose and discuss examples of such techniques, aimed to support user interfaces on mobile devices, by tackling design, development and evaluation of specific solutions for portable terminals as well as for enabling interoperability across diverse devices (including desktops, handhelds, smartphones). Usefulness and usability aspects are taken into great consideration by the main research questions that drove the activities of the study. With respect the such questions, the three central chapters of the dissertation are respectively aimed at evaluating: hardware/software solutions for edutainment and accessibility in mobile museum guides, visualization strategies for mobile users visiting smart environments, and techniques for user interface migration across diverse devices in multi-user contexts. Motivations, design, implementation and evaluation about a number of solutions aimed to support several dimensions of user interfaces for mobile devices are widely discussed throughout the dissertation, and some findings are drawn. Each one of the prototypes described in the following chapters has been entirely developed within the research activities of the laboratory where the author performed his PhD. Most activities were related to tasks of international research projects and the organization of this dissertation reflects their evolution chronology

Cross-Device Taxonomy:Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices

Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with digital content. Research addressing the opportunities and challenges of interactions with multiple devices in concert is of continued focus in HCI research. To inform the future research agenda of this field, we contribute an analysis and taxonomy of a corpus of 510 papers in the cross- device computing domain. For both new and experienced researchers in the field we provide: an overview, historic trends and unified terminology of cross-device research; discussion of major and under-explored application areas; mapping of enabling technologies; synthesis of key interaction techniques spanning across multiple devices; and review of common evaluation strategies. We close with a discussion of open issues. Our taxonomy aims to create a unified terminology and common understanding for researchers in order to facilitate and stimulate future cross-device research

Proceedings of Cross-Surface 2015: Workshop on Interacting with Multi-Device Ecologies in the Wild

In this workshop, we reviewed and discussed opportunities, technical challenges and problems with cross-device interactions in real world interactive multi-surface and multi-device ecologies. We aim to bring together researchers and practitioners currently working on novel techniques for cross-surface interactions, focusing both on technical as well as interaction challenges for introducing these technologies into the wild, and highlighting opportunities for further research. The workshop will help to facilitate knowledge exchange on the inherent challenges of building robust and intuitive cross-surface interactions, identify application domains and enabling technologies for cross-surface interactions in the wild, and establish a research community to develop effective strategies for successful design of crossdevice interactions. Please find more details about the workshop, in the submitted proposal. The workshop was held in conjunction with the 2015 ACM International Conference on Interactive Tabletops and Surfaces, that took place from November 15 to 18 in Funchal in Madeira, Portugal