IOWAState: Models and Design Patterns for Identity-Aware User Interfaces Based on State Machines

International audienceThe emergence of interactive surfaces and technologies able to differentiate users allows the design and development of Identity-Aware (IA) interfaces, a new and richer set of user interfaces (UIs). Such user interfaces are able to adapt their behavior depending on who is interacting. However, existing implementations, mostly as software toolkits, are still ad-hoc and mostly based on existing GUI toolkits which are not designed to support user differentiation. The problem is that the development of IA interfaces is more complex than the development of traditional UIs and still requires extra programming efforts. To address these issues, we present a set of implementation models, named IOWAState models, to specify the behavior as state machines, the architecture and the components of IA interfaces. In addition, based on our IOWAState models and a classification of IA user interfaces, we detail a set of design patterns to implement the behavior of IA user interfaces

Assessing Feedback for Indirect Shared Interaction

Interaction is a key element in turning public displays into a platform for social interaction, making them more engaging and valuable. However, interactive features are still rare in public displays, due to the lack of generalised abstractions for incorporating interactivity. In our work, we explore to what extent the concept of interaction widget, which was so successful on desktop computers, could also be used as an abstraction for remote, shared interaction with public displays. A particular challenge is presenting input feedback in this shared, multi-user, and indirect interaction setting. In this paper, we present a study on the feedback mechanisms of these widgets, to determine if users are able to understand the results of interactions in single and multi-user settings. We have evaluated three feedback mechanisms and the results indicate that the general mechanism provides an appropriate sense of what is happening and could in fact provide general awareness of the interaction alternatives and current status, even in multi-user scenarios

Light on horizontal interactive surfaces: Input space for tabletop computing

In the last 25 years we have witnessed the rise and growth of interactive tabletop research, both in academic and in industrial settings. The rising demand for the digital support of human activities motivated the need to bring computational power to table surfaces. In this article, we review the state of the art of tabletop computing, highlighting core aspects that frame the input space of interactive tabletops: (a) developments in hardware technologies that have caused the proliferation of interactive horizontal surfaces and (b) issues related to new classes of interaction modalities (multitouch, tangible, and touchless). A classification is presented that aims to give a detailed view of the current development of this research area and define opportunities and challenges for novel touch- and gesture-based interactions between the human and the surrounding computational environment. © 2014 ACM.This work has been funded by Integra (Amper Sistemas and CDTI, Spanish Ministry of Science and Innovation) and TIPEx (TIN2010-19859-C03-01) projects and Programa de Becas y Ayudas para la Realización de Estudios Oficiales de Máster y Doctorado en la Universidad Carlos III de Madrid, 2010

Brainstorming und Mind-Mapping im Multi-Device-Kontext. Konzeption und prototypische Implementierung für Multi-Touch-Tabletop und Smartphone

Die vorliegende Arbeit beschreibt die Konzeption und prototypische Implemen­tierung einer Anwendung zur elektronischen Unterstützung von Brainstorming- und Mind-Mapping-Sitzungen an einem multitouchfähigen Tabletop mit Smart­phones. Während der Tabletop durch seine große, horizontal ausgerichtete Oberfläche die kollaborative Erstellung und Strukturierung von Ideen in Gruppen unterstützt, werden Smartphones einen zusätzlichen Eingabekanal zur Verfügung stellen sowie Individualarbeit fördern. Somit wird zunächst einerseits die Motivation für die Verwirklichung einer Anwendung zum Brainstorming und Mind- Mapping am Tabletop und andererseits das Potential einer Ergänzung eines sol­chen Systems durch zusätzliche private Geräte im Sinne der Multi-Device-Inter­aktion erläutert werden. Um eine geeignete theoretische Basis zu schaffen, wird darüber hinaus ein detaillierter Überblick über die Entwicklung der Tabletop­technologie in den letzten Jahren und den dabei zentralen Forschungs- und Pro­blemfeldern im Kontext der Entwicklung von Benutzerschnittstellen für Tabletopsysteme gegeben. Zudem werden schließlich auch im Rahmen der Arbeit relevante Aspekte der Multi-Device-Interaktion skizziert. Diesem theoretisch ausgerichteten Teil der Arbeit folgt schließlich eine Beschreibung der Entwicklung des Anwendungskonzepts und der dabei formulierten Anforderungen und Designzielen, welche in Form eines ersten, papierbasierten Prototypen visualisiert werden. Aufbauend auf diesen konzeptuellen Überlegungen wird schließlich die konkrete technische Umsetzung der Anwendung Multi/Touch/Device Mind­ Mapper in Form eines High-Level-Prototypen auf Basis des MT4j-Frameworks (Tabletop) und des Android-Betriebssystems (Smartphone) beleuchtet. Eine Dis­kussion dieses finalen Prototypen und erster praktischer Erfahrungen sowie ein Ausblick auf Erweiterungsmöglichkeiten des Systems und über den Rahmen der Arbeit hinaus gehende Fragestellungen schließen die Arbeit ab. Schlussendlich kann gezeigt werden, dass die im Rahmen des Anwendungskonzept definierten Anforderungen mit Hilfe der verwendeten Frameworks bis auf wenige Ausnahmen erfolgreich umgesetzt werden konnten. Darüber hinaus kann der Anwendung durch die aus dem Testbetrieb gewonnenen Erkenntnisse eine grundsätzli­ che Praktikabilität attestiert, ebenso konnten einige Ansatzpunkte für weitere Verbesserungen und Tests aufgedeckt werden

Real-time person re-identification for interactive environments

The work presented in this thesis was motivated by a vision of the future in which intelligent environments in public spaces such as galleries and museums, deliver useful and personalised services to people via natural interaction, that is, without the need for people to provide explicit instructions via tangible interfaces. Delivering the right services to the right people requires a means of biometrically identifying individuals and then re-identifying them as they move freely through the environment. Delivering the service they desire requires sensing their context, for example, sensing their location or proximity to resources. This thesis presents both a context-aware system and a person re-identification method. A tabletop display was designed and prototyped with an infrared person-sensing context function. In experimental evaluation it exhibited tracking performance comparable to other more complex systems. A real-time, viewpoint invariant, person re-identification method is proposed based on a novel set of Viewpoint Invariant Multi-modal (ViMM) feature descriptors collected from depth-sensing cameras. The method uses colour and a combination of anthropometric properties logged as a function of body orientation. A neural network classifier is used to perform re-identification

Supporting Reflection and Classroom Orchestration with Tangible Tabletops

Tangible tabletop systems have been extensively proven to be able to enhance participation and engagement as well as enable many exciting activities, particularly in the education domain. However, it remains unclear as to whether students really benefit from using them for tasks that require a high level of reflection. Moreover, most existing tangible tabletops are designed as stand-alone systems or devices. Increasingly, this design assumption is no longer sufficient, especially in realistic learning settings. Due to the technological evolution in schools, multiple activities, resources, and constraints in the classroom ecosystem are now involved in the learning process. The way teachers manage technology-enhanced classrooms and the involved activities and constraints in real-time, also known as classroom orchestration, is a crucial aspect for the materialization of reflection and learning. This thesis aims to explore how educational tangible tabletop systems affect reflection, how reflection and orchestration are related, and how we can support reflection and orchestration to improve learning. It presents the design, implementation, and evaluations of three tangible tabletop systems – the DockLamp, the TinkerLamp, and the TinkerLamp 2.0 – in different learning contexts. Our experience with these systems, both inside and outside of the laboratory, results in an insightful understanding of the impacts of tangible tabletops on learning and the conditions for their effective use as well as deployment. These findings can be beneficial to the researchers and designers of learning environments using tangible tabletop and similar interfaces

Factores para la diferenciación de usuarios sobre mesas interactivas. Estudio de un enfoque basado en interacciones simultáneas

[EN] Differentiating between users that interact simultaneously on a tabletop could be beneficial for collaborative tasks. Hence, the applications could support more effectively several territoriality-oriented features and they could provide a more efficient space management or a better presentation of the contents. This work presents a series of software factors which could be used to obtain such differentiation. Also, we design a novel algorithm for the controls segmentation according to their owner based on one of those factors: the simultaneous manipulation of controls. This is a potential differentiating factor that has remained unexplored so far, and in combination with others may become relevant to successfully accomplish such differentiation task. Basically it relies on the idea that users manipulate digital elements with a single hand, and therefore, if two controls are being used at the same time, they most likely belong to different users. On the generic algorithm, three different versions have been implemented that include several heuristics to address the problem. The comparison under a simulated experiment shows that the heuristic involving more knowledge on distances on user controls performed better according to different goodness functions. The results show this factor promising to further development and refinement of the approach by expanding it with other potential factors to eventually build a robust user differentiation subsystem[ES] Diferenciar entre los usuarios que están trabajando simultáneamente entorno a una superficie interactiva podría ser beneficioso para tareas colaborativas. Así, las aplicaciones podrían soportar de manera más efectiva ciertos aspectos orientados a la territorialidad, a la gestión eficiente del espacio disponible y a la representación de los contenidos. En este trabajo se comenta una serie de factores software que podrían usarse para realizar dicha diferenciación, y se diseña un algoritmo para la agrupación de controles según su dueño basado en uno de estos factores: la manipulación simultánea de controles. Este factor no ha sido explorado en detalle hasta este momento, y, mediante su combinación con otros factores, podría resultar relevante para el objetivo buscado. Básicamente, se basa en la idea de que los usuarios suelen manipular los elementos de interfaz con una sola mano y, por lo tanto, dos controles que están siendo manipulados al mismo tiempo es muy probable que pertenezcan a usuarios distintos. Se han realizado tres versiones diferentes del algoritmo anterior, cada una basada en una heurística diferente. La comparación de las mismas bajo un experimento simulado muestra que la heurística que involucra más información sobre la distancia física entre los controles se comporta mejor de acuerdo a una serie de métricas definidas. Los resultados se muestran relevantes para seguir explorando y refinando este enfoque, expandiéndolo con otros factores potenciales, y así conseguir construir un subsistema de diferenciación potente y robusto.García Sanjuan, F. (2013). Factores para la diferenciación de usuarios sobre mesas interactivas. Estudio de un enfoque basado en interacciones simultáneas. http://hdl.handle.net/10251/44577Archivo delegad

Designing for Cross-Device Interactions

Driven by technological advancements, we now own and operate an ever-growing number of digital devices, leading to an increased amount of digital data we produce, use, and maintain. However, while there is a substantial increase in computing power and availability of devices and data, many tasks we conduct with our devices are not well connected across multiple devices. We conduct our tasks sequentially instead of in parallel, while collaborative work across multiple devices is cumbersome to set up or simply not possible. To address these limitations, this thesis is concerned with cross-device computing. In particular it aims to conceptualise, prototype, and study interactions in cross-device computing. This thesis contributes to the field of Human-Computer Interaction (HCI)—and more specifically to the area of cross-device computing—in three ways: first, this work conceptualises previous work through a taxonomy of cross-device computing resulting in an in-depth understanding of the field, that identifies underexplored research areas, enabling the transfer of key insights into the design of interaction techniques. Second, three case studies were conducted that show how cross-device interactions can support curation work as well as augment users’ existing devices for individual and collaborative work. These case studies incorporate novel interaction techniques for supporting cross-device work. Third, through studying cross-device interactions and group collaboration, this thesis provides insights into how researchers can understand and evaluate multi- and cross-device interactions for individual and collaborative work. We provide a visualization and querying tool that facilitates interaction analysis of spatial measures and video recordings to facilitate such evaluations of cross-device work. Overall, the work in this thesis advances the field of cross-device computing with its taxonomy guiding research directions, novel interaction techniques and case studies demonstrating cross-device interactions for curation, and insights into and tools for effective evaluation of cross-device systems

Interactive Spaces Natural interfaces supporting gestures and manipulations in interactive spaces

This doctoral dissertation focuses on the development of interactive spaces through the use of natural interfaces based on gestures and manipulative actions. In the real world people use their senses to perceive the external environment and they use manipulations and gestures to explore the world around them, communicate and interact with other individuals. From this perspective the use of natural interfaces that exploit the human sensorial and explorative abilities helps filling the gap between physical and digital world. In the first part of this thesis we describe the work made for improving interfaces and devices for tangible, multi touch and free hand interactions. The idea is to design devices able to work also in uncontrolled environments, and in situations where control is mostly of the physical type where even the less experienced users can express their manipulative exploration and gesture communication abilities. We also analyze how it can be possible to mix these techniques to create an interactive space, specifically designed for teamwork where the natural interfaces are distributed in order to encourage collaboration. We then give some examples of how these interactive scenarios can host various types of applications facilitating, for instance, the exploration of 3D models, the enjoyment of multimedia contents and social interaction. Finally we discuss our results and put them in a wider context, focusing our attention particularly on how the proposed interfaces actually improve people’s lives and activities and the interactive spaces become a place of aggregation where we can pursue objectives that are both personal and shared with others