Designing Hybrid Interactions through an Understanding of the Affordances of Physical and Digital Technologies

Two recent technological advances have extended the diversity of domains and social contexts of Human-Computer Interaction: the embedding of computing capabilities into physical hand-held objects, and the emergence of large interactive surfaces, such as tabletops and wall boards. Both interactive surfaces and small computational devices usually allow for direct and space-multiplex input, i.e., for the spatial coincidence of physical action and digital output, in multiple points simultaneously. Such a powerful combination opens novel opportunities for the design of what are considered as hybrid interactions in this work. This thesis explores the affordances of physical interaction as resources for interface design of such hybrid interactions. The hybrid systems that are elaborated in this work are envisioned to support specific social and physical contexts, such as collaborative cooking in a domestic kitchen, or collaborative creativity in a design process. In particular, different aspects of physicality characteristic of those specific domains are explored, with the aim of promoting skill transfer across domains. irst, different approaches to the design of space-multiplex, function-specific interfaces are considered and investigated. Such design approaches build on related work on Graspable User Interfaces and extend the design space to direct touch interfaces such as touch-sensitive surfaces, in different sizes and orientations (i.e., tablets, interactive tabletops, and walls). These approaches are instantiated in the design of several experience prototypes: These are evaluated in different settings to assess the contextual implications of integrating aspects of physicality in the design of the interface. Such implications are observed both at the pragmatic level of interaction (i.e., patterns of users' behaviors on first contact with the interface), as well as on user' subjective response. The results indicate that the context of interaction affects the perception of the affordances of the system, and that some qualities of physicality such as the 3D space of manipulation and relative haptic feedback can affect the feeling of engagement and control. Building on these findings, two controlled studies are conducted to observe more systematically the implications of integrating some of the qualities of physical interaction into the design of hybrid ones. The results indicate that, despite the fact that several aspects of physical interaction are mimicked in the interface, the interaction with digital media is quite different and seems to reveal existing mental models and expectations resulting from previous experience with the WIMP paradigm on the desktop PC

The role of tangible interaction to communicate tacit knowledge of built heritage

Meanings and values of built heritage vary from factual and explicit meanings which are relatively easy to present, to more tacit knowledge, which is typically more challenging to communicate due to its implicit and often abstract character. In this paper, we investigate how tangible interaction influences the communication of this tacit knowledge of built heritage, and howit affects the experience of visitors. Through a between-group comparative study in a real-world museum context, we examined howthe tangible characteristics of an interactive prototypemuseuminstallation influence how visitors perceive a particular story containing tacit heritage knowledge. The communicated story relates a historical journey in ancient Egypt to the physical and architectural characteristics of the entrance colonnade at the Djoser Complex in Saqqara. Our experimental conditions consisted of an interactive navigation (input) and a passive representation (output) components, ranging from traditional digital displays to fully tangible means of interaction. We report on our findings, which showed various differences and commonalities between our three experimental conditions. We conclude with a number of discussion points and design recommendations: (a) to strive for balance between navigation and representation modalities in terms of affordance and the required cognitive effort; (b) to take advantage of physical representation and grasping, such as conveying particular physical details and characteristics; and (c) to consider design aspects of embodiment, physical abstraction and materiality for future research or potential further development of communicating the meanings and values of heritage

Exploring Natural User Abstractions For Shared Perceptual Manipulator Task Modeling & Recovery

State-of-the-art domestic robot assistants are essentially autonomous mobile manipulators capable of exerting human-scale precision grasps. To maximize utility and economy, non-technical end-users would need to be nearly as efficient as trained roboticists in control and collaboration of manipulation task behaviors. However, it remains a significant challenge given that many WIMP-style tools require superficial proficiency in robotics, 3D graphics, and computer science for rapid task modeling and recovery. But research on robot-centric collaboration has garnered momentum in recent years; robots are now planning in partially observable environments that maintain geometries and semantic maps, presenting opportunities for non-experts to cooperatively control task behavior with autonomous-planning agents exploiting the knowledge. However, as autonomous systems are not immune to errors under perceptual difficulty, a human-in-the-loop is needed to bias autonomous-planning towards recovery conditions that resume the task and avoid similar errors. In this work, we explore interactive techniques allowing non-technical users to model task behaviors and perceive cooperatively with a service robot under robot-centric collaboration. We evaluate stylus and touch modalities that users can intuitively and effectively convey natural abstractions of high-level tasks, semantic revisions, and geometries about the world. Experiments are conducted with \u27pick-and-place\u27 tasks in an ideal \u27Blocks World\u27 environment using a Kinova JACO six degree-of-freedom manipulator. Possibilities for the architecture and interface are demonstrated with the following features; (1) Semantic \u27Object\u27 and \u27Location\u27 grounding that describe function and ambiguous geometries (2) Task specification with an unordered list of goal predicates, and (3) Guiding task recovery with implied scene geometries and trajectory via symmetry cues and configuration space abstraction. Empirical results from four user studies show our interface was much preferred than the control condition, demonstrating high learnability and ease-of-use that enable our non-technical participants to model complex tasks, provide effective recovery assistance, and teleoperative control

An illuminated workspace for iterative design

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Architecture, 2002.Includes bibliographical references (leaves [79]-81).The rapid development of digital technology has led to the widespread of virtual or CAD based design environments. However, these tools lack a means of combing physical and digital representations in order to support the iterative design process. The illuminated workspace provides a self-contained design environment in which a number of physical, computational, statistical, and visual representations can be combined in a single creative space. This thesis describes a landscape design workbench called Illuminating Clay that allows designers to intuitively manipulate free-form physical models and simultaneously interact with computational analysis of these physical representations. This thesis shows how the system supports multiple forms of representation - physical models, 2-D images, digital models, and dynamic simulations - in the early stage of design and allows for an easy transition between physical and digital modeling in the process of landscape design.by Yao Wang.S.M