Establishing the design knowledge for emerging interaction platforms

While awaiting a variety of innovative interactive products and services to appear in the market in the near future such as interactive tabletops, interactive TVs, public multi-touch walls, and other embedded appliances, this paper calls for preparation for the arrival of such interactive platforms based on their interactivity. We advocate studying, understanding and establishing the foundation for interaction characteristics and affordances and design implications for these platforms which we know will soon emerge and penetrate our everyday lives. We review some of the archetypal interaction platform categories of the future and highlight the current status of the design knowledge-base accumulated to date and the current rate of growth for each of these. We use example designs illustrating design issues and considerations based on the authors’ 12-year experience in pioneering novel applications in various forms and styles

Rethinking 'multi-user': an in-the-wild study of how groups approach a walk-up-and-use tabletop interface

Multi-touch tabletops have been much heralded as an innovative technology that can facilitate new ways of group working. However, there is little evidence of these materialising outside of research lab settings. We present the findings of a 5-week in-the-wild study examining how a shared planning application – designed to run on a walk-up- and-use tabletop – was used when placed in a tourist information centre. We describe how groups approached, congregated and interacted with it and the social interactions that took place – noting how they were quite different from research findings describing the ways groups work around a tabletop in lab settings. We discuss the implications of such situated group work for designing collaborative tabletop applications for use in public settings

Behavioral patterns of individuals and groups during co-located collaboration on large, high-resolution displays

Collaboration among multiple users on large screens leads to complicated behavior patterns and group dynamics. To gain a deeper understanding of collaboration on vertical, large, high-resolution screens, this dissertation builds on previous research and gains novel insights through new observational studies. Among other things, the collected results reveal new patterns of collaborative coupling, suggest that territorial behavior is less critical than shown in previous research, and demonstrate that workspace awareness can also negatively affect the effectiveness of individual users

Virtual Valcamonica: collaborative exploration of prehistoric petroglyphs and their surrounding environment in multi-user virtual reality

In this paper, we present a novel, multi-user, virtual reality environment for the interactive, collaborative 3D analysis of large 3D scans and the technical advancements that were necessary to build it: a multi-view rendering system for large 3D point clouds, a suitable display infrastructure and a suite of collaborative 3D interaction techniques. The cultural heritage site of Valcamonica in Italy with its large collection of prehistoric rock-art served as an exemplary use case for evaluation. The results show that our output-sensitive level-of-detail rendering system is capable of visualizing a 3D dataset with an aggregate size of more than 14 billion points at interactive frame rates. The system design in this exemplar application results from close exchange with a small group of potential users: archaeologists with expertise in rock-art and allows them to explore the prehistoric art and its spatial context with highly realistic appearance. A set of dedicated interaction techniques was developed to facilitate collaborative visual analysis. A multi-display workspace supports the immediate comparison of geographically distributed artifacts. An expert review of the final demonstrator confirmed the potential for added value in rock-art research and the usability of our collaborative interaction techniques

Reducing the effect of network delay on tightly-coupled interaction

Tightly-coupled interaction is shared work in which each person’s actions immediately and continuously influence the actions of others. Tightly-coupled interaction is a hallmark of expert behaviour in face-to-face activity, but becomes extremely difficult to accomplish in distributed groupware. The main cause of this difficulty is network delay – even amounts as small as 100ms – that disrupts people’s ability to synchronize their actions with another person. To reduce the effects of delay on tightly-coupled interaction, I introduce a new technique called Feedback-Feedthrough Synchronization (FFS). FFS causes visual feedback from an action to occur at approximately the same time for both the local and the remote person, preventing one person from getting ahead of the other in the coordinated interaction. I tested the effects of FFS on group performance in several delay conditions, and my study showed that FFS substantially improved users’ performance: accuracy was significantly improved at all levels of delay, and without noticeable increase in perceived effort or frustration. Techniques like FFS that support the requirements of tightly-coupled interaction provide new means for improving the usability of groupware that operates on real-world networks

Group reaching over digital tabletops with digital arm embodiments

In almost all collaborative tabletop tasks, groups require coordinated access to the shared objects on the table’s surface. The physical social norms of close-proximity interactions built up over years of interacting around other physical bodies cause people to avoid interfering with other people (e.g., avoiding grabbing the same object simultaneously). However, some digital tabletop situations require the use of indirect input (e.g., when using mice, and when supporting remote users). With indirect input, people are no longer physically embodied during their reaching gestures, so most systems provide digital embodiments – visual representations of each person – to provide feedback to both the person who is reaching and to the other group members. Tabletop arm embodiments have been shown to better support group interactions than simple visual designs, providing awareness of actions to the group. However, researchers and digital tabletop designers know little of how the design of digital arm embodiments affects the fundamental group tabletop interaction of reaching for objects. Therefore, in this thesis, we evaluate how people coordinate their interactions over digital tabletops when using different types of embodiments. Specifically, in a series of studies, we investigate how the visual design (what they look like) and interaction design (how they work) of digital arm embodiments affects a group’s coordinative behaviours in an open- ended parallel tabletop task. We evaluated visual factors of size, transparency, and realism (through pictures and videos of physical arms), as well as interaction factors of input and augmentations (feedback of interactions), in both a co-located and distributed environment. We found that the visual design had little effect on a group’s ability to coordinate access to shared tabletop items, that embodiment augmentations are useful to support group coordinative actions, and that there are large differences when the person is not physically co-present. Our results demonstrate an initial exploration into the design of digital arm embodiments, providing design guidelines for future researchers and designers to use when designing the next generation of shared digital spaces

3D Multi-user interactive visualization with a shared large-scale display

When the multiple users interact with a virtual environment on a largescale display there are several issues that need to be addressed to facilitate the interaction. In the thesis, three main topics for collaborative visualization are discussed; display setup, interactive visualization, and visual fatigue. The problems that the author is trying to address in this thesis are how multiple users can interact with a shared large-scale display depending on the display setups and how they can interact with the shared visualization in a way that doesn’t lead to visual fatigue. The first user study (Chapter 3) explores the display setups for multi-user interaction with a shared large-display. The author describes the design of the three main display setups (a shared view, a split screen, and a split screen with navigation information) and a demonstration using these setups. The user study found that the split screen and the split screen with navigation information can improve users’ confidence and reduce frustration level and are more preferred than a shared view. However, a shared view can still provide effective interaction and collaboration and the display setups cannot have a large impact on usability and workload. From the first study, the author employed a shared view for multi-user interactive visualization with a shared large-scale display due to the advantages of the shared view. To improve interactive visualization with a shared view for multiple users, the author designed and conducted the second user study (Chapter 4). A conventional interaction technique, the mean tracking method, was not effective for more than three users. In order to overcome the limitation of the current multi-user interactive visualization techniques, two interactive visualization techniques (the Object Shift Technique and Activity-based Weighted Mean Tracking method) were developed and were evaluated in the second user study. The Object Shift Technique translates the virtual objects in the opposite direction of movement of the Point of View (PoV) and the Activity-based Weighted Mean Tracking method assigns the higher weight to active users in comparison with stationary users to determine the location of the PoV. The results of the user study showed that these techniques can support collaboration, improve interactivity, and provide similar visual discomfort compared to the conventional method. The third study (Chapter 5) describes how to reduce visual fatigue for 3D stereoscopic visualization with a single point of view (PoV). When multiple users interact with 3D stereoscopic VR using multi-user interactive visualization techniques and they are close to the virtual objects, they can perceive 3D visual fatigue from the large disparity. To reduce the 3D visual fatigue, an Adaptive Interpupillary Distance (Adaptive IPD) adjustment technique was developed. To evaluate the Adaptive IPD method, the author compared to traditional 3D stereoscopic and the monoscopic visualization techniques. Through the user experiments, the author was able to confirm that the proposed method can reduce visual discomfort, yet maintain compelling depth perception as the result provided the most preferable 3D stereoscopic visualization experience. For these studies, the author developed a software framework and designed a set of experiments (Chapter 6). The framework architecture that contains the three main ideas are described. A demonstration application for multidimensional decision making was developed using the framework. The primary contributions of this thesis include a literature review of multiuser interaction with a shared large-scale display, deeper insights into three display setups for multi-user interaction, development of the Object Shift Techniques, the Activity-based Weighted Mean Tracking method, and the Adaptive Interpupillary Distance Adjustment technique, the evaluation of the three novel interaction techniques, development of a framework for supporting a multi-user interaction with a shared large-scale display and its application to multi-dimensional decision making VR system

Improving digital object handoff using the space above the table

Object handoff – that is, passing an object or tool to another person – is an extremely common activity in collaborative tabletop work. On digital tables, object handoff is typically accomplished by sliding the object on the table surface – but surface-only interactions can be slow and error-prone, particularly when there are multiple people carrying out multiple handoffs. An alternative approach is to use the space above the table for object handoff; this provides more room to move, but requires above-surface tracking. I developed two above-the-surface handoff techniques that use simple and inexpensive tracking: a force-field technique that uses a depth camera to determine hand proximity, and an electromagnetic-field technique called ElectroTouch that provides positive indication when people touch hands over the table. These new techniques were compared to three kinds of existing surface-only handoff (sliding, flicking, and surface-only Force-Fields). The study showed that the above-surface techniques significantly improved both speed and accuracy, and that ElectroTouch was the best technique overall. Also, as object interactions are moved above-the-surface of the table the representation of off-table objects becomes crucial. To address the issue of off-table digital object representation several object designs were created an evaluated. The result of the present research provides designers with practical new techniques for substantially increasing performance and interaction richness on digital tables

Waves: A Collaborative Navigation Technique for Large Interactive Surfaces

Digital tables offer the possibility of performing collaborative tasks where two or more people can share artifacts in the same virtual space. However, most interactive methods of navigating through virtual space most commonly have the effect of changing the entire digital display simultaneously. In this thesis, I performed an exploratory study providing evidence for differences between two popular collaborative navigation techniques used in video games, split screens and single shared screen, in situational awareness, interference between collaborators, and difficulties with automatic view adjustment. Drawing inspiration from guidelines formulated from the results of the exploratory study, as well as previous work in interactive tabletops, collaboration, and navigation in information visualization, I designed and implemented Waves, a collaborative navigation technique for the tabletop. Waves simultaneously supports multiple personal workspaces, provides group workspace awareness, and mediates interference between workspaces