Increasing Passersby Engagement with Public Large Interactive Displays: A Study of Proxemics and Conation

This is the author’s version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in the Proceedings of the 2018 International Conference on Human Factors on Computing Systems on the ACM Digital Library at https://dx.doi.org/10.1145/3279778.3279789Prior research has shown that large interactive displays de- ployed in public spaces are often underutilized, or even un- noticed, phenomena connected to ‘interaction’ and ‘display blindness’, respectively. To better understand how designers can mitigate these issues, we conducted a field experiment that compared how different visual cues impacted engagement with a public display. The deployed interfaces were designed to progressively reveal more information about the display and entice interaction through the use of visual content designed to evoke direct or indirect conation (the mental faculty related to purpose or will to perform an action), and different ani- mation triggers (random or proxemic). Our results show that random triggers were more effective than proxemic triggers at overcoming display and interaction blindness. Our study of conation – the first we are aware of – found that “conceptual” visuals designed to evoke indirect conation were also useful in attracting people’s attention.Natural Sciences and Engineering Research Council of Canad

Investigating the Impact of Proximity and Visual Conation Modes on Enhancing Engagement with Public Large Interactive Displays

Deployment of large interactive displays (LIDs) to public spaces has provided new ways for passersby to gain information. This medium plays the role of transmitter for information visualizations designed to communicate certain messages or provide specific digital experiences. However, prior research has shown that these forms of interactive surfaces are often highly underutilized, even unnoticed, when installed in public spaces. When LIDs are unnoticed, or fail to sufficiently engage passersby, the intended message(s) cannot be transmitted or perceived successfully. To mitigate this problem, this research leverages empirical and theoretical frameworks from the field of Communication Studies, and from the subfield of Symbol Interaction as well as various message functions. Accordingly, we generated several animated visual cues to examine the impact of proximity and conation (persuasion) modes. We also ran a field study to evaluate the interface design. Through implementing of the data analysis, we learned that animation effects are useful assets in order to obviate the conative function of communication (persuade passersby to become engaged with the LID). Our findings emphasize that self-revealing systems design may encourage the user to become engaged with the LID. It was also revealed that randomized animated visual effects had more impact on the passersby touch behaviour

Investigating menu discoverability on a digital tabletop in a public setting

A common challenge to the design of digital tabletops for public settings is how to effectively invite and guide passersby--who often have no prior experience with such technology--to interact using unfamiliar interaction methods and interfaces. We characterize such enticement from the system interface as the system's discoverability. A particular challenge to modern surface interfaces is the discoverability of system functionality: does the system require gestures? are there system menus? if so, how are they invoked? This research focuses on the discoverability of system menus on digital tabletops designed for public settings. An observational study of menu invocation methods in a museum setting is reported. Study findings suggest that discernible and recognizable interface elements, such as buttons, supported by the use of animation, can effectively attract and guide the discovery of menus. Design recommendations for improving menu discoverability are also presented.Ye

Enhanced Multi-Touch Gestures for Complex Tasks

Recent technological advances have resulted in a major shift, from high-performance notebook and desktop computers -- devices that rely on keyboard and mouse for input -- towards smaller, personal devices like smartphones, tablets and smartwatches which rely primarily on touch input. Users of these devices typically have a relatively high level of skill in using multi-touch gestures to interact with them, but the multi-touch gesture sets that are supported are often restricted to a small subset of one and two-finger gestures, such as tap, double tap, drag, flick, pinch and spread. This is not due to technical limitations, since modern multi-touch smartphones and tablets are capable of accepting at least ten simultaneous points of contact. Likewise, human movement models suggest that humans are capable of richer and more expressive forms of interaction that utilize multiple fingers. This suggests a gap between the technical capabilities of multi-touch devices, the physical capabilities of end-users, and the gesture sets that have been implemented for these devices. Our work explores ways in which we can enrich multi-touch interaction on these devices by expanding these common gesture sets. Simple gestures are fine for simple use cases, but if we want to support a wide range of sophisticated behaviours -- the types of interactions required by expert users -- we need equally sophisticated capabilities from our devices. In this thesis, we refer to these more sophisticated, complex interactions as `enhanced gestures' to distinguish them from common but simple gestures, and to suggest the types of expert scenarios that we are targeting in their design. We do not need to necessarily replace current, familiar gestures, but it makes sense to consider augmenting them as multi-touch becomes more prevalent, and is applied to more sophisticated problems. This research explores issues of approachability and user acceptance around gesture sets. Using pinch-to-zoom as an example, we establish design guidelines for enhanced gestures, and systematically design, implement and evaluate two different types of expert gestures, illustrative of the type of functionality that we might build into future systems

Increasing Passersby Engagement with Public Large Interactive Surfaces

Despite the proliferation of Public Large Interactive Surfaces (PLISs), and their potential to provide a more engaging and interactive user experience, these surfaces often go unnoticed by passersby, or not immediately comprehensible in terms of usage. Current research in addressing this problem involves modeling the user-surface interaction through observational studies, and deriving recommendations for interface design to facilitate the interaction. This approach is often context-specific, requires elaborate setup, and lacks experimental control. To mitigate this problem, an interaction model, named DISCOVER, was developed by drawing ideas from classic usability research and focusing on the discoverability aspect of the interaction. This approach allows the model to serve as a lens for understanding and synthesizing existing work on PLISs, and to be used as an evaluation framework to assess effectiveness of potential designs. To accompany this evaluation capability, a laboratory-based evaluation methodology was developed to allow researchers to quickly implement and evaluate potential designs, particularly for the early stages of interaction that precede the more commonly studied explicit and direct interaction (e.g., touches, mid-air gestures). Using the model and the evaluation methodology, a proximity-based interaction mechanism using animated content and shadow visualizations was designed and evaluated as an effective technique in drawing attention from unknowing study participants. A follow-up, more conventional in-the-wild study also verified this finding, and further demonstrated the usefulness of shadow visualizations in drawing attention from passersby, retaining them, and enticing playful interaction. The goal of this thesis is to better equip researchers and practitioners of PLISs with tools that allow them to evaluate and improve existing interfaces, and to provide them with insights into designing future ones employing better and more engaging technologies

Supporting Situation Awareness and Workspace Awareness in Co-located Collaborative Systems Involving Dynamic Data

Co-located technologies can provide digital functionality to support collaborative work for multiple users in the same physical space. For example, digital tabletop computers — large interactive tables that allow users to directly interact with the content — can provide the most up-to-date map information while users can work together face-to-face. Combinations of interactive devices, large and small, can also be used together in a multi-device environment to support collaborative work of large groups. This environment allows individuals to utilize different networked devices. In some co-located group work, integrating automation into the available technologies can provide benefits such as automatically switching between different data views or updating map information based on underlying changes in deployed field agents’ locations. However, dynamic changes in the system state can create confusion for users and lead to low situation awareness. Furthermore, with the large size of a tabletop system or with multiple devices being used in the workspace, users may not be able to observe collaborators’ actions due to physical separations between users. Consequently, workspace awareness — knowledge of collaborators’ up-to-the-moment actions — can be difficult to maintain. As a result, users may be frustrated, and the collaboration may become inefficient or ineffective. The current tabletop applications involving dynamic data focus on interaction and information sharing techniques for collaboration rather than providing situation awareness support. Moreover, the situation awareness literature focuses primarily on single-user applications, whereas, the literature in workspace awareness primarily focuses on remote collaborative work. The aim of this dissertation was in supporting situation awareness of system-automated dynamic changes and workspace awareness of collaborators’ actions. The first study (Timeline Study) presented in this dissertation used tabletop systems to investigate supporting situation awareness of automated changes and workspace awareness, and the second study (Callout Bubble Study) followed up to further investigate workspace awareness support in the context of multi-device classrooms. Digital tabletop computers are increasingly being used for complex domains involving dynamic data, such as coastal surveillance and emergency response. Maintaining situation awareness of these changes driven by the system is crucial for quick and appropriate response when problems arise. However, distractors in the environment can make users miss the changes and negatively impact their situation awareness, e.g., the large size of the table and conversations with team members. As interactive event timelines have been shown to improve response time and decision accuracy after interruptions, in this dissertation they were adapted to the context of collaborative tabletop applications to address the lack of situation awareness due to dynamic changes. A user study was conducted to understand design factors related to the adaption and their impacts on situation awareness and workspace awareness. The Callout Bubble Study investigated workspace awareness support for multi-device classrooms, where students were co-located with their personal devices and were connected through a large shared virtual canvas. This context was chosen due to the environment’s ability to support work in large groups and the increasing prevalence of individual devices in co-located collaborative workspaces. By studying another co-located context, this research also sought to combine the lessons learned and provide a set of more generalized design recommendations for co-located technologies. Existing work on workspace awareness focuses on remote collaboration; however, the co-located users may not need all the information beneficial for remote work. This study aimed to balance awareness and distraction to improve students’ workspace awareness maintenance while minimizing distraction to their learning. A Callout Bubble was designed to augment students’ interactions in the shared online workspace, and a field study was conducted to understand how it impacted the students’ collaboration behaviour. Overall, the research presented in this dissertation aimed to investigate information visualizations for supporting situation awareness and workspace awareness in co-located collaborative environments. The contributions included the design of an interactive event timeline and an investigation of how the control placement (how many timelines and where they should be located) and feedback location (whether to display feedback to the group or to individuals when users interact with timelines) factors affected situation awareness. The empirical results revealed that individual timelines were more effective in facilitating situation awareness maintenance and the timelines were used mainly for perceiving new changes. Furthermore, this dissertation contributed in the design of a workspace awareness cue, Callout Bubble. The field study revealed that Callout Bubbles were effective in improving students’ coordination and self-monitoring behaviours, which in turn reduced teachers’ workloads. The dissertation provided overall design lessons learned for supporting awareness in co-located collaborative environments

Perceptible affordances and feedforward for gestural interfaces: Assessing effectiveness of gesture acquisition with unfamiliar interactions

The move towards touch-based interfaces disrupts the established ways in which users manipulate and control graphical user interfaces. The predominant mode of interaction established by the desktop interface is to ‘double-click’ an icon in order to open an application, file or folder. Icons show users where to click and their shape, colour and graphic style suggests how they respond to user action. In sharp contrast, in a touch-based interface, an action may require a user to form a gesture with a certain number of fingers, a particular movement, and in a specific place. Often, none of this is suggested in the interface. This thesis adopts the approach of research through design to address the problem of how to inform the user about which gestures are available in a given touch-based interface, how to perform each gesture, and, finally, the effect of each gesture on the underlying system. Its hypothesis is that presenting automatic and animated visual prompts that depict touch and preview gesture execution will mitigate the problems users encounter when they execute commands within unfamiliar gestural interfaces. Moreover, the thesis claims the need for a new framework to assess the efficiency of gestural UI designs. A significant aspect of this new framework is a rating system that was used to assess distinct phases within the users’ evaluation and execution of a gesture. In order to support the thesis hypothesis, two empirical studies were conducted. The first introduces the visual prompts in support of training participants in unfamiliar gestures and gauges participants’ interpretation of their meaning. The second study consolidates the design features that yielded fewer error rates in the first study and assesses different interaction techniques, such as the moment to display the visual prompt. Both studies demonstrate the benefits in providing visual prompts to improve user awareness of available gestures. In addition, both studies confirm the efficiency of the rating system in identifying the most common problems users have with gestures and identifying possible design features to mitigate such problems. The thesis contributes: 1) a gesture-and-effect model and a corresponding rating system that can be used to assess gestural user interfaces, 2) the identification of common problems users have with unfamiliar gestural interfaces and design recommendations to mitigate these problems, and 3) a novel design technique that will improve user awareness of unfamiliar gestures within novel gestural interfaces