Designing for Cross-Device Interactions

Driven by technological advancements, we now own and operate an ever-growing number of digital devices, leading to an increased amount of digital data we produce, use, and maintain. However, while there is a substantial increase in computing power and availability of devices and data, many tasks we conduct with our devices are not well connected across multiple devices. We conduct our tasks sequentially instead of in parallel, while collaborative work across multiple devices is cumbersome to set up or simply not possible. To address these limitations, this thesis is concerned with cross-device computing. In particular it aims to conceptualise, prototype, and study interactions in cross-device computing. This thesis contributes to the field of Human-Computer Interaction (HCI)—and more specifically to the area of cross-device computing—in three ways: first, this work conceptualises previous work through a taxonomy of cross-device computing resulting in an in-depth understanding of the field, that identifies underexplored research areas, enabling the transfer of key insights into the design of interaction techniques. Second, three case studies were conducted that show how cross-device interactions can support curation work as well as augment users’ existing devices for individual and collaborative work. These case studies incorporate novel interaction techniques for supporting cross-device work. Third, through studying cross-device interactions and group collaboration, this thesis provides insights into how researchers can understand and evaluate multi- and cross-device interactions for individual and collaborative work. We provide a visualization and querying tool that facilitates interaction analysis of spatial measures and video recordings to facilitate such evaluations of cross-device work. Overall, the work in this thesis advances the field of cross-device computing with its taxonomy guiding research directions, novel interaction techniques and case studies demonstrating cross-device interactions for curation, and insights into and tools for effective evaluation of cross-device systems

WRISTBAND.IO:expanding input and output spaces of a Smartwatch

Smartwatches are characterized by their small size designed for wearability, discretion, and mobile interactions. Most of the interactivity, however, is limited to the size of the display, introducing issues such as screen occlusion and limited information density. We introduce Wristband.io, a smartwatch with an extended interaction space along the wristband, enabling (i) back-of-band interaction using a touchpad, (ii) a low resolution ambient watchband display for offscreen notification, and (iii) tangible buttons for quick, eyes-free input. Insights gained through a study show that back-of-band input increases accuracy and task completion rates for smaller on-screen targets. We probe the design space of Wristband.io with three applications

WRISTBAND.IO:expanding input and output spaces of a Smartwatch

Smartwatches are characterized by their small size designed for wearability, discretion, and mobile interactions. Most of the interactivity, however, is limited to the size of the display, introducing issues such as screen occlusion and limited information density. We introduce Wristband.io, a smartwatch with an extended interaction space along the wristband, enabling (i) back-of-band interaction using a touchpad, (ii) a low resolution ambient watchband display for offscreen notification, and (iii) tangible buttons for quick, eyes-free input. Insights gained through a study show that back-of-band input increases accuracy and task completion rates for smaller on-screen targets. We probe the design space of Wristband.io with three applications

CurationSpace:Cross-Device Content Curation Using Instrumental Interaction

For digital content curation of historical artefacts, curators collaboratively collect, analyze and edit documents, images, and other digital resources in order to display and share new representations of that information to an audience. Despite their increasing reliance on digital documents and tools, current technologies provide little support for these specific collaborative content curation activities. We introduce CurationSpace - a novel cross-device system - to provide more expressive tools for curating and composing digital historical artefacts. Based on the concept of Instrumental Interaction, CurationSpace allows users to interact with digital curation artefacts on shared interactive surfaces using personal smartwatches as selectors for instruments or modifiers (applied to either the whole curation space, individual documents, or fragments). We introduce a range of novel interaction techniques that allow individuals or groups of curators to more easily create, navigate and share resources during content curation. We report insights from our user study about people's use of instruments and modifiers for curation activities

EvalMe:Exploring the value of new technologies for in situ evaluation of learning experiences

Tangible interfaces have much potential for engendering shared interaction and reflection, as well as for promoting playful experiences. How can their properties be capitalised on to enable students to reflect on their learning, both individually and together, throughout learning sessions? This Research through Design paper describes our development of EvalMe, a flexible, tangible tool aimed at being playful, enjoyable to use and enabling children to reflect on their learning, both in the moment and after a learning session has ended. We discuss the insights gained through the process of designing EvalMe, co-defining its functionality with two groups of collaborators and deploying it in two workshop settings. Through this process, we map key contextual considerations for the design of technologies for in situ evaluation of learning experiences. Finally, we discuss how tangible evaluation technologies deployed throughout a learning session, can positively contribute to students’ reflection about their learning

Is Anyone Looking? Mitigating Shoulder Surfing on Public Displays through Awareness and Protection

Displays are growing in size, and are increasingly deployed in semi-public and public areas. When people use these public displays to pursue personal work, they expose their activities and sensitive data to passers-by. In most cases, such shoulder-surfing by others is likely voyeuristic vs. a deliberate attempt to steal information. Even so, safeguards are needed. Our goal is to mitigate shoulder-surfing problems in such settings. Our method leverages notions of territoriality and proxemics, where we sense and take action based on the spatial relationships between the passerby, the user of the display, and the display itself. First, we provide participants with awareness of shoulder-surfing moments, which in turn helps both parties regulate their behaviours and mediate further social interactions. Second, we provide methods that protect information when shoulder-surfing is detected. Here, users can move or hide information through easy to perform explicit actions. Alternately, the system itself can mask information from the passerby’s view when it detects shoulder-surfing moments

Investigating the Role of an Overview Device in Multi-Device Collaboration

The availability of mobile device ecologies enables new types of ad-hoc co-located decision-making and sensemaking practices in which people find, collect, discuss, and share information. However, little is known about what kind of device configurations are suitable for these types of tasks. This paper contributes new insights into how people use configurations of devices for one representative example task: collaborative co-located trip-planning. We present an empirical study that explores and compares three strategies to use multiple devices: no-overview, overview on own device, and a separate overview device. The results show that the overview facilitated decision- and sensemaking during a collaborative trip-planning task by aiding groups to iterate their itinerary, organize locations and timings efficiently, and discover new insights. Groups shared and discussed more opinions, resulting in more democratic decision-making. Groups provided with a separate overview device engaged more frequently and spent more time in closely-coupled collaboration

EagleView:A Video Analysis Tool for Visualising and Querying Spatial Interactions of People and Devices

To study and understand group collaborations involving multiple handheld devices and large interactive displays, researchers frequently analyse video recordings of interaction studies to interpret people's interactions with each other and/or devices. Advances in ubicomp technologies allow researchers to record spatial information through sensors in addition to video material. However, the volume of video data and high number of coding parameters involved in such an interaction analysis makes this a time-consuming and labour-intensive process. We designed EagleView, which provides analysts with real-time visualisations during playback of videos and an accompanying data-stream of tracked interactions. Real-time visualisations take into account key proxemic dimensions, such as distance and orientation. Overview visualisations show people's position and movement over longer periods of time. EagleView also allows the user to query people's interactions with an easy-to-use visual interface. Results are highlighted on the video player's timeline, enabling quick review of relevant instances. Our evaluation with expert users showed that EagleView is easy to learn and use, and the visualisations allow analysts to gain insights into collaborative activities

The Making of Meaning through Dyadic Haptic Affective Touch

Despite the importance of touch in human-human relations, research in affective tactile practices is in its infancy, lacking in-depth understanding needed to inform the design of remote digital touch communication. This paper reports two qualitative studies that explore tactile affective communication in specific social contexts, and the bi-directional creation, sending and interpretation of digital touch messages using a purpose-built research tool, the Tactile Emoticon. The system comprises a pair of remotely connected mitts which enable users in different locations to communicate through tactile messages, by orchestrating duration and level of three haptic sensations: vibration, pressure and temperature. Qualitative analysis shows the nuanced ways in which 68 participants configured these elements to make meaning from touch messages they sent and received. It points to the affect and emotion of touch, its sensoriality and ambiguity, the significance of context, social norms and expectations of touch participants. Findings suggest key design considerations for digital touch communication, where the emphasis shifts from generating ‘recognizable touches’ to tools that allow people to shape their touches and establish common understanding about their meaning