Digital tabletops and collaborative learning

People collaborate around tables at home, school and work. Digital tabletop technology presents an opportunity to bring computer support to these traditional face-to-face collaborative settings. This thesis principally addresses the challenge of designing digital tabletop applications for small group learning in the classroom and makes contributions in two distinct, but closely related areas: (i) interaction techniques for digital tabletops; and (ii) the design and evaluation of a digital tabletop-based system for supporting collaborative learning. A review of previous literature combined with a preliminary observational study on collaboration around traditional tables indentifies a number of requirements for tabletop interaction. These include the need for fluid interaction techniques that allow control of interface object attributes when these objects are moved between tabletop territories. Attribute gates are proposed as a solution to this problem through utilizing a novel, crossing-based, interaction technique. A recognition of the territorial focus in existing interaction techniques, and their limiting assumption that users work at relatively fixed locations around the table, led to the identification of another challenge, supporting the mobility of users around the shared workspace of the table. TANGISOFT is presented as a hybrid tangible-soft keyboard designed specifically for applications that require mobile users with moderate text entry requirements. The investigation of the potential of tabletop technology to support collaborative learning was carried out through the design, development, and evaluation of Digital Mysteries. From an interaction design perspective, the design aimed to utilize the unique affordances of tabletops in terms of combining the benefits of traditional tables and digital technology. From a learning perspective, the design aimed to support higher-level thinking skills, feedback, reflection, and metacognition by focusing on activities that promote these skills and supporting effective collaboration. The evaluation of Digital Mysteries demonstrated that the design was successful in encouraging the targeted learning activities. The design process and validation of Digital Mysteries embody a significant contribution to the development of our understanding of digital tabletop technology at the application level, and collaborative learning applications in particular. This understanding is summarized in the form of general guidelines for designing collaborative learning applications for digital tabletop technology.EThOS - Electronic Theses Online ServiceDiwan Software LtdGBUnited Kingdo

Territoriality and Behaviour On and Around Large Vertical Publicly-Shared Displays

Large displays and information kiosks are becoming increasingly common installations in public venues to provide an efficient self-serve means for patrons to access information and/or services. They have evolved over a relatively short period of time from non-digital, non-interactive static displays to more elaborate media-rich digital interactive systems. While the content and purposes of kiosks have changed, they are still largely based on the traditional single-user-driven design paradigm despite the fact that people often venture to these venues in small social groups, i.e., with family and/or friends. This often limits how groups collaborate and forces transactions to be serialized. This thesis explores design constraints for interaction by multiple social groups in parallel on shared large vertical displays. To better understand design requirements for these systems, this research is separated into two parts: a preliminary observational field study and a follow-up controlled study. Using an observational field study, fundamental patterns of how people use existing public displays are studied: their orientation, positioning, group identification, and behaviour within and between social groups just-before, during, and just-after usage. These results are then used to motivate a controlled experiment where two individuals or two pairs of individuals complete tasks concurrently on a low-fidelity large vertical display. Results from the studies demonstrate that vertical surface territories are similar to those found in horizontal tabletops in function, but their definitions and social conventions are different. In addition, the nature of use-while-standing systems results in more complex and dynamic physical territories around the display. We show that the anthropological notion of personal space must be slightly refined for application to vertical displays

An Activity-Centric Approach to Configuration Work in Distributed Interaction

The widespread introduction of new types of computing devices, such as smartphones, tablet computers, large interactive displays or even wearable devices, has led to setups in which users are interacting with a rich ecology of devices. These new device ecologies have the potential to introduce a whole new set of cross-device and cross-user interactions as well as to support seamless distributed workspaces that facilitate coordination and communication with other users. Because of the distributed nature of this paradigm, there is an intrinsic difficulty and overhead in managing and using these kind of complex device ecologies, which I refer to as configuration work. It is the effort required to set up, manage, communicate, understand and use information, applications and services that are distributed over all devices in use and people involved. Because current devices and their containing software are still document- and application-centric, they fail to capture and support the rich activities and context in which they are being used. This leaves users without a stable concept for cross-device information management, forcing them to perform a large amount of manual configuration work. In this dissertation, I explore an activity-centric approach to configuration work in distributed interaction. The central goal of this dissertation is to develop and apply concepts and ideas from Activity-Centric Computing to distributed interaction. Using the triangulation approach, I explore these concepts on a conceptual, empirical and technological level and present a framework and use cases for designing activitycentric configurations in multi-device information systems. The dissertation presents two major contributions: First, I introduce the term configuration work as an abstract analytical unit that describes and captures the problems and challenges of distributed interaction. Using both empirical data and related work, I argue that configuration work is composed of: curation work, task resumption lag, mobility work, physical handling and articulation work. Using configuration work as a problem description, I operationalize Activity Theory and Activity-Centric Computing to mitigate and reduce configuration work in distributed interaction. By allowing users to interact with computational representations of their real-world activities, creating complex multi-user device ecologies and switching between cross-device information configurations will be more efficient, more effective and provide better support for users’ mental model about a multi-user and multi-device environment. Using activity configuration as a central concept, I introduce a framework that describes how digital representations of human activity can be distributed, fragmented and used across multiple devices and users. Second, I present a technical infrastructure and four applications that apply the concepts of activity configuration. The infrastructure is a general purpose platform for the design, development and deployment of distributed activitycentric systems. The infrastructure simplifies the development of activity-centric systems as it presents complex distributed computing processes and services into high level activity system abstractions. Using this infrastructure and conceptual framework, I describe four fully working applications that explore multi-device interactions in two specific domains: office work and hospital work. The systems are evaluated and tested with end-users in a number of lab and field studies

Interaction Design for Mixed-Focus Collaboration in Cross-Device Environments

The proliferation of interactive technologies has resulted in a multitude of form factors for computer devices, such as tablets and phones, and large tabletop and wall displays. Investigating how these devices may be used together as Cross-Device Environments (XDEs) to facilitate collaboration is an active area of research in Human Computer Interaction (HCI) and Computer-Supported Cooperative Work (CSCW). The research community has explored the role of personal and shared devices in supporting group work and has introduced a number of cross-device interaction techniques to enable interaction among devices in an XDE. However, there is little understanding of how the interface design of those techniques may change the way people conduct collaboration, which, in turn, could influence the outcome of the activity. This thesis studies the impact of cross-device interaction techniques on collaborative processes. In particular, I investigated how interface design of cross-device interaction techniques may impact communication and coordination during group work. First, I studied the impact of two specific cross-device interaction techniques on collaboration in an XDE comprised of tablets and a tabletop. The findings confirmed that the choice of interaction techniques mattered when it came to facilitating both independent and joint work periods during group work. The study contributes knowledge towards problematizing the impact of cross-device interaction techniques on collaboration in HCI research. This early work gave rise to deeper questions regarding coordination in cross-device transfer and leveraging that to support the flexibility of work periods in collaborative activities. Consequently, I explored a range of interface design choices that varied the degree of synchronicity in coordinating data transfer across two devices. Additionally, I studied the impact of those interface designs on collaborative processes. My findings resulted in design considerations as well as adapting a synchronicity framework to articulate the impact of cross-device transfer techniques on collaboration. While performing the two research projects, I identified a need for a tool to articulate the impact of specific user interface elements on collaboration. Through a series of case studies, I developed a visual framework that researchers can use as a formative and summative method to understand if a given interaction technique hinders or supports collaboration in the specific task context. I discuss the contributions of my work to the field of HCI, design implications beyond the environments studied, and future research directions to build on and extend my findings

Designing Hybrid Interactions through an Understanding of the Affordances of Physical and Digital Technologies

Two recent technological advances have extended the diversity of domains and social contexts of Human-Computer Interaction: the embedding of computing capabilities into physical hand-held objects, and the emergence of large interactive surfaces, such as tabletops and wall boards. Both interactive surfaces and small computational devices usually allow for direct and space-multiplex input, i.e., for the spatial coincidence of physical action and digital output, in multiple points simultaneously. Such a powerful combination opens novel opportunities for the design of what are considered as hybrid interactions in this work. This thesis explores the affordances of physical interaction as resources for interface design of such hybrid interactions. The hybrid systems that are elaborated in this work are envisioned to support specific social and physical contexts, such as collaborative cooking in a domestic kitchen, or collaborative creativity in a design process. In particular, different aspects of physicality characteristic of those specific domains are explored, with the aim of promoting skill transfer across domains. irst, different approaches to the design of space-multiplex, function-specific interfaces are considered and investigated. Such design approaches build on related work on Graspable User Interfaces and extend the design space to direct touch interfaces such as touch-sensitive surfaces, in different sizes and orientations (i.e., tablets, interactive tabletops, and walls). These approaches are instantiated in the design of several experience prototypes: These are evaluated in different settings to assess the contextual implications of integrating aspects of physicality in the design of the interface. Such implications are observed both at the pragmatic level of interaction (i.e., patterns of users' behaviors on first contact with the interface), as well as on user' subjective response. The results indicate that the context of interaction affects the perception of the affordances of the system, and that some qualities of physicality such as the 3D space of manipulation and relative haptic feedback can affect the feeling of engagement and control. Building on these findings, two controlled studies are conducted to observe more systematically the implications of integrating some of the qualities of physical interaction into the design of hybrid ones. The results indicate that, despite the fact that several aspects of physical interaction are mimicked in the interface, the interaction with digital media is quite different and seems to reveal existing mental models and expectations resulting from previous experience with the WIMP paradigm on the desktop PC

The cockpit for the 21st century

Interactive surfaces are a growing trend in many domains. As one possible manifestation of Mark Weiser’s vision of ubiquitous and disappearing computers in everywhere objects, we see touchsensitive screens in many kinds of devices, such as smartphones, tablet computers and interactive tabletops. More advanced concepts of these have been an active research topic for many years. This has also influenced automotive cockpit development: concept cars and recent market releases show integrated touchscreens, growing in size. To meet the increasing information and interaction needs, interactive surfaces offer context-dependent functionality in combination with a direct input paradigm. However, interfaces in the car need to be operable while driving. Distraction, especially visual distraction from the driving task, can lead to critical situations if the sum of attentional demand emerging from both primary and secondary task overextends the available resources. So far, a touchscreen requires a lot of visual attention since its flat surface does not provide any haptic feedback. There have been approaches to make direct touch interaction accessible while driving for simple tasks. Outside the automotive domain, for example in office environments, concepts for sophisticated handling of large displays have already been introduced. Moreover, technological advances lead to new characteristics for interactive surfaces by enabling arbitrary surface shapes. In cars, two main characteristics for upcoming interactive surfaces are largeness and shape. On the one hand, spatial extension is not only increasing through larger displays, but also by taking objects in the surrounding into account for interaction. On the other hand, the flatness inherent in current screens can be overcome by upcoming technologies, and interactive surfaces can therefore provide haptically distinguishable surfaces. This thesis describes the systematic exploration of large and shaped interactive surfaces and analyzes their potential for interaction while driving. Therefore, different prototypes for each characteristic have been developed and evaluated in test settings suitable for their maturity level. Those prototypes were used to obtain subjective user feedback and objective data, to investigate effects on driving and glance behavior as well as usability and user experience. As a contribution, this thesis provides an analysis of the development of interactive surfaces in the car. Two characteristics, largeness and shape, are identified that can improve the interaction compared to conventional touchscreens. The presented studies show that large interactive surfaces can provide new and improved ways of interaction both in driver-only and driver-passenger situations. Furthermore, studies indicate a positive effect on visual distraction when additional static haptic feedback is provided by shaped interactive surfaces. Overall, various, non-exclusively applicable, interaction concepts prove the potential of interactive surfaces for the use in automotive cockpits, which is expected to be beneficial also in further environments where visual attention needs to be focused on additional tasks.Der Einsatz von interaktiven Oberflächen weitet sich mehr und mehr auf die unterschiedlichsten Lebensbereiche aus. Damit sind sie eine mögliche Ausprägung von Mark Weisers Vision der allgegenwärtigen Computer, die aus unserer direkten Wahrnehmung verschwinden. Bei einer Vielzahl von technischen Geräten des täglichen Lebens, wie Smartphones, Tablets oder interaktiven Tischen, sind berührungsempfindliche Oberflächen bereits heute in Benutzung. Schon seit vielen Jahren arbeiten Forscher an einer Weiterentwicklung der Technik, um ihre Vorteile auch in anderen Bereichen, wie beispielsweise der Interaktion zwischen Mensch und Automobil, nutzbar zu machen. Und das mit Erfolg: Interaktive Benutzeroberflächen werden mittlerweile serienmäßig in vielen Fahrzeugen eingesetzt. Der Einbau von immer größeren, in das Cockpit integrierten Touchscreens in Konzeptfahrzeuge zeigt, dass sich diese Entwicklung weiter in vollem Gange befindet. Interaktive Oberflächen ermöglichen das flexible Anzeigen von kontextsensitiven Inhalten und machen eine direkte Interaktion mit den Bildschirminhalten möglich. Auf diese Weise erfüllen sie die sich wandelnden Informations- und Interaktionsbedürfnisse in besonderem Maße. Beim Einsatz von Bedienschnittstellen im Fahrzeug ist die gefahrlose Benutzbarkeit während der Fahrt von besonderer Bedeutung. Insbesondere visuelle Ablenkung von der Fahraufgabe kann zu kritischen Situationen führen, wenn Primär- und Sekundäraufgaben mehr als die insgesamt verfügbare Aufmerksamkeit des Fahrers beanspruchen. Herkömmliche Touchscreens stellen dem Fahrer bisher lediglich eine flache Oberfläche bereit, die keinerlei haptische Rückmeldung bietet, weshalb deren Bedienung besonders viel visuelle Aufmerksamkeit erfordert. Verschiedene Ansätze ermöglichen dem Fahrer, direkte Touchinteraktion für einfache Aufgaben während der Fahrt zu nutzen. Außerhalb der Automobilindustrie, zum Beispiel für Büroarbeitsplätze, wurden bereits verschiedene Konzepte für eine komplexere Bedienung großer Bildschirme vorgestellt. Darüber hinaus führt der technologische Fortschritt zu neuen möglichen Ausprägungen interaktiver Oberflächen und erlaubt, diese beliebig zu formen. Für die nächste Generation von interaktiven Oberflächen im Fahrzeug wird vor allem an der Modifikation der Kategorien Größe und Form gearbeitet. Die Bedienschnittstelle wird nicht nur durch größere Bildschirme erweitert, sondern auch dadurch, dass Objekte wie Dekorleisten in die Interaktion einbezogen werden können. Andererseits heben aktuelle Technologieentwicklungen die Restriktion auf flache Oberflächen auf, so dass Touchscreens künftig ertastbare Strukturen aufweisen können. Diese Dissertation beschreibt die systematische Untersuchung großer und nicht-flacher interaktiver Oberflächen und analysiert ihr Potential für die Interaktion während der Fahrt. Dazu wurden für jede Charakteristik verschiedene Prototypen entwickelt und in Testumgebungen entsprechend ihres Reifegrads evaluiert. Auf diese Weise konnten subjektives Nutzerfeedback und objektive Daten erhoben, und die Effekte auf Fahr- und Blickverhalten sowie Nutzbarkeit untersucht werden. Diese Dissertation leistet den Beitrag einer Analyse der Entwicklung von interaktiven Oberflächen im Automobilbereich. Weiterhin werden die Aspekte Größe und Form untersucht, um mit ihrer Hilfe die Interaktion im Vergleich zu herkömmlichen Touchscreens zu verbessern. Die durchgeführten Studien belegen, dass große Flächen neue und verbesserte Bedienmöglichkeiten bieten können. Außerdem zeigt sich ein positiver Effekt auf die visuelle Ablenkung, wenn zusätzliches statisches, haptisches Feedback durch nicht-flache Oberflächen bereitgestellt wird. Zusammenfassend zeigen verschiedene, untereinander kombinierbare Interaktionskonzepte das Potential interaktiver Oberflächen für den automotiven Einsatz. Zudem können die Ergebnisse auch in anderen Bereichen Anwendung finden, in denen visuelle Aufmerksamkeit für andere Aufgaben benötigt wird

Natural group binding and cross-display object movement methods for wearable devices

As wearable devices become more popular, situations where there are multiple persons present with such devices will become commonplace. In these situations, wearable devices could support collaborative tasks and experiences between co-located persons through multi-user applications. We present an elicitation study that gathers from end users interaction methods for wearable devices for two common tasks in co-located interaction: group binding and cross-display object movement. We report a total of 154 methods collected from 30 participants. We categorize the methods based on the metaphor and modality of interaction, and discuss the strengths and weaknesses of each category based on qualitative and quantitative feedback given by the participants.acceptedVersionPeer reviewe

Analysing, visualising and supporting collaborative learning using interactive tabletops

The key contribution of this thesis is a novel approach to design, implement and evaluate the conceptual and technological infrastructure that captures student’s activity at interactive tabletops and analyses these data through Interaction Data Analytics techniques to provide support to teachers by enhancing their awareness of student’s collaboration. To achieve the above, this thesis presents a series of carefully designed user studies to understand how to capture, analyse and distil indicators of collaborative learning. We perform this in three steps: the exploration of the feasibility of the approach, the construction of a novel solution and the execution of the conceptual proposal, both under controlled conditions and in the wild. A total of eight datasets were analysed for the studies that are described in this thesis. This work pioneered in a number of areas including the application of data mining techniques to study collaboration at the tabletop, a plug-in solution to add user-identification to a regular tabletop using a depth sensor and the first multi-tabletop classroom used to run authentic collaborative activities associated with the curricula. In summary, while the mechanisms, interfaces and studies presented in this thesis were mostly explored in the context of interactive tabletops, the findings are likely to be relevant to other forms of groupware and learning scenarios that can be implemented in real classrooms. Through the mechanisms, the studies conducted and our conceptual framework this thesis provides an important research foundation for the ways in which interactive tabletops, along with data mining and visualisation techniques, can be used to provide support to improve teacher’s understanding about student’s collaboration and learning in small groups