The Lifecycle of a Whiteboard Photo: Post-meeting Usage of Whiteboard Content Captured with Mobile Devices

The traditional dry-erase whiteboard is a ubiquitous tool in the workplace, particularly in meeting spaces where they are a key collaboration tool. Although research on whiteboard use and the development of whiteboard systems have been a staple in the Human-Computer Interaction (HCI) literature, there is a lack of how whiteboard content is used to direct actions outside the meeting. In today’s mobile-centric world, knowledge workers capture whiteboard content after a meeting by taking photos of them with mobile devices such as cellphones. This thesis empirically investigated post-meeting practices with whiteboard photos to explore how these practices might be better supported by technology. In particular, this thesis investigated the main post-meeting activities that whiteboard photos support, how people value whiteboard photos, and how they manage them. Nineteen knowledge workers from a variety of professions, companies, and industries across North America who were regular users of whiteboards were interviewed using a semi-structured protocol. A Thematic Analysis of the data revealed that whiteboard photos were primarily used to create superseding documents and were sometimes used as evidence of agreement. Whiteboard photos were used mostly in the short-term, and their value was transient. Also, an analysis of the minute detailed actions that interviewees reported taking with one or two recent whiteboard photos revealed that the typical lifecycle of those whiteboard photos involved a seven-stage lifecycle. Moreover, a memory recall task with six of the interviewees about previous meetings where a whiteboard was used revealed that general characteristics of those meetings were well remembered while characteristics about the whiteboard content were not well remembered. The findings suggested a set of unmet design needs for the development of improved mobile-centric whiteboard capture systems. The suggested design implications include the need for a mobile application that supports quick capture and the effortless transfer of whiteboard photos to productivity-oriented devices, and the need for a desktop application that supports the extraction of whiteboard content to aid users in creating superseding documents

PAPIERCRAFT: A PAPER-BASED INTERFACE TO SUPPORT INTERACTION WITH DIGITAL DOCUMENTS

Many researchers extensively interact with documents using both computers and paper printouts, which provide an opposite set of supports. Paper is comfortable to read from and write on, and it is flexible to be arranged in space; computers provide an efficient way to archive, transfer, search, and edit information. However, due to the gap between the two media, it is difficult to seamlessly integrate them together to optimize the user's experience of document interaction. Existing solutions either sacrifice inherent paper flexibility or support very limited digital functionality on paper. In response, we have proposed PapierCraft, a novel paper-based interface that supports rich digital facilities on paper without sacrificing paper's flexibility. By employing the emerging digital pen technique and multimodal pen-top feedback, PapierCraft allows people to use a digital pen to draw gesture marks on a printout, which are captured, interpreted, and applied to the corresponding digital copy. Conceptually, the pen and the paper form a paper-based computer, able to interact with other paper sheets and computing devices for operations like copy/paste, hyperlinking, and web searches. Furthermore, it retains the full range of paper advantages through the light-weighted, pen-paper-only interface. By combining the advantages of paper and digital media and by supporting the smooth transition between them, PapierCraft bridges the paper-computer gap. The contributions of this dissertation focus on four respects. First, to accommodate the static nature of paper, we proposed a pen-gesture command system that does not rely on screen-rendered feedback, but rather on the self-explanatory pen ink left on the paper. Second, for more interactive tasks, such as searching for keywords on paper, we explored pen-top multimodal (e.g. auditory, visual, and tactile) feedback that enhances the command system without sacrificing the inherent paper flexibility. Third, we designed and implemented a multi-tier distributed infrastructure to map pen-paper interactions to digital operations and to unify document interaction on paper and on computers. Finally, we systematically evaluated PapierCraft through three lab experiments and two application deployments in the areas of field biology and e-learning. Our research has demonstrated the feasibility, usability, and potential applications of the paper-based interface, shedding light on the design of the future interface for digital document interaction. More generally, our research also contributes to ubiquitous computing, mobile interfaces, and pen-computing

Supporting Reflection and Classroom Orchestration with Tangible Tabletops

Tangible tabletop systems have been extensively proven to be able to enhance participation and engagement as well as enable many exciting activities, particularly in the education domain. However, it remains unclear as to whether students really benefit from using them for tasks that require a high level of reflection. Moreover, most existing tangible tabletops are designed as stand-alone systems or devices. Increasingly, this design assumption is no longer sufficient, especially in realistic learning settings. Due to the technological evolution in schools, multiple activities, resources, and constraints in the classroom ecosystem are now involved in the learning process. The way teachers manage technology-enhanced classrooms and the involved activities and constraints in real-time, also known as classroom orchestration, is a crucial aspect for the materialization of reflection and learning. This thesis aims to explore how educational tangible tabletop systems affect reflection, how reflection and orchestration are related, and how we can support reflection and orchestration to improve learning. It presents the design, implementation, and evaluations of three tangible tabletop systems – the DockLamp, the TinkerLamp, and the TinkerLamp 2.0 – in different learning contexts. Our experience with these systems, both inside and outside of the laboratory, results in an insightful understanding of the impacts of tangible tabletops on learning and the conditions for their effective use as well as deployment. These findings can be beneficial to the researchers and designers of learning environments using tangible tabletop and similar interfaces

Mobile Pen and Paper Interaction

Although smartphones, tablets and other mobile devices become increasingly popular, pen and paper continue to play an important role in mobile settings, such as note taking or creative discussions. However, information on paper documents remains static and usage practices involving sharing, researching, linking or in any other way digitally processing information on paper are hindered by the gap between the digital and physical worlds. A considerable body of research has leveraged digital pen technology in order to overcome this problem with respect to static settings, however, systematically neglecting the mobile domain. Only recently, several approaches began exploring the mobile domain and developing initial insights into mobile pen-and-paper interaction (mPPI), e.g., to publish digital sketches, [Cowan et al., 2011], link paper and digital artifacts, [Pietrzak et al., 2012] or compose music, [Tsandilas, 2012]. However, applications designed to integrate the most common mobile tools pen, paper and mobile devices, thereby combining the benefits of both worlds in a hybrid mPPI ensemble, are hindered by the lack of supporting infrastructures and limited theoretical understanding of interaction design in the domain. This thesis advances the field by contributing a novel infrastructural approach toward supporting mPPI. It allows applications employing digital pen technology in controlling interactive functionality while preserving mobile characteristics of pen and paper. In addition, it contributes a conceptual framework of user interaction in the domain suiting to serve as basis for novel mPPI toolkits. Such toolkits ease development of mPPI solutions by focusing on expressing interaction rather than designing user interfaces by means of rigid widget sets. As such, they provide the link between infrastructure and interaction in the domain. Lastly, this thesis presents a novel, empirically substantiated theory of interaction in hybrid mPPI ensembles. This theory informs interaction design of mPPI, ultimately allowing to develop compelling and engaging interactive systems employing this modality