Mobile Pen and Paper Interaction

Although smartphones, tablets and other mobile devices become increasingly popular, pen and paper continue to play an important role in mobile settings, such as note taking or creative discussions. However, information on paper documents remains static and usage practices involving sharing, researching, linking or in any other way digitally processing information on paper are hindered by the gap between the digital and physical worlds. A considerable body of research has leveraged digital pen technology in order to overcome this problem with respect to static settings, however, systematically neglecting the mobile domain. Only recently, several approaches began exploring the mobile domain and developing initial insights into mobile pen-and-paper interaction (mPPI), e.g., to publish digital sketches, [Cowan et al., 2011], link paper and digital artifacts, [Pietrzak et al., 2012] or compose music, [Tsandilas, 2012]. However, applications designed to integrate the most common mobile tools pen, paper and mobile devices, thereby combining the benefits of both worlds in a hybrid mPPI ensemble, are hindered by the lack of supporting infrastructures and limited theoretical understanding of interaction design in the domain. This thesis advances the field by contributing a novel infrastructural approach toward supporting mPPI. It allows applications employing digital pen technology in controlling interactive functionality while preserving mobile characteristics of pen and paper. In addition, it contributes a conceptual framework of user interaction in the domain suiting to serve as basis for novel mPPI toolkits. Such toolkits ease development of mPPI solutions by focusing on expressing interaction rather than designing user interfaces by means of rigid widget sets. As such, they provide the link between infrastructure and interaction in the domain. Lastly, this thesis presents a novel, empirically substantiated theory of interaction in hybrid mPPI ensembles. This theory informs interaction design of mPPI, ultimately allowing to develop compelling and engaging interactive systems employing this modality

Paper-based mobile access to databases

Our demonstration is a paper-based interactive guide for visitors to the world's largest international arts festival that was developed as part of a project investigating new forms of context-aware information delivery and interaction in mobile environments. Information stored in a database is accessed from a set of interactive paper documents, including a printed festival brochure, a city map and a bookmark. Active areas are defined within the documents and selection of these using a special digital pen causes the corresponding query request along with context data to be sent to a festival application database and the response is returned to the visitor in the form of generated speech output. In addition to paper-based information browsing and transactions such as ticket booking, the digital pen can also be applied for data capture of event ratings and handwritten comments on events. The system integrates three main database components - a cross-media information platform, a content management framework for multi-channel context-aware publishing of data and the festival application database