A Spatial Informance Design Method to Elicit Early Interface Prototypes for Augmented Reality

Designing for augmented reality (AR) applications is difficult and expensive. A rapid system for the early design process of spatial interfaces is required. Previous research has used video for mobile AR design, but this is not extensible to head-mounted AR. AR is an emergent technology with no prior design precedent, requiring designers to allow free speculation or risk the pitfalls of ‘path dependence’. In this paper, a participatory elicitation method we call ‘spatial informance design’ is presented. We found combining ‘informance design’, ‘Wizard of Oz’, improvisation, and ‘paper prototyping’, to be a fast and lightweight solution for ideation of rich designs for spatial interfaces. A study using our method with 11 participants, produced similar and wildly different interface configurations and interactions for an augmented reality email application. Based on our findings we propose design implications and an evaluation of our method using spatial informance for the design of head-mounted AR applications

Source Code Interaction on Touchscreens

Direct interaction with touchscreens has become a primary way of using a device. This work seeks to devise interaction methods for editing textual source code on touch-enabled devices. With the advent of the “Post-PC Era”, touch-centric interaction has received considerable attention in both research and development. However, various limitations have impeded widespread adoption of programming environments on modern platforms. Previous attempts have mainly been successful by simplifying or constraining conventional programming but have only insufficiently supported source code written in mainstream programming languages. This work includes the design, development, and evaluation of techniques for editing, selecting, and creating source code on touchscreens. The results contribute to text editing and entry methods by taking the syntax and structure of programming languages into account while exploiting the advantages of gesture-driven control. Furthermore, this work presents the design and software architecture of a mobile development environment incorporating touch-enabled modules for typical software development tasks

Designing Hybrid Interactions through an Understanding of the Affordances of Physical and Digital Technologies

Two recent technological advances have extended the diversity of domains and social contexts of Human-Computer Interaction: the embedding of computing capabilities into physical hand-held objects, and the emergence of large interactive surfaces, such as tabletops and wall boards. Both interactive surfaces and small computational devices usually allow for direct and space-multiplex input, i.e., for the spatial coincidence of physical action and digital output, in multiple points simultaneously. Such a powerful combination opens novel opportunities for the design of what are considered as hybrid interactions in this work. This thesis explores the affordances of physical interaction as resources for interface design of such hybrid interactions. The hybrid systems that are elaborated in this work are envisioned to support specific social and physical contexts, such as collaborative cooking in a domestic kitchen, or collaborative creativity in a design process. In particular, different aspects of physicality characteristic of those specific domains are explored, with the aim of promoting skill transfer across domains. irst, different approaches to the design of space-multiplex, function-specific interfaces are considered and investigated. Such design approaches build on related work on Graspable User Interfaces and extend the design space to direct touch interfaces such as touch-sensitive surfaces, in different sizes and orientations (i.e., tablets, interactive tabletops, and walls). These approaches are instantiated in the design of several experience prototypes: These are evaluated in different settings to assess the contextual implications of integrating aspects of physicality in the design of the interface. Such implications are observed both at the pragmatic level of interaction (i.e., patterns of users' behaviors on first contact with the interface), as well as on user' subjective response. The results indicate that the context of interaction affects the perception of the affordances of the system, and that some qualities of physicality such as the 3D space of manipulation and relative haptic feedback can affect the feeling of engagement and control. Building on these findings, two controlled studies are conducted to observe more systematically the implications of integrating some of the qualities of physical interaction into the design of hybrid ones. The results indicate that, despite the fact that several aspects of physical interaction are mimicked in the interface, the interaction with digital media is quite different and seems to reveal existing mental models and expectations resulting from previous experience with the WIMP paradigm on the desktop PC

Beat Browser

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (p. 51-53).Beat Browser is a music browsing environment that delivers immediate audio feedback while browsing arbitrarily large music collections. The goal of Beat Browser is to give users a sense of exploring "live" and continuous audio while rapidly moving between sources by mouse. It appears their entire universe of music is playing all the time, whether they're there listening or not. Beat Browser's Universal Time Base architecture keeps a central clock running that manages the playback position of every piece of music launclied, orchestrating this perceptual illusion.by Jefferey D. Goldenson.S.M

Experimental Object-Oriented Modelling

This thesis examines object-oriented modelling in experimental system development. Object-oriented modelling aims at representing concepts and phenomena of a problem domain in terms of classes and objects. Experimental system development seeks active experimentation in a system development project through, e.g., technical prototyping and active user involvement. We introduce and examine "experimental object-oriented modelling" as the intersection of these practices

Cruiser and PhoTable: Exploring Tabletop User Interface Software for Digital Photograph Sharing and Story Capture

Digital photography has not only changed the nature of photography and the photographic process, but also the manner in which we share photographs and tell stories about them. Some traditional methods, such as the family photo album or passing around piles of recently developed snapshots, are lost to us without requiring the digital photos to be printed. The current, purely digital, methods of sharing do not provide the same experience as printed photographs, and they do not provide effective face-to-face social interaction around photographs, as experienced during storytelling. Research has found that people are often dissatisfied with sharing photographs in digital form. The recent emergence of the tabletop interface as a viable multi-user direct-touch interactive large horizontal display has provided the hardware that has the potential to improve our collocated activities such as digital photograph sharing. However, while some software to communicate with various tabletop hardware technologies exists, software aspects of tabletop user interfaces are still at an early stage and require careful consideration in order to provide an effective, multi-user immersive interface that arbitrates the social interaction between users, without the necessary computer-human interaction interfering with the social dialogue. This thesis presents PhoTable, a social interface allowing people to effectively share, and tell stories about, recently taken, unsorted digital photographs around an interactive tabletop. In addition, the computer-arbitrated digital interaction allows PhoTable to capture the stories told, and associate them as audio metadata to the appropriate photographs. By leveraging the tabletop interface and providing a highly usable and natural interaction we can enable users to become immersed in their social interaction, telling stories about their photographs, and allow the computer interaction to occur as a side-effect of the social interaction. Correlating the computer interaction with the corresponding audio allows PhoTable to annotate an automatically created digital photo album with audible stories, which may then be archived. These stories remain useful for future sharing -- both collocated sharing and remote (e.g. via the Internet) -- and also provide a personal memento both of the event depicted in the photograph (e.g. as a reminder) and of the enjoyable photo sharing experience at the tabletop. To provide the necessary software to realise an interface such as PhoTable, this thesis explored the development of Cruiser: an efficient, extensible and reusable software framework for developing tabletop applications. Cruiser contributes a set of programming libraries and the necessary application framework to facilitate the rapid and highly flexible development of new tabletop applications. It uses a plugin architecture that encourages code reuse, stability and easy experimentation, and leverages the dedicated computer graphics hardware and multi-core processors of modern consumer-level systems to provide a responsive and immersive interactive tabletop user interface that is agnostic to the tabletop hardware and operating platform, using efficient, native cross-platform code. Cruiser's flexibility has allowed a variety of novel interactive tabletop applications to be explored by other researchers using the framework, in addition to PhoTable. To evaluate Cruiser and PhoTable, this thesis follows recommended practices for systems evaluation. The design rationale is framed within the above scenario and vision which we explore further, and the resulting design is critically analysed based on user studies, heuristic evaluation and a reflection on how it evolved over time. The effectiveness of Cruiser was evaluated in terms of its ability to realise PhoTable, use of it by others to explore many new tabletop applications, and an analysis of performance and resource usage. Usability, learnability and effectiveness of PhoTable was assessed on three levels: careful usability evaluations of elements of the interface; informal observations of usability when Cruiser was available to the public in several exhibitions and demonstrations; and a final evaluation of PhoTable in use for storytelling, where this had the side effect of creating a digital photo album, consisting of the photographs users interacted with on the table and associated audio annotations which PhoTable automatically extracted from the interaction. We conclude that our approach to design has resulted in an effective framework for creating new tabletop interfaces. The parallel goal of exploring the potential for tabletop interaction as a new way to share digital photographs was realised in PhoTable. It is able to support the envisaged goal of an effective interface for telling stories about one's photos. As a serendipitous side-effect, PhoTable was effective in the automatic capture of the stories about individual photographs for future reminiscence and sharing. This work provides foundations for future work in creating new ways to interact at a tabletop and to the ways to capture personal stories around digital photographs for sharing and long-term preservation

Virtual library:a technical implementation for a virtual reality library interface

Abstract. Libraries, as traditional information storages, serve an important role in providing free access to civilization’s knowledge to the public. In the advent of the information society, conventional means of knowledge seeking slowly become obsolete. As technologies advance, libraries shall discover how available inventions could be utilized in enhancing library services and raising public awareness. The offer to discover Virtual Library project was given to the author by the Center for Ubiquitous Computing in the University of Oulu, with the idea originating from Oulu City Library. The goal of the project was to implement an interactive application that would provide a unique experience for library users and show off capabilities of modern virtual technologies. The requirements for it were elicited and refined in a series of participatory design workshops held in Oulu City Library, and as a result, the content of initial web prototype was utilized in the creation of standalone virtual reality application on Unreal Engine 4. The application works with Oculus Rift headset and Oculus Touch motion controllers and lets the user explore the virtual model of Oulu City Library premises as well as some fictional places, added for diverse experience, and experience several activities. Virtual Library was evaluated in a series of testing sessions held in the Oulu City Library with a total of 12 participants, which were primarily the library’s staff members. Overall, the application was positively acclaimed, providing an interesting and unusual library experience, and presenting capabilities of modern virtual reality technologies.Virtuaalikirjasto : tekninen kuvaus virtuaalitodellisuutta hyödyntävälle kirjastopalvelulle. Tiivistelmä. Kirjastoilla on perinteisesti tärkeä rooli tiedon välittäjinä ja tallentajina. Kirjastojen tulee myös mahdollistaa pääsy tiedon ääreen. Siirryttäessä lähemmäksi tietoyhteiskuntaa, perinteiset tavat käsitellä ja havainnoida tietoa ovat jäämässä historiaan. Myös kirjastot kehittyvät ja oppivat kuinka uusia teknologioita on mahdollista hyödyntää tiedon välittämisessä yhteisöille. Virtuaalikirjasto-projekti lähti Oulun kaupungin kirjaston pyynnöstä Oulun yliopiston Jokapaikan tietotekniikan tutkimusryhmälle. Projektin päämääränä oli toteuttaa interaktiivinen sovellus, joka tarjoaisi kirjaston asiakkaille elämyksellistä sisältöä samalla havainnollistaen virtuaalitodellisuuden ja siihen liittyvien teknologioiden mahdollisuuksia. Sovelluksen toteutusta hiottiin Oulun kaupunginkirjastossa sarjalla osallistavan suunnittelun mukaisia työpajoja. Virtuaalitodellisuussovellus toteutettiin Unreal Engine 4 -pelimoottorilla. Se toimii Oculus Rift -virtuaalilaseilla ja Oculus Touch -ohjaimilla. Sovelluksessa on mahdollista kulkea vapaasti virtuaalisessa Oulun kaupunginkirjastossa, sekä tilaan liitetyissä mielikuvituksellisemmissa maailmoissa. Virtuaalikirjasto-sovellus evaluoitiin Oulun kaupunginkirjastossa sarjalla käyttäjätestejä. Testikäyttäjiä oli kaikkiaan 12 ja he olivat pääosin kirjaston työntekijöitä. Sovelluksen vastaanotto oli positiivinen. Käyttäjät totesivat sen havainnollistavan käytettävissä olevan tekniikan mahdollisuuksia samalla tarjoten mielenkiintoisen ja epätavallisen kirjastokokemuksen