Spatially Aware Computing for Natural Interaction

Spatial information refers to the location of an object in a physical or digital world. Besides, it also includes the relative position of an object related to other objects around it. In this dissertation, three systems are designed and developed. All of them apply spatial information in different fields. The ultimate goal is to increase the user friendliness and efficiency in those applications by utilizing spatial information. The first system is a novel Web page data extraction application, which takes advantage of 2D spatial information to discover structured records from a Web page. The extracted information is useful to re-organize the layout of a Web page to fit mobile browsing. The second application utilizes the 3D spatial information of a mobile device within a large paper-based workspace to implement interactive paper that combines the merits of paper documents and mobile devices. This application can overlay digital information on top of a paper document based on the location of a mobile device within a workspace. The third application further integrates 3D space information with sound detection to realize an automatic camera management system. This application automatically controls multiple cameras in a conference room, and creates an engaging video by intelligently switching camera shots among meeting participants based on their activities. Evaluations have been made on all three applications, and the results are promising. In summary, this dissertation comprehensively explores the usage of spatial information in various applications to improve the usability

Exploring information delivery on a guided tour using mobile projection and visual markers

We present an in-the-wild user study (n=27) investigating the combination of two mobile technologies – picoprojectors and marker based information browsing. We studied a tour, where the tour guide used combinations of fixed and projected elements to present information, and compare four cases: A) as a baseline, a traditional paper poster, B) a projected poster, C) a printed paper fiducial marker, viewed through a mobile device browser application, and D) a projected fiducial marker viewed through a mobile device browser application. As a contribution, we present a novel approach to ad hoc projection of markers, and the findings of the user study. Here, the salient findings suggest that the techniques using markers have the potential to enhance the tour participants’ engagement with the tour guide, attention, group cohesion and responsiveness to contextual factors, but face practical challenges due to lighting conditions and image stability

PaperChains: Dynamic Sketch+Voice Annotations

In this paper we present a novel interface for collaborative creation of evolving audio-visual documents. PaperChains allows users to sketch on paper and then augment with digital audio, allowing both the physical and digital objects to evolve simultaneously over time. The technique we have developed focuses on affordability and accessibility in its design, using standard cameraphones and telephone connections, which allow it to be used in regions where literacy, technological ex-perience and data connections cannot necessarily be taken for granted. The main use-case that we focus on in this paper is for collaborative storytelling, an area which has been well studied and previously proven to be of value in resource constrained environments. To investigate the relevance of the approach in these contexts, we undertook two usability evaluations in India and South Africa. Results from these investigations indicate users ’ ability to both create and interpret stories using the software, as well as demonstrating high overall usability and enjoyment. We end with a discussion of the implications of our design and opportunities for use in other contexts. Author Keywords Sketching; audio; photographs; cameraphones

SnapLink: Fast and Accurate Vision-Based Appliance Control in Large Commercial Buildings

As the number and heterogeneity of appliances in smart buildings increases, identifying and controlling them becomes challenging. Existing methods face various challenges when deployed in large commercial buildings. For example, voice command assistants require users to memorize many control commands. Attaching Bluetooth dongles or QR codes to appliances introduces considerable deployment overhead. In comparison, identifying an appliance by simply pointing a smartphone camera at it and controlling the appliance using a graphical overlay interface is more intuitive. We introduce SnapLink, a responsive and accurate vision-based system for mobile appliance identification and interaction using image localization. Compared to the image retrieval approaches used in previous vision-based appliance control systems, SnapLink exploits 3D models to improve identification accuracy and reduce deployment overhead via quick video captures and a simplified labeling process. We also introduce a feature sub-sampling mechanism to achieve low latency at the scale of a commercial building. To evaluate SnapLink, we collected training videos from 39 rooms to represent the scale of a modern commercial building. It achieves a 94% successful appliance identification rate among 1526 test images of 179 appliances within 120 ms average server processing time. Furthermore, we show that SnapLink is robust to viewing angle and distance differences, illumination changes, as well as daily changes in the environment. We believe the SnapLink use case is not limited to appliance control: it has the potential to enable various new smart building applications.</jats:p

Interaction avec un picoprojecteur : État de l'art et analyse des attentes des utilisateurs

4 pagesNational audienceUn picoprojecteur est un vidéoprojecteur portatif aux dimensions réduites. On appelle également picophone, un smartphone qui intègre un tel dispositif. Encore très peu diffusé, ce nouveau dispositif interactif mobile est commercialisé depuis 2 ans. Cet article dresse un état de l'art des recherches actuellement menées sur le sujet, puis présente les attentes d'un panel de 50 utilisateurs potentiels

Image Retrieval within Augmented Reality

Die vorliegende Arbeit untersucht das Potenzial von Augmented Reality zur Verbesserung von Image Retrieval Prozessen. Herausforderungen in Design und Gebrauchstauglichkeit wurden für beide Forschungsbereiche dargelegt und genutzt, um Designziele für Konzepte zu entwerfen. Eine Taxonomie für Image Retrieval in Augmented Reality wurde basierend auf der Forschungsarbeit entworfen und eingesetzt, um verwandte Arbeiten und generelle Ideen für Interaktionsmöglichkeiten zu strukturieren. Basierend auf der Taxonomie wurden Anwendungsszenarien als weitere Anforderungen für Konzepte formuliert. Mit Hilfe der generellen Ideen und Anforderungen wurden zwei umfassende Konzepte für Image Retrieval in Augmented Reality ausgearbeitet. Eins der Konzepte wurde auf einer Microsoft HoloLens umgesetzt und in einer Nutzerstudie evaluiert. Die Studie zeigt, dass das Konzept grundsätzlich positiv aufgenommen wurde und bietet Erkenntnisse über unterschiedliches Verhalten im Raum und verschiedene Suchstrategien bei der Durchführung von Image Retrieval in der erweiterten Realität.:1 Introduction 1.1 Motivation and Problem Statement 1.1.1 Augmented Reality and Head-Mounted Displays 1.1.2 Image Retrieval 1.1.3 Image Retrieval within Augmented Reality 1.2 Thesis Structure 2 Foundations of Image Retrieval and Augmented Reality 2.1 Foundations of Image Retrieval 2.1.1 Definition of Image Retrieval 2.1.2 Classification of Image Retrieval Systems 2.1.3 Design and Usability in Image Retrieval 2.2 Foundations of Augmented Reality 2.2.1 Definition of Augmented Reality 2.2.2 Augmented Reality Design and Usability 2.3 Taxonomy for Image Retrieval within Augmented Reality 2.3.1 Session Parameters 2.3.2 Interaction Process 2.3.3 Summary of the Taxonomy 3 Concepts for Image Retrieval within Augmented Reality 3.1 Related Work 3.1.1 Natural Query Specification 3.1.2 Situated Result Visualization 3.1.3 3D Result Interaction 3.1.4 Summary of Related Work 3.2 Basic Interaction Concepts for Image Retrieval in Augmented Reality 3.2.1 Natural Query Specification 3.2.2 Situated Result Visualization 3.2.3 3D Result Interaction 3.3 Requirements for Comprehensive Concepts 3.3.1 Design Goals 3.3.2 Application Scenarios 3.4 Comprehensive Concepts 3.4.1 Tangible Query Workbench 3.4.2 Situated Photograph Queries 3.4.3 Conformance of Concept Requirements 4 Prototypic Implementation of Situated Photograph Queries 4.1 Implementation Design 4.1.1 Implementation Process 4.1.2 Structure of the Implementation 4.2 Developer and User Manual 4.2.1 Setup of the Prototype 4.2.2 Usage of the Prototype 4.3 Discussion of the Prototype 5 Evaluation of Prototype and Concept by User Study 5.1 Design of the User Study 5.1.1 Usability Testing 5.1.2 Questionnaire 5.2 Results 5.2.1 Logging of User Behavior 5.2.2 Rating through Likert Scales 5.2.3 Free Text Answers and Remarks during the Study 5.2.4 Observations during the Study 5.2.5 Discussion of Results 6 Conclusion 6.1 Summary of the Present Work 6.2 Outlook on Further WorkThe present work investigates the potential of augmented reality for improving the image retrieval process. Design and usability challenges were identified for both fields of research in order to formulate design goals for the development of concepts. A taxonomy for image retrieval within augmented reality was elaborated based on research work and used to structure related work and basic ideas for interaction. Based on the taxonomy, application scenarios were formulated as further requirements for concepts. Using the basic interaction ideas and the requirements, two comprehensive concepts for image retrieval within augmented reality were elaborated. One of the concepts was implemented using a Microsoft HoloLens and evaluated in a user study. The study showed that the concept was rated generally positive by the users and provided insight in different spatial behavior and search strategies when practicing image retrieval in augmented reality.:1 Introduction 1.1 Motivation and Problem Statement 1.1.1 Augmented Reality and Head-Mounted Displays 1.1.2 Image Retrieval 1.1.3 Image Retrieval within Augmented Reality 1.2 Thesis Structure 2 Foundations of Image Retrieval and Augmented Reality 2.1 Foundations of Image Retrieval 2.1.1 Definition of Image Retrieval 2.1.2 Classification of Image Retrieval Systems 2.1.3 Design and Usability in Image Retrieval 2.2 Foundations of Augmented Reality 2.2.1 Definition of Augmented Reality 2.2.2 Augmented Reality Design and Usability 2.3 Taxonomy for Image Retrieval within Augmented Reality 2.3.1 Session Parameters 2.3.2 Interaction Process 2.3.3 Summary of the Taxonomy 3 Concepts for Image Retrieval within Augmented Reality 3.1 Related Work 3.1.1 Natural Query Specification 3.1.2 Situated Result Visualization 3.1.3 3D Result Interaction 3.1.4 Summary of Related Work 3.2 Basic Interaction Concepts for Image Retrieval in Augmented Reality 3.2.1 Natural Query Specification 3.2.2 Situated Result Visualization 3.2.3 3D Result Interaction 3.3 Requirements for Comprehensive Concepts 3.3.1 Design Goals 3.3.2 Application Scenarios 3.4 Comprehensive Concepts 3.4.1 Tangible Query Workbench 3.4.2 Situated Photograph Queries 3.4.3 Conformance of Concept Requirements 4 Prototypic Implementation of Situated Photograph Queries 4.1 Implementation Design 4.1.1 Implementation Process 4.1.2 Structure of the Implementation 4.2 Developer and User Manual 4.2.1 Setup of the Prototype 4.2.2 Usage of the Prototype 4.3 Discussion of the Prototype 5 Evaluation of Prototype and Concept by User Study 5.1 Design of the User Study 5.1.1 Usability Testing 5.1.2 Questionnaire 5.2 Results 5.2.1 Logging of User Behavior 5.2.2 Rating through Likert Scales 5.2.3 Free Text Answers and Remarks during the Study 5.2.4 Observations during the Study 5.2.5 Discussion of Results 6 Conclusion 6.1 Summary of the Present Work 6.2 Outlook on Further Wor

Government E-Service Delivery in Rural Bangladesh: A Public-Private Partnership Approach

This paper reviews and analyzes government e-service delivery through Union Digital Centers (UDCs), being identified as telecenters. With the emergence of e-governance, e-services delivery has become expedited across different countries in the world including Bangladesh to foster socio-economic development. However, the concept of ‘digital divide’ or ‘digital gap’ limits the ultimate success of e-delivery services by increasing the gap between rich and poor. In connection to the digital gap, literature evidence that telecenters have largely failed to reach targeted hardcore poor. This research has used a survey technique to collect data from 383 respondents located at 14 different UDCs and seven divisions. The findings of the paper contemplate a positive response in terms of availability, cost, convenience, and delivery of services. Nevertheless, strengthening the approach of Public-Private Partnerships (PPP) remains a necessity to reach the success goal in e-governance. This study would be particularly helpful for practitioners or government policy-making agencies to identify perceptions on e-services at root level. Keywords: telecenters, e-governance, e-service delivery, socio-economic development, public-private partnership, Bangladesh DOI: 10.7176/EJBM/11-2-1