Image Retrieval within Augmented Reality

Die vorliegende Arbeit untersucht das Potenzial von Augmented Reality zur Verbesserung von Image Retrieval Prozessen. Herausforderungen in Design und Gebrauchstauglichkeit wurden für beide Forschungsbereiche dargelegt und genutzt, um Designziele für Konzepte zu entwerfen. Eine Taxonomie für Image Retrieval in Augmented Reality wurde basierend auf der Forschungsarbeit entworfen und eingesetzt, um verwandte Arbeiten und generelle Ideen für Interaktionsmöglichkeiten zu strukturieren. Basierend auf der Taxonomie wurden Anwendungsszenarien als weitere Anforderungen für Konzepte formuliert. Mit Hilfe der generellen Ideen und Anforderungen wurden zwei umfassende Konzepte für Image Retrieval in Augmented Reality ausgearbeitet. Eins der Konzepte wurde auf einer Microsoft HoloLens umgesetzt und in einer Nutzerstudie evaluiert. Die Studie zeigt, dass das Konzept grundsätzlich positiv aufgenommen wurde und bietet Erkenntnisse über unterschiedliches Verhalten im Raum und verschiedene Suchstrategien bei der Durchführung von Image Retrieval in der erweiterten Realität.:1 Introduction 1.1 Motivation and Problem Statement 1.1.1 Augmented Reality and Head-Mounted Displays 1.1.2 Image Retrieval 1.1.3 Image Retrieval within Augmented Reality 1.2 Thesis Structure 2 Foundations of Image Retrieval and Augmented Reality 2.1 Foundations of Image Retrieval 2.1.1 Definition of Image Retrieval 2.1.2 Classification of Image Retrieval Systems 2.1.3 Design and Usability in Image Retrieval 2.2 Foundations of Augmented Reality 2.2.1 Definition of Augmented Reality 2.2.2 Augmented Reality Design and Usability 2.3 Taxonomy for Image Retrieval within Augmented Reality 2.3.1 Session Parameters 2.3.2 Interaction Process 2.3.3 Summary of the Taxonomy 3 Concepts for Image Retrieval within Augmented Reality 3.1 Related Work 3.1.1 Natural Query Specification 3.1.2 Situated Result Visualization 3.1.3 3D Result Interaction 3.1.4 Summary of Related Work 3.2 Basic Interaction Concepts for Image Retrieval in Augmented Reality 3.2.1 Natural Query Specification 3.2.2 Situated Result Visualization 3.2.3 3D Result Interaction 3.3 Requirements for Comprehensive Concepts 3.3.1 Design Goals 3.3.2 Application Scenarios 3.4 Comprehensive Concepts 3.4.1 Tangible Query Workbench 3.4.2 Situated Photograph Queries 3.4.3 Conformance of Concept Requirements 4 Prototypic Implementation of Situated Photograph Queries 4.1 Implementation Design 4.1.1 Implementation Process 4.1.2 Structure of the Implementation 4.2 Developer and User Manual 4.2.1 Setup of the Prototype 4.2.2 Usage of the Prototype 4.3 Discussion of the Prototype 5 Evaluation of Prototype and Concept by User Study 5.1 Design of the User Study 5.1.1 Usability Testing 5.1.2 Questionnaire 5.2 Results 5.2.1 Logging of User Behavior 5.2.2 Rating through Likert Scales 5.2.3 Free Text Answers and Remarks during the Study 5.2.4 Observations during the Study 5.2.5 Discussion of Results 6 Conclusion 6.1 Summary of the Present Work 6.2 Outlook on Further WorkThe present work investigates the potential of augmented reality for improving the image retrieval process. Design and usability challenges were identified for both fields of research in order to formulate design goals for the development of concepts. A taxonomy for image retrieval within augmented reality was elaborated based on research work and used to structure related work and basic ideas for interaction. Based on the taxonomy, application scenarios were formulated as further requirements for concepts. Using the basic interaction ideas and the requirements, two comprehensive concepts for image retrieval within augmented reality were elaborated. One of the concepts was implemented using a Microsoft HoloLens and evaluated in a user study. The study showed that the concept was rated generally positive by the users and provided insight in different spatial behavior and search strategies when practicing image retrieval in augmented reality.:1 Introduction 1.1 Motivation and Problem Statement 1.1.1 Augmented Reality and Head-Mounted Displays 1.1.2 Image Retrieval 1.1.3 Image Retrieval within Augmented Reality 1.2 Thesis Structure 2 Foundations of Image Retrieval and Augmented Reality 2.1 Foundations of Image Retrieval 2.1.1 Definition of Image Retrieval 2.1.2 Classification of Image Retrieval Systems 2.1.3 Design and Usability in Image Retrieval 2.2 Foundations of Augmented Reality 2.2.1 Definition of Augmented Reality 2.2.2 Augmented Reality Design and Usability 2.3 Taxonomy for Image Retrieval within Augmented Reality 2.3.1 Session Parameters 2.3.2 Interaction Process 2.3.3 Summary of the Taxonomy 3 Concepts for Image Retrieval within Augmented Reality 3.1 Related Work 3.1.1 Natural Query Specification 3.1.2 Situated Result Visualization 3.1.3 3D Result Interaction 3.1.4 Summary of Related Work 3.2 Basic Interaction Concepts for Image Retrieval in Augmented Reality 3.2.1 Natural Query Specification 3.2.2 Situated Result Visualization 3.2.3 3D Result Interaction 3.3 Requirements for Comprehensive Concepts 3.3.1 Design Goals 3.3.2 Application Scenarios 3.4 Comprehensive Concepts 3.4.1 Tangible Query Workbench 3.4.2 Situated Photograph Queries 3.4.3 Conformance of Concept Requirements 4 Prototypic Implementation of Situated Photograph Queries 4.1 Implementation Design 4.1.1 Implementation Process 4.1.2 Structure of the Implementation 4.2 Developer and User Manual 4.2.1 Setup of the Prototype 4.2.2 Usage of the Prototype 4.3 Discussion of the Prototype 5 Evaluation of Prototype and Concept by User Study 5.1 Design of the User Study 5.1.1 Usability Testing 5.1.2 Questionnaire 5.2 Results 5.2.1 Logging of User Behavior 5.2.2 Rating through Likert Scales 5.2.3 Free Text Answers and Remarks during the Study 5.2.4 Observations during the Study 5.2.5 Discussion of Results 6 Conclusion 6.1 Summary of the Present Work 6.2 Outlook on Further Wor

Augmented reality meeting table: a novel multi-user interface for architectural design

Immersive virtual environments have received widespread attention as providing possible replacements for the media and systems that designers traditionally use, as well as, more generally, in providing support for collaborative work. Relatively little attention has been given to date however to the problem of how to merge immersive virtual environments into real world work settings, and so to add to the media at the disposal of the designer and the design team, rather than to replace it. In this paper we report on a research project in which optical see-through augmented reality displays have been developed together with prototype decision support software for architectural and urban design. We suggest that a critical characteristic of multi user augmented reality is its ability to generate visualisations from a first person perspective in which the scale of rendition of the design model follows many of the conventions that designers are used to. Different scales of model appear to allow designers to focus on different aspects of the design under consideration. Augmenting the scene with simulations of pedestrian movement appears to assist both in scale recognition, and in moving from a first person to a third person understanding of the design. This research project is funded by the European Commission IST program (IST-2000-28559)

Stereoscopic bimanual interaction for 3D visualization

Virtual Environments (VE) are being widely used in various research fields for several decades such as 3D visualization, education, training and games. VEs have the potential to enhance the visualization and act as a general medium for human-computer interaction (HCI). However, limited research has evaluated virtual reality (VR) display technologies, monocular and binocular depth cues, for human depth perception of volumetric (non-polygonal) datasets. In addition, a lack of standardization of three-dimensional (3D) user interfaces (UI) makes it challenging to interact with many VE systems. To address these issues, this dissertation focuses on evaluation of effects of stereoscopic and head-coupled displays on depth judgment of volumetric dataset. It also focuses on evaluation of a two-handed view manipulation techniques which support simultaneous 7 degree-of-freedom (DOF) navigation (x,y,z + yaw,pitch,roll + scale) in a multi-scale virtual environment (MSVE). Furthermore, this dissertation evaluates auto-adjustment of stereo view parameters techniques for stereoscopic fusion problems in a MSVE. Next, this dissertation presents a bimanual, hybrid user interface which combines traditional tracking devices with computer-vision based "natural" 3D inputs for multi-dimensional visualization in a semi-immersive desktop VR system. In conclusion, this dissertation provides a guideline for research design for evaluating UI and interaction techniques

Light on horizontal interactive surfaces: Input space for tabletop computing

In the last 25 years we have witnessed the rise and growth of interactive tabletop research, both in academic and in industrial settings. The rising demand for the digital support of human activities motivated the need to bring computational power to table surfaces. In this article, we review the state of the art of tabletop computing, highlighting core aspects that frame the input space of interactive tabletops: (a) developments in hardware technologies that have caused the proliferation of interactive horizontal surfaces and (b) issues related to new classes of interaction modalities (multitouch, tangible, and touchless). A classification is presented that aims to give a detailed view of the current development of this research area and define opportunities and challenges for novel touch- and gesture-based interactions between the human and the surrounding computational environment. © 2014 ACM.This work has been funded by Integra (Amper Sistemas and CDTI, Spanish Ministry of Science and Innovation) and TIPEx (TIN2010-19859-C03-01) projects and Programa de Becas y Ayudas para la Realización de Estudios Oficiales de Máster y Doctorado en la Universidad Carlos III de Madrid, 2010

Analysis domain model for shared virtual environments

The field of shared virtual environments, which also encompasses online games and social 3D environments, has a system landscape consisting of multiple solutions that share great functional overlap. However, there is little system interoperability between the different solutions. A shared virtual environment has an associated problem domain that is highly complex raising difficult challenges to the development process, starting with the architectural design of the underlying system. This paper has two main contributions. The first contribution is a broad domain analysis of shared virtual environments, which enables developers to have a better understanding of the whole rather than the part(s). The second contribution is a reference domain model for discussing and describing solutions - the Analysis Domain Model

Representing and Indexing Archaeological Information

The need to preserve and remember the past is a particular human trait. The richness of our cultural history is approached by a vast array of disciplines, that investigate and manage it. However, their effectiveness can be hindered by several technical issues. One of the concerns of experts in this area is the way the importance of cultural heritage is communicated in order to cultivate interest, curiosity and respect. Another concern is the lack of suitable tools that can handle the dimension and complexity of the collections with which they interact. With the emergence of digital tools and the creation of online repositories for the collections of cultural institutions, it is possible to suggest different solutions to tackle these problems. The proposed solution aims to facilitate access and interaction with cultural information, through the implementation of an application capable of integrating multiple forms of representation of historical artifacts. The application tackles two problems that arise from distinct goals. One is the need to represent, in a single view, collections of related items from different repositories. The other is how to, effectively, communicate the information associated with an artifact and its context. This MSc dissertation is part of a collaborative effort between NOVA LINCS researchers and several archaeological institutions of the Iberian Extremadura, aiming to develop tools that will support research and help sharing the cultural wealth of archaeological sites and artifacts from the region. In this dissertation, the developed application covers a general view of the aforementioned problems, while being flexible to the customization of the representation of cultural data. The solution was evaluated on usability and effectiveness on reaching the proposed goals, during a process that involved target audience users and experts in the area of culture and history, as well as human-computer interaction. The results provided positive conclusions

ISAR: Ein Autorensystem für Interaktive Tische

Developing augmented reality systems involves several challenges, that prevent end users and experts from non-technical domains, such as education, to experiment with this technology. In this research we introduce ISAR, an authoring system for augmented reality tabletops targeting users from non-technical domains. ISAR allows non-technical users to create their own interactive tabletop applications and experiment with the use of this technology in domains such as educations, industrial training, and medical rehabilitation.Die Entwicklung von Augmented-Reality-Systemen ist mit mehreren Herausforderungen verbunden, die Endbenutzer und Experten aus nicht-technischen Bereichen, wie z.B. dem Bildungswesen, daran hindern, mit dieser Technologie zu experimentieren. In dieser Forschung stellen wir ISAR vor, ein Autorensystem für Augmented-Reality-Tabletops, das sich an Benutzer aus nicht-technischen Bereichen richtet. ISAR ermöglicht es nicht-technischen Anwendern, ihre eigenen interaktiven Tabletop-Anwendungen zu erstellen und mit dem Einsatz dieser Technologie in Bereichen wie Bildung, industrieller Ausbildung und medizinischer Rehabilitation zu experimentieren

Multimodal metaphors for generic interaction tasks in virtual environments

Virtual Reality (VR) Systeme bieten zusätzliche Ein- und Ausgabekanäle für die Interaktion zwischen Mensch und Computer in virtuellen Umgebungen. Solche VR Technologien ermöglichen den Anwendern bessere Einblicke in hochkomplexe Datenmengen, stellen allerdings auch hohe Anforderungen an den Benutzer bezüglich der Fähigkeiten mit virtuellen Objekten zu interagieren. In dieser Arbeit werden sowohl die Entwicklung und Evaluierung neuer multimodaler Interaktionsmetaphern für generische Interaktionsaufgaben in virtuellen Umgebungen vorgestellt und diskutiert. Anhand eines VR Systems wird der Einsatz dieser Konzepte an zwei Fallbeispielen aus den Domänen der 3D-Stadtvisualisierung und seismischen Volumendarstellung aufgezeigt