Image Retrieval within Augmented Reality

Die vorliegende Arbeit untersucht das Potenzial von Augmented Reality zur Verbesserung von Image Retrieval Prozessen. Herausforderungen in Design und Gebrauchstauglichkeit wurden für beide Forschungsbereiche dargelegt und genutzt, um Designziele für Konzepte zu entwerfen. Eine Taxonomie für Image Retrieval in Augmented Reality wurde basierend auf der Forschungsarbeit entworfen und eingesetzt, um verwandte Arbeiten und generelle Ideen für Interaktionsmöglichkeiten zu strukturieren. Basierend auf der Taxonomie wurden Anwendungsszenarien als weitere Anforderungen für Konzepte formuliert. Mit Hilfe der generellen Ideen und Anforderungen wurden zwei umfassende Konzepte für Image Retrieval in Augmented Reality ausgearbeitet. Eins der Konzepte wurde auf einer Microsoft HoloLens umgesetzt und in einer Nutzerstudie evaluiert. Die Studie zeigt, dass das Konzept grundsätzlich positiv aufgenommen wurde und bietet Erkenntnisse über unterschiedliches Verhalten im Raum und verschiedene Suchstrategien bei der Durchführung von Image Retrieval in der erweiterten Realität.:1 Introduction 1.1 Motivation and Problem Statement 1.1.1 Augmented Reality and Head-Mounted Displays 1.1.2 Image Retrieval 1.1.3 Image Retrieval within Augmented Reality 1.2 Thesis Structure 2 Foundations of Image Retrieval and Augmented Reality 2.1 Foundations of Image Retrieval 2.1.1 Definition of Image Retrieval 2.1.2 Classification of Image Retrieval Systems 2.1.3 Design and Usability in Image Retrieval 2.2 Foundations of Augmented Reality 2.2.1 Definition of Augmented Reality 2.2.2 Augmented Reality Design and Usability 2.3 Taxonomy for Image Retrieval within Augmented Reality 2.3.1 Session Parameters 2.3.2 Interaction Process 2.3.3 Summary of the Taxonomy 3 Concepts for Image Retrieval within Augmented Reality 3.1 Related Work 3.1.1 Natural Query Specification 3.1.2 Situated Result Visualization 3.1.3 3D Result Interaction 3.1.4 Summary of Related Work 3.2 Basic Interaction Concepts for Image Retrieval in Augmented Reality 3.2.1 Natural Query Specification 3.2.2 Situated Result Visualization 3.2.3 3D Result Interaction 3.3 Requirements for Comprehensive Concepts 3.3.1 Design Goals 3.3.2 Application Scenarios 3.4 Comprehensive Concepts 3.4.1 Tangible Query Workbench 3.4.2 Situated Photograph Queries 3.4.3 Conformance of Concept Requirements 4 Prototypic Implementation of Situated Photograph Queries 4.1 Implementation Design 4.1.1 Implementation Process 4.1.2 Structure of the Implementation 4.2 Developer and User Manual 4.2.1 Setup of the Prototype 4.2.2 Usage of the Prototype 4.3 Discussion of the Prototype 5 Evaluation of Prototype and Concept by User Study 5.1 Design of the User Study 5.1.1 Usability Testing 5.1.2 Questionnaire 5.2 Results 5.2.1 Logging of User Behavior 5.2.2 Rating through Likert Scales 5.2.3 Free Text Answers and Remarks during the Study 5.2.4 Observations during the Study 5.2.5 Discussion of Results 6 Conclusion 6.1 Summary of the Present Work 6.2 Outlook on Further WorkThe present work investigates the potential of augmented reality for improving the image retrieval process. Design and usability challenges were identified for both fields of research in order to formulate design goals for the development of concepts. A taxonomy for image retrieval within augmented reality was elaborated based on research work and used to structure related work and basic ideas for interaction. Based on the taxonomy, application scenarios were formulated as further requirements for concepts. Using the basic interaction ideas and the requirements, two comprehensive concepts for image retrieval within augmented reality were elaborated. One of the concepts was implemented using a Microsoft HoloLens and evaluated in a user study. The study showed that the concept was rated generally positive by the users and provided insight in different spatial behavior and search strategies when practicing image retrieval in augmented reality.:1 Introduction 1.1 Motivation and Problem Statement 1.1.1 Augmented Reality and Head-Mounted Displays 1.1.2 Image Retrieval 1.1.3 Image Retrieval within Augmented Reality 1.2 Thesis Structure 2 Foundations of Image Retrieval and Augmented Reality 2.1 Foundations of Image Retrieval 2.1.1 Definition of Image Retrieval 2.1.2 Classification of Image Retrieval Systems 2.1.3 Design and Usability in Image Retrieval 2.2 Foundations of Augmented Reality 2.2.1 Definition of Augmented Reality 2.2.2 Augmented Reality Design and Usability 2.3 Taxonomy for Image Retrieval within Augmented Reality 2.3.1 Session Parameters 2.3.2 Interaction Process 2.3.3 Summary of the Taxonomy 3 Concepts for Image Retrieval within Augmented Reality 3.1 Related Work 3.1.1 Natural Query Specification 3.1.2 Situated Result Visualization 3.1.3 3D Result Interaction 3.1.4 Summary of Related Work 3.2 Basic Interaction Concepts for Image Retrieval in Augmented Reality 3.2.1 Natural Query Specification 3.2.2 Situated Result Visualization 3.2.3 3D Result Interaction 3.3 Requirements for Comprehensive Concepts 3.3.1 Design Goals 3.3.2 Application Scenarios 3.4 Comprehensive Concepts 3.4.1 Tangible Query Workbench 3.4.2 Situated Photograph Queries 3.4.3 Conformance of Concept Requirements 4 Prototypic Implementation of Situated Photograph Queries 4.1 Implementation Design 4.1.1 Implementation Process 4.1.2 Structure of the Implementation 4.2 Developer and User Manual 4.2.1 Setup of the Prototype 4.2.2 Usage of the Prototype 4.3 Discussion of the Prototype 5 Evaluation of Prototype and Concept by User Study 5.1 Design of the User Study 5.1.1 Usability Testing 5.1.2 Questionnaire 5.2 Results 5.2.1 Logging of User Behavior 5.2.2 Rating through Likert Scales 5.2.3 Free Text Answers and Remarks during the Study 5.2.4 Observations during the Study 5.2.5 Discussion of Results 6 Conclusion 6.1 Summary of the Present Work 6.2 Outlook on Further Wor

Visual exploration and retrieval of XML document collections with the generic system X2

This article reports on the XML retrieval system X2 which has been developed at the University of Munich over the last five years. In a typical session with X2, the user first browses a structural summary of the XML database in order to select interesting elements and keywords occurring in documents. Using this intermediate result, queries combining structure and textual references are composed semiautomatically. After query evaluation, the full set of answers is presented in a visual and structured way. X2 largely exploits the structure found in documents, queries and answers to enable new interactive visualization and exploration techniques that support mixed IR and database-oriented querying, thus bridging the gap between these three views on the data to be retrieved. Another salient characteristic of X2 which distinguishes it from other visual query systems for XML is that it supports various degrees of detailedness in the presentation of answers, as well as techniques for dynamically reordering and grouping retrieved elements once the complete answer set has been computed

Experiencing OptiqueVQS: A Multi-paradigm and Ontology-based Visual Query System for End Users

This is author's post-print version, published version available on http://link.springer.com/article/10.1007%2Fs10209-015-0404-5Data access in an enterprise setting is a determining factor for value creation processes, such as sense-making, decision-making, and intelligence analysis. Particularly, in an enterprise setting, intuitive data access tools that directly engage domain experts with data could substantially increase competitiveness and profitability. In this respect, the use of ontologies as a natural communication medium between end users and computers has emerged as a prominent approach. To this end, this article introduces a novel ontology-based visual query system, named OptiqueVQS, for end users. OptiqueVQS is built on a powerful and scalable data access platform and has a user-centric design supported by a widget-based flexible and extensible architecture allowing multiple coordinated representation and interaction paradigms to be employed. The results of a usability experiment performed with non-expert users suggest that OptiqueVQS provides a decent level of expressivity and high usability and hence is quite promising

Ontology-based end-user visual query formulation: Why, what, who, how, and which?

Value creation in an organisation is a time-sensitive and data-intensive process, yet it is often delayed and bounded by the reliance on IT experts extracting data for domain experts. Hence, there is a need for providing people who are not professional developers with the flexibility to pose relatively complex and ad hoc queries in an easy and intuitive way. In this respect, visual methods for query formulation undertake the challenge of making querying independent of users’ technical skills and the knowledge of the underlying textual query language and the structure of data. An ontology is more promising than the logical schema of the underlying data for guiding users in formulating queries, since it provides a richer vocabulary closer to the users’ understanding. However, on the one hand, today the most of world’s enterprise data reside in relational databases rather than triple stores, and on the other, visual query formulation has become more compelling due to ever-increasing data size and complexity—known as Big Data. This article presents and argues for ontology-based visual query formulation for end-users; discusses its feasibility in terms of ontology-based data access, which virtualises legacy relational databases as RDF, and the dimensions of Big Data; presents key conceptual aspects and dimensions, challenges, and requirements; and reviews, categorises, and discusses notable approaches and systems

Information Technology and Computing Topics and Their Relevance to Medical Undergraduate and Graduate Program Curricula at RIT

Two healthcare domain related programs in which this author has curricular relationships are the undergraduate Diagnostic Ultrasound (DU), and the graduate Master of Science in Health Informatics (MSHI). He teaches one course in the former and is the program coordinator for the latter. The undergraduate course is titled, “Computers in Medicine”, and is a rough 50% combination of a first-semester computing hardware course taught to our IT undergrads and another 50% of material from a textbook covering all the ways in which computing has benefitted various healthcare domains like, surgery, pharmacy, imaging, dentistry, psychiatry, remote medicine and the like. The MSHI program is a 30 semester credit hour program offered in an online format with a capstone experience (no thesis required) that was designed for professionals expecting to retool themselves for continued employment in a healthcare setting. This paper will discuss the details of the DU course and the MSHI program, the kind of computing content covered in each, and the rationale for and program design input of each. In conclusion, the reader will be left with an understanding of the what, when, how and why computing topics are necessarily required by these curricula, our justification for such, and how we might use that information in the development of future healthcare-related computing courses and potential programs. Course definition and program outline documents will be attached as appendices to the paper

Vehicle Surveillance Alarm System in Focus of Application Software

Society has been very concern about the important of safety and security that leads the manufacturing company to design several of security systems ranging from the simple to complex systems. This report present information on the research involves in Vehicle Surveillance Alarm System, which is purposely design for vehicle security and safety since most of the road transportation lack of security and safety features, which may cause vehicle fatalities that increase from year to year. Hence, the function of the Vehicle Surveillance Alarm System is to trigger an alarm if any abnormal condition happens to inform and alert the vehicle driver about the situation through the use of advance warning messages. This research covers the studies of the application software that involves with manipulating data from the serial port and displaying the data into a proper GUI design. This report features the introduction that explains the background of study, problem statement, significant of the project, objectives and the scope of study of the project research. The author also writes the literature review or theory from several researchers for supporting information or reference to support the project research. The software development methodology use by the author for the project research is taken from several steps of Software Development Life Cycle and Waterfall Model in order to suit the research needs, which also used the concept of static data applied in the prototype. Procedure identification is briefly discussed in order to complete the research and describe tools of hardware and software use to design the product. The author visualize the results of the findings and discusses more details on the serial interface, database storage and GUI interface of the system including the software chosen to accomplish the project of the Vehicle Surveillance Alarm System. The author writes the conclusions and recommendations for further research and future enhancement that includes in the final chapter of the report. The end-result of the research is the static vehicle alarm system software prototype that satisfies the basic needs of vehicle alarm system. The prototype visualize how the vehicle alarm system enables the vehicle driver to be alert by any uncertainty behaviors from the vehicle component, retrieve and view available data of the surveillance history records through the system

Sketch-based Queries in Mobile GIS-Environments

Recent achievements in the field of mobile computing and wireless communication promise data retrieval anywhere and anytime. This development provided the basis to expand GIs technology to handheld devices, such as personal digital assistants (PDAs). Although traditional GIs technology is well suited for desktop workstations, it needs to be adapted in order to satisfy the requirements of users using handheld computing devices. This adaptation is necessary because the usability of traditional GISs depends on characteristics of desktop computers, such as their relatively large user interfaces (e.g., displays, keyboards, pointing devices), considerable computing resources (i.e., CPU, memory, storage, operating systems), and high bandwidth network connectivity. Small devices possess few of these characteristics, hence, requiring new and efficient methods for interaction with spatial databases. We propose a concept that supports sketch-based querying in mobile GIs environments. This concept combines newest techniques for spatial querying and mobile technologies. Such a combination is beneficial for users because it allows them to formulate queries by drawing the desired configuration with a pen on the touch-sensitive PDA screen, and consequently avoids typing complex statements in some SQL-like query language. Client-server architectures in mobile environments are characterized by low and fluctuating bandwidth, and by frequent disconnections. We discuss client-server strategies in mobile environments, suggest an adaptive client-server architecture for geomobile querying, and analyze the performance. It is shown that adaptation to the mobile environment is necessary in order to ensure efficiency of geo-mobile queries

Towards a New Design Metaphor: Supporting Boundary Objects as Means of Knowledge Sharing in Community Networks

Working in the knowledge sector means dealing with increasing amounts of information, technology and people. Organizations as well as individuals in communities need to constantly maintain large repositories and networks of people, including colleagues, clients, experts, acquaintances and friends. This situation leads to complexity where person’s cognitive capability is insufficient when dealing with huge repositories of information and interaction. Viewing it as an individual problem has resulted in applications that highlight the need for structure and organization. We here define these applications in different levels where the first level is the office application generation, referring to “desktops” metaphors. The next generation, groupware applications, offers structure and process support for collaboration, but is still a rather limited “forum” metaphor. Our main argument is that current application generations and design metaphors are too limited when supporting the sharing of thoughts and associations in different community networks. We believe that a large portion of this problem is not related to information itself, but rather to processes of information categorization, navigation and interaction within and between communities. In our results we advocate the need for a new application generation and a new design metaphor, i.e. brainware applications based on “neural” metaphors. The result is a review of three application generations based on different design metaphors. We discuss several implications for a new design metaphor and suggest a design draft that supports boundary objects as means of knowledge sharing within and between communities