A Generic Framework and Library for Exploration of Small Multiples through Interactive Piling

Small multiples are miniature representations of visual information used generically across many domains. Handling large numbers of small multiples imposes challenges on many analytic tasks like inspection, comparison, navigation, or annotation. To address these challenges, we developed a framework and implemented a library called Piling.js for designing interactive piling interfaces. Based on the piling metaphor, such interfaces afford flexible organization, exploration, and comparison of large numbers of small multiples by interactively aggregating visual objects into piles. Based on a systematic analysis of previous work, we present a structured design space to guide the design of visual piling interfaces. To enable designers to efficiently build their own visual piling interfaces, Piling.js provides a declarative interface to avoid having to write low-level code and implements common aspects of the design space. An accompanying GUI additionally supports the dynamic configuration of the piling interface. We demonstrate the expressiveness of Piling.js with examples from machine learning, immunofluorescence microscopy, genomics, and public health.Comment: - Extended Section 4 to improve the clarity of our rationale - Expanded Section 7 to elaborate on the intended target user, the lessons learned from implementing the use cases, and the limitations of visual piling interfaces - Added Figure S1 and S4 and Table S1 to the supplementary material - Improved the clarity of our writing in several other sections, and we corrected grammar and typo

Image Retrieval within Augmented Reality

Die vorliegende Arbeit untersucht das Potenzial von Augmented Reality zur Verbesserung von Image Retrieval Prozessen. Herausforderungen in Design und Gebrauchstauglichkeit wurden für beide Forschungsbereiche dargelegt und genutzt, um Designziele für Konzepte zu entwerfen. Eine Taxonomie für Image Retrieval in Augmented Reality wurde basierend auf der Forschungsarbeit entworfen und eingesetzt, um verwandte Arbeiten und generelle Ideen für Interaktionsmöglichkeiten zu strukturieren. Basierend auf der Taxonomie wurden Anwendungsszenarien als weitere Anforderungen für Konzepte formuliert. Mit Hilfe der generellen Ideen und Anforderungen wurden zwei umfassende Konzepte für Image Retrieval in Augmented Reality ausgearbeitet. Eins der Konzepte wurde auf einer Microsoft HoloLens umgesetzt und in einer Nutzerstudie evaluiert. Die Studie zeigt, dass das Konzept grundsätzlich positiv aufgenommen wurde und bietet Erkenntnisse über unterschiedliches Verhalten im Raum und verschiedene Suchstrategien bei der Durchführung von Image Retrieval in der erweiterten Realität.:1 Introduction 1.1 Motivation and Problem Statement 1.1.1 Augmented Reality and Head-Mounted Displays 1.1.2 Image Retrieval 1.1.3 Image Retrieval within Augmented Reality 1.2 Thesis Structure 2 Foundations of Image Retrieval and Augmented Reality 2.1 Foundations of Image Retrieval 2.1.1 Definition of Image Retrieval 2.1.2 Classification of Image Retrieval Systems 2.1.3 Design and Usability in Image Retrieval 2.2 Foundations of Augmented Reality 2.2.1 Definition of Augmented Reality 2.2.2 Augmented Reality Design and Usability 2.3 Taxonomy for Image Retrieval within Augmented Reality 2.3.1 Session Parameters 2.3.2 Interaction Process 2.3.3 Summary of the Taxonomy 3 Concepts for Image Retrieval within Augmented Reality 3.1 Related Work 3.1.1 Natural Query Specification 3.1.2 Situated Result Visualization 3.1.3 3D Result Interaction 3.1.4 Summary of Related Work 3.2 Basic Interaction Concepts for Image Retrieval in Augmented Reality 3.2.1 Natural Query Specification 3.2.2 Situated Result Visualization 3.2.3 3D Result Interaction 3.3 Requirements for Comprehensive Concepts 3.3.1 Design Goals 3.3.2 Application Scenarios 3.4 Comprehensive Concepts 3.4.1 Tangible Query Workbench 3.4.2 Situated Photograph Queries 3.4.3 Conformance of Concept Requirements 4 Prototypic Implementation of Situated Photograph Queries 4.1 Implementation Design 4.1.1 Implementation Process 4.1.2 Structure of the Implementation 4.2 Developer and User Manual 4.2.1 Setup of the Prototype 4.2.2 Usage of the Prototype 4.3 Discussion of the Prototype 5 Evaluation of Prototype and Concept by User Study 5.1 Design of the User Study 5.1.1 Usability Testing 5.1.2 Questionnaire 5.2 Results 5.2.1 Logging of User Behavior 5.2.2 Rating through Likert Scales 5.2.3 Free Text Answers and Remarks during the Study 5.2.4 Observations during the Study 5.2.5 Discussion of Results 6 Conclusion 6.1 Summary of the Present Work 6.2 Outlook on Further WorkThe present work investigates the potential of augmented reality for improving the image retrieval process. Design and usability challenges were identified for both fields of research in order to formulate design goals for the development of concepts. A taxonomy for image retrieval within augmented reality was elaborated based on research work and used to structure related work and basic ideas for interaction. Based on the taxonomy, application scenarios were formulated as further requirements for concepts. Using the basic interaction ideas and the requirements, two comprehensive concepts for image retrieval within augmented reality were elaborated. One of the concepts was implemented using a Microsoft HoloLens and evaluated in a user study. The study showed that the concept was rated generally positive by the users and provided insight in different spatial behavior and search strategies when practicing image retrieval in augmented reality.:1 Introduction 1.1 Motivation and Problem Statement 1.1.1 Augmented Reality and Head-Mounted Displays 1.1.2 Image Retrieval 1.1.3 Image Retrieval within Augmented Reality 1.2 Thesis Structure 2 Foundations of Image Retrieval and Augmented Reality 2.1 Foundations of Image Retrieval 2.1.1 Definition of Image Retrieval 2.1.2 Classification of Image Retrieval Systems 2.1.3 Design and Usability in Image Retrieval 2.2 Foundations of Augmented Reality 2.2.1 Definition of Augmented Reality 2.2.2 Augmented Reality Design and Usability 2.3 Taxonomy for Image Retrieval within Augmented Reality 2.3.1 Session Parameters 2.3.2 Interaction Process 2.3.3 Summary of the Taxonomy 3 Concepts for Image Retrieval within Augmented Reality 3.1 Related Work 3.1.1 Natural Query Specification 3.1.2 Situated Result Visualization 3.1.3 3D Result Interaction 3.1.4 Summary of Related Work 3.2 Basic Interaction Concepts for Image Retrieval in Augmented Reality 3.2.1 Natural Query Specification 3.2.2 Situated Result Visualization 3.2.3 3D Result Interaction 3.3 Requirements for Comprehensive Concepts 3.3.1 Design Goals 3.3.2 Application Scenarios 3.4 Comprehensive Concepts 3.4.1 Tangible Query Workbench 3.4.2 Situated Photograph Queries 3.4.3 Conformance of Concept Requirements 4 Prototypic Implementation of Situated Photograph Queries 4.1 Implementation Design 4.1.1 Implementation Process 4.1.2 Structure of the Implementation 4.2 Developer and User Manual 4.2.1 Setup of the Prototype 4.2.2 Usage of the Prototype 4.3 Discussion of the Prototype 5 Evaluation of Prototype and Concept by User Study 5.1 Design of the User Study 5.1.1 Usability Testing 5.1.2 Questionnaire 5.2 Results 5.2.1 Logging of User Behavior 5.2.2 Rating through Likert Scales 5.2.3 Free Text Answers and Remarks during the Study 5.2.4 Observations during the Study 5.2.5 Discussion of Results 6 Conclusion 6.1 Summary of the Present Work 6.2 Outlook on Further Wor

Interaction for Immersive Analytics

International audienceIn this chapter, we briefly review the development of natural user interfaces and discuss their role in providing human-computer interaction that is immersive in various ways. Then we examine some opportunities for how these technologies might be used to better support data analysis tasks. Specifically, we review and suggest some interaction design guidelines for immersive analytics. We also review some hardware setups for data visualization that are already archetypal. Finally, we look at some emerging system designs that suggest future directions

Interactive Visualization Lenses:: Natural Magic Lens Interaction for Graph Visualization

Information visualization is an important research field concerned with making sense and inferring knowledge from data collections. Graph visualizations are specific techniques for data representation relevant in diverse application domains among them biology, software-engineering, and business finance. These data visualizations benefit from the display space provided by novel interactive large display environments. However, these environments also cause new challenges and result in new requirements regarding the need for interaction beyond the desktop and according redesign of analysis tools. This thesis focuses on interactive magic lenses, specialized locally applied tools that temporarily manipulate the visualization. These may include magnification of focus regions but also more graph-specific functions such as pulling in neighboring nodes or locally reducing edge clutter. Up to now, these lenses have mostly been used as single-user, single-purpose tools operated by mouse and keyboard. This dissertation presents the extension of magic lenses both in terms of function as well as interaction for large vertical displays. In particular, this thesis contributes several natural interaction designs with magic lenses for the exploration of graph data in node-link visualizations using diverse interaction modalities. This development incorporates flexible switches between lens functions, adjustment of individual lens properties and function parameters, as well as the combination of lenses. It proposes interaction techniques for fluent multi-touch manipulation of lenses, controlling lenses using mobile devices in front of large displays, and a novel concept of body-controlled magic lenses. Functional extensions in addition to these interaction techniques convert the lenses to user-configurable, personal territories with use of alternative interaction styles. To create the foundation for this extension, the dissertation incorporates a comprehensive design space of magic lenses, their function, parameters, and interactions. Additionally, it provides a discussion on increased embodiment in tool and controller design, contributing insights into user position and movement in front of large vertical displays as a result of empirical investigations and evaluations.Informationsvisualisierung ist ein wichtiges Forschungsfeld, das das Analysieren von Daten unterstützt. Graph-Visualisierungen sind dabei eine spezielle Variante der Datenrepräsentation, deren Nutzen in vielerlei Anwendungsfällen zum Einsatz kommt, u.a. in der Biologie, Softwareentwicklung und Finanzwirtschaft. Diese Datendarstellungen profitieren besonders von großen Displays in neuen Displayumgebungen. Jedoch bringen diese Umgebungen auch neue Herausforderungen mit sich und stellen Anforderungen an Nutzerschnittstellen jenseits der traditionellen Ansätze, die dadurch auch Anpassungen von Analysewerkzeugen erfordern. Diese Dissertation befasst sich mit interaktiven „Magischen Linsen“, spezielle lokal-angewandte Werkzeuge, die temporär die Visualisierung zur Analyse manipulieren. Dabei existieren zum Beispiel Vergrößerungslinsen, aber auch Graph-spezifische Manipulationen, wie das Anziehen von Nachbarknoten oder das Reduzieren von Kantenüberlappungen im lokalen Bereich. Bisher wurden diese Linsen vor allem als Werkzeug für einzelne Nutzer mit sehr spezialisiertem Effekt eingesetzt und per Maus und Tastatur bedient. Die vorliegende Doktorarbeit präsentiert die Erweiterung dieser magischen Linsen, sowohl in Bezug auf die Funktionalität als auch für die Interaktion an großen, vertikalen Displays. Insbesondere trägt diese Dissertation dazu bei, die Exploration von Graphen mit magischen Linsen durch natürliche Interaktion mit unterschiedlichen Modalitäten zu unterstützen. Dabei werden flexible Änderungen der Linsenfunktion, Anpassungen von individuellen Linseneigenschaften und Funktionsparametern, sowie die Kombination unterschiedlicher Linsen ermöglicht. Es werden Interaktionstechniken für die natürliche Manipulation der Linsen durch Multitouch-Interaktion, sowie das Kontrollieren von Linsen durch Mobilgeräte vor einer Displaywand vorgestellt. Außerdem wurde ein neuartiges Konzept körpergesteuerter magischer Linsen entwickelt. Funktionale Erweiterungen in Kombination mit diesen Interaktionskonzepten machen die Linse zu einem vom Nutzer einstellbaren, persönlichen Arbeitsbereich, der zudem alternative Interaktionsstile erlaubt. Als Grundlage für diese Erweiterungen stellt die Dissertation eine umfangreiche analytische Kategorisierung bisheriger Forschungsarbeiten zu magischen Linsen vor, in der Funktionen, Parameter und Interaktion mit Linsen eingeordnet werden. Zusätzlich macht die Arbeit Vor- und Nachteile körpernaher Interaktion für Werkzeuge bzw. ihre Steuerung zum Thema und diskutiert dabei Nutzerposition und -bewegung an großen Displaywänden belegt durch empirische Nutzerstudien

Panel: The Architectural Touch: Gestural Approaches to Library Search

This panel centers on the LibViz project—a touch and gesture-based interface that allows users to navigate through library collections using visual queries—and the issues surrounding such efforts. The LibViz project, for which we have done initial research and constructed a prototype, aims to increase the discoverability of library materials, particularly those of non-textual objects, which are difficult to access via traditional search and which do not circulate. Many collections are currently preparing large scale digitizing of threedimensional objects and it is imperative to develop appropriate methods to work with this new kind of data. The established methods only do a poor job at providing access to 3D-object data. Based in theories of “grounded cognition,” the LibViz interface will be optimized for use on personal mobile devices, but it can also be used on large format touch screens equipped with depth cameras that track user gestures. In other words, the interactive flow of LibViz allows both gestural interaction and touch commands, effectively extending the sensory modalities involved in the cognitive processing of the search results. By engaging a fuller range of human cognitive capabilities, the LibViz interface also hopes to help transform search. The amount of data generated in the digital era is growing exponentially, and so we must find novel ways of analyzing and interpreting these vast data archives. Moreover, the ways in which information is categorized and databases are created are value laden. As such, the processes by which these structures are established should be more transparent than conventional systems currently allow. The project turns library search into a powerful and pleasurable experience, stimulating engagement with the collections and the library itself

Managing a document-based information space

We present a novel user interface in the form of a complementary virtual environment for managing personal document archives, i.e., for document filing and retrieval. Our implementation of a spatial medium for document interaction, exploratory search and active navigation plays to the strengths of human visual information processing and further stimulates it. Our system provides a high degree of immersion so that the user readily forgets the artificiality of our environment. Three well-integrated features support this immersion: first, we enable users to interact more naturally through gestures and postures (the system can be taught custom ones); second, we exploit 3D display technology; and third, we allow users to manage arrangements (manually edited structures, as well as computer-generated semantic structures). Our ongoing evaluation indicates that even non-expert users can efficiently work with the information in a document collection and that the process can actually be enjoyable. ACM Classification: H5.2 [Information interfaces and presentation]

Designing Digital Art and Communication Tools Inspired by Traditional Craft

Ph.DDOCTOR OF PHILOSOPH

Touching 3D data:interactive visualization of cosmological simulations

Visualisatie richt zich op het aanvullen van de zintuigen en het voorstellingsvermogen van wetenschappers zodat ze hun gegevens beter kunnen begrijpen. Dit is een interactief en iteratief proces waar informatierepresentatie, interactieve verkenning en het nemen van beslissingen een grote rol spelen. Het doel is om door middel van dit iteratieve proces inzicht te verkrijgen in het probleem en de onderliggende gegevens totdat er voldoende begrip is. Bij deze visuele verkenning is een hoge mate van interactiviteit essentieel voor het op eem efficiënte manier behalen van dit doel omdat het de gebruiker dan de mogelijkheid geeft om nieuwe ideëen uit te proberen, terugkoppeling te verkrijgen en de verkenning op basis daarvan bij te sturen. In dit proefschrift rapporteren we over ons onderzoek naar de uitdagingen voor natuurlijke interactie en de verkenning van gegevens die gerepresenteerd worden in drie dimensies. In dit proefschrift hebben we de visualisatie van astronomische gegevens als centraal voorbeeld genomen van een toepassingsdomein waarin de nadruk ligt op op drie-dimensionale puntwolkgegevens van numerieke simulaties, zoals simulaties van galactische dynamica of hoog-dimensionale informatie uit deeltjessystemen. We introduceren twee intuïtieve en efficiënte interactie-technieken voor de verkenning van gegevens in drie dimensies. De eerste techniek helpt gebruikers te navigeren in drie dimensies en de tweede techniek geeft gebruikers de mogelijkheid om eenvoudig een subset van deeltjes te selecteren. Daarnaast integreren we deze twee technieken in een visueel-analytische computerapplicatie om zo wetenschappers te helpen nuttige informatie te extraheren en inzicht te verkrijgen