Grand Challenges in Immersive Analytics

The definitive version will be published in CHI 2021, May 8–13, 2021, Yokohama, JapanInternational audienceImmersive Analytics is a quickly evolving field that unites several areas such as visualisation, immersive environments, and humancomputer interaction to support human data analysis with emerging technologies. This research has thrived over the past years with multiple workshops, seminars, and a growing body of publications, spanning several conferences. Given the rapid advancement of interaction technologies and novel application domains, this paper aims toward a broader research agenda to enable widespread adoption. We present 17 key research challenges developed over multiple sessions by a diverse group of 24 international experts, initiated from a virtual scientific workshop at ACM CHI 2020. These challenges aim to coordinate future work by providing a systematic roadmap of current directions and impending hurdles to facilitate productive and effective applications for Immersive Analytics

Eyewear Computing \u2013 Augmenting the Human with Head-Mounted Wearable Assistants

The seminar was composed of workshops and tutorials on head-mounted eye tracking, egocentric vision, optics, and head-mounted displays. The seminar welcomed 30 academic and industry researchers from Europe, the US, and Asia with a diverse background, including wearable and ubiquitous computing, computer vision, developmental psychology, optics, and human-computer interaction. In contrast to several previous Dagstuhl seminars, we used an ignite talk format to reduce the time of talks to one half-day and to leave the rest of the week for hands-on sessions, group work, general discussions, and socialising. The key results of this seminar are 1) the identification of key research challenges and summaries of breakout groups on multimodal eyewear computing, egocentric vision, security and privacy issues, skill augmentation and task guidance, eyewear computing for gaming, as well as prototyping of VR applications, 2) a list of datasets and research tools for eyewear computing, 3) three small-scale datasets recorded during the seminar, 4) an article in ACM Interactions entitled \u201cEyewear Computers for Human-Computer Interaction\u201d, as well as 5) two follow-up workshops on \u201cEgocentric Perception, Interaction, and Computing\u201d at the European Conference on Computer Vision (ECCV) as well as \u201cEyewear Computing\u201d at the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp)

Universal Gesture Tracking Framework in OpenISS and ROS and its Applications

In this research work, we present a common and extensible framework that abstracts different vision-based gesture recognition middleware and provides uniform gesture recognition data obtained from those via its simpler API to enable hand gesture interaction in different kinds of applications. We demonstrate various aspects of our framework via instrumentation and enable gesture interaction for our two specific yet different needs. Firstly, we alleviate limited gesture tracking functionality in ISSv2 aka. (Illimitable Space System v2), an interactive and configurable artists' toolbox that is used to create music visualizations, visual effects and interactive documentary film based on the inputs from users such as gestures, voice, motion, etc. Secondly, we provide a proof-of-concept solution to enhance and demonstrate limited language usability of the FORENSIC LUCID language's composition and compiler interactivity by enabling a forensic investigator to create partial FORENSIC LUCID encoded programs which require manipulation of preloaded digital evidence objects in a 3D warehouse-like application (DigiEVISS) via hand gesture interaction. We also leverage Robot Operating System (ROS), an open source set of tools and libraries for its communication middleware to broadcast our framework data over the network. We provide this framework as a specialization of the OpenISS core framework and evaluate our framework on various aspects. We employ metrics such as effective frame rate and delay to evaluate our exemplified scenarios that represent our needs

Interaction for Immersive Analytics

International audienceIn this chapter, we briefly review the development of natural user interfaces and discuss their role in providing human-computer interaction that is immersive in various ways. Then we examine some opportunities for how these technologies might be used to better support data analysis tasks. Specifically, we review and suggest some interaction design guidelines for immersive analytics. We also review some hardware setups for data visualization that are already archetypal. Finally, we look at some emerging system designs that suggest future directions

Communication of Digital Material Appearance Based on Human Perception

Im alltägliche Leben begegnen wir digitalen Materialien in einer Vielzahl von Situationen wie beispielsweise bei Computerspielen, Filmen, Reklamewänden in zB U-Bahn Stationen oder beim Online-Kauf von Kleidungen. Während einige dieser Materialien durch digitale Modelle repräsentiert werden, welche das Aussehen einer bestimmten Oberfläche in Abhängigkeit des Materials der Fläche sowie den Beleuchtungsbedingungen beschreiben, basieren andere digitale Darstellungen auf der simplen Verwendung von Fotos der realen Materialien, was zB bei Online-Shopping häufig verwendet wird. Die Verwendung von computer-generierten Materialien ist im Vergleich zu einzelnen Fotos besonders vorteilhaft, da diese realistische Erfahrungen im Rahmen von virtuellen Szenarien, kooperativem Produkt-Design, Marketing während der prototypischen Entwicklungsphase oder der Ausstellung von Möbeln oder Accesoires in spezifischen Umgebungen erlauben. Während mittels aktueller Digitalisierungsmethoden bereits eine beeindruckende Reproduktionsqualität erzielt wird, wird eine hochpräzise photorealistische digitale Reproduktion von Materialien für die große Vielfalt von Materialtypen nicht erreicht. Daher verwenden viele Materialkataloge immer noch Fotos oder sogar physikalische Materialproben um ihre Kollektionen zu repräsentieren. Ein wichtiger Grund für diese Lücke in der Genauigkeit des Aussehens von digitalen zu echten Materialien liegt darin, dass die Zusammenhänge zwischen physikalischen Materialeigenschaften und der vom Menschen wahrgenommenen visuellen Qualität noch weitgehend unbekannt sind. Die im Rahmen dieser Arbeit durchgeführten Untersuchungen adressieren diesen Aspekt. Zu diesem Zweck werden etablierte digitalie Materialmodellen bezüglich ihrer Eignung zur Kommunikation von physikalischen und sujektiven Materialeigenschaften untersucht, wobei Beobachtungen darauf hinweisen, dass ein Teil der fühlbaren/haptischen Informationen wie z.B. Materialstärke oder Härtegrad aufgrund der dem Modell anhaftenden geometrische Abstraktion verloren gehen. Folglich wird im Rahmen der Arbeit das Zusammenspiel der verschiedenen Sinneswahrnehmungen (mit Fokus auf die visuellen und akustischen Modalitäten) untersucht um festzustellen, welche Informationen während des Digitalisierungsprozesses verloren gehen. Es zeigt sich, dass insbesondere akustische Informationen in Kombination mit der visuellen Wahrnehmung die Einschätzung fühlbarer Materialeigenschaften erleichtert. Eines der Defizite bei der Analyse des Aussehens von Materialien ist der Mangel bezüglich sich an der Wahnehmung richtenden Metriken die eine Beantwortung von Fragen wie z.B. "Sind die Materialien A und B sich ähnlicher als die Materialien C und D?" erlauben, wie sie in vielen Anwendungen der Computergrafik auftreten. Daher widmen sich die im Rahmen dieser Arbeit durchgeführten Studien auch dem Vergleich von unterschiedlichen Materialrepräsentationen im Hinblick auf. Zu diesem Zweck wird eine Methodik zur Berechnung der wahrgenommenen paarweisen Ähnlichkeit von Material-Texturen eingeführt, welche auf der Verwendung von Textursyntheseverfahren beruht und sich an der Idee/dem Begriff der geradenoch-wahrnehmbaren Unterschiede orientiert. Der vorgeschlagene Ansatz erlaubt das Überwinden einiger Probleme zuvor veröffentlichter Methoden zur Bestimmung der Änhlichkeit von Texturen und führt zu sinnvollen/plausiblen Distanzen von Materialprobem. Zusammenfassend führen die im Rahmen dieser Dissertation dargestellten Inhalte/Verfahren zu einem tieferen Verständnis bezüglich der menschlichen Wahnehmung von digitalen bzw. realen Materialien über unterschiedliche Sinne, einem besseren Verständnis bzgl. der Bewertung der Ähnlichkeit von Texturen durch die Entwicklung einer neuen perzeptuellen Metrik und liefern grundlegende Einsichten für zukünftige Untersuchungen im Bereich der Perzeption von digitalen Materialien.In daily life, we encounter digital materials and interact with them in numerous situations, for instance when we play computer games, watch a movie, see billboard in the metro station or buy new clothes online. While some of these virtual materials are given by computational models that describe the appearance of a particular surface based on its material and the illumination conditions, some others are presented as simple digital photographs of real materials, as is usually the case for material samples from online retailing stores. The utilization of computer-generated materials entails significant advantages over plain images as they allow realistic experiences in virtual scenarios, cooperative product design, advertising in prototype phase or exhibition of furniture and wearables in specific environments. However, even though exceptional material reproduction quality has been achieved in the domain of computer graphics, current technology is still far away from highly accurate photo-realistic virtual material reproductions for the wide range of existing categories and, for this reason, many material catalogs still use pictures or even physical material samples to illustrate their collections. An important reason for this gap between digital and real material appearance is that the connections between physical material characteristics and the visual quality perceived by humans are far from well-understood. Our investigations intend to shed some light in this direction. Concretely, we explore the ability of state-of-the-art digital material models in communicating physical and subjective material qualities, observing that part of the tactile/haptic information (eg thickness, hardness) is missing due to the geometric abstractions intrinsic to the model. Consequently, in order to account for the information deteriorated during the digitization process, we investigate the interplay between different sensing modalities (vision and hearing) and discover that particular sound cues, in combination with visual information, facilitate the estimation of such tactile material qualities. One of the shortcomings when studying material appearance is the lack of perceptually-derived metrics able to answer questions like "are materials A and B more similar than C and D?", which arise in many computer graphics applications. In the absence of such metrics, our studies compare different appearance models in terms of how capable are they to depict/transmit a collection of meaningful perceptual qualities. To address this problem, we introduce a methodology to compute the perceived pairwise similarity between textures from material samples that makes use of patch-based texture synthesis algorithms and is inspired on the notion of Just-Noticeable Differences. Our technique is able to overcome some of the issues posed by previous texture similarity collection methods and produces meaningful distances between samples. In summary, with the contents presented in this thesis we are able to delve deeply in how humans perceive digital and real materials through different senses, acquire a better understanding of texture similarity by developing a perceptually-based metric and provide a groundwork for further investigations in the perception of digital materials

Applied Cognitive Sciences

Cognitive science is an interdisciplinary field in the study of the mind and intelligence. The term cognition refers to a variety of mental processes, including perception, problem solving, learning, decision making, language use, and emotional experience. The basis of the cognitive sciences is the contribution of philosophy and computing to the study of cognition. Computing is very important in the study of cognition because computer-aided research helps to develop mental processes, and computers are used to test scientific hypotheses about mental organization and functioning. This book provides a platform for reviewing these disciplines and presenting cognitive research as a separate discipline

Spatial Interaction for Immersive Mixed-Reality Visualizations

Growing amounts of data, both in personal and professional settings, have caused an increased interest in data visualization and visual analytics. Especially for inherently three-dimensional data, immersive technologies such as virtual and augmented reality and advanced, natural interaction techniques have been shown to facilitate data analysis. Furthermore, in such use cases, the physical environment often plays an important role, both by directly influencing the data and by serving as context for the analysis. Therefore, there has been a trend to bring data visualization into new, immersive environments and to make use of the physical surroundings, leading to a surge in mixed-reality visualization research. One of the resulting challenges, however, is the design of user interaction for these often complex systems. In my thesis, I address this challenge by investigating interaction for immersive mixed-reality visualizations regarding three core research questions: 1) What are promising types of immersive mixed-reality visualizations, and how can advanced interaction concepts be applied to them? 2) How does spatial interaction benefit these visualizations and how should such interactions be designed? 3) How can spatial interaction in these immersive environments be analyzed and evaluated? To address the first question, I examine how various visualizations such as 3D node-link diagrams and volume visualizations can be adapted for immersive mixed-reality settings and how they stand to benefit from advanced interaction concepts. For the second question, I study how spatial interaction in particular can help to explore data in mixed reality. There, I look into spatial device interaction in comparison to touch input, the use of additional mobile devices as input controllers, and the potential of transparent interaction panels. Finally, to address the third question, I present my research on how user interaction in immersive mixed-reality environments can be analyzed directly in the original, real-world locations, and how this can provide new insights. Overall, with my research, I contribute interaction and visualization concepts, software prototypes, and findings from several user studies on how spatial interaction techniques can support the exploration of immersive mixed-reality visualizations.Zunehmende Datenmengen, sowohl im privaten als auch im beruflichen Umfeld, führen zu einem zunehmenden Interesse an Datenvisualisierung und visueller Analyse. Insbesondere bei inhärent dreidimensionalen Daten haben sich immersive Technologien wie Virtual und Augmented Reality sowie moderne, natürliche Interaktionstechniken als hilfreich für die Datenanalyse erwiesen. Darüber hinaus spielt in solchen Anwendungsfällen die physische Umgebung oft eine wichtige Rolle, da sie sowohl die Daten direkt beeinflusst als auch als Kontext für die Analyse dient. Daher gibt es einen Trend, die Datenvisualisierung in neue, immersive Umgebungen zu bringen und die physische Umgebung zu nutzen, was zu einem Anstieg der Forschung im Bereich Mixed-Reality-Visualisierung geführt hat. Eine der daraus resultierenden Herausforderungen ist jedoch die Gestaltung der Benutzerinteraktion für diese oft komplexen Systeme. In meiner Dissertation beschäftige ich mich mit dieser Herausforderung, indem ich die Interaktion für immersive Mixed-Reality-Visualisierungen im Hinblick auf drei zentrale Forschungsfragen untersuche: 1) Was sind vielversprechende Arten von immersiven Mixed-Reality-Visualisierungen, und wie können fortschrittliche Interaktionskonzepte auf sie angewendet werden? 2) Wie profitieren diese Visualisierungen von räumlicher Interaktion und wie sollten solche Interaktionen gestaltet werden? 3) Wie kann räumliche Interaktion in diesen immersiven Umgebungen analysiert und ausgewertet werden? Um die erste Frage zu beantworten, untersuche ich, wie verschiedene Visualisierungen wie 3D-Node-Link-Diagramme oder Volumenvisualisierungen für immersive Mixed-Reality-Umgebungen angepasst werden können und wie sie von fortgeschrittenen Interaktionskonzepten profitieren. Für die zweite Frage untersuche ich, wie insbesondere die räumliche Interaktion bei der Exploration von Daten in Mixed Reality helfen kann. Dabei betrachte ich die Interaktion mit räumlichen Geräten im Vergleich zur Touch-Eingabe, die Verwendung zusätzlicher mobiler Geräte als Controller und das Potenzial transparenter Interaktionspanels. Um die dritte Frage zu beantworten, stelle ich schließlich meine Forschung darüber vor, wie Benutzerinteraktion in immersiver Mixed-Reality direkt in der realen Umgebung analysiert werden kann und wie dies neue Erkenntnisse liefern kann. Insgesamt trage ich mit meiner Forschung durch Interaktions- und Visualisierungskonzepte, Software-Prototypen und Ergebnisse aus mehreren Nutzerstudien zu der Frage bei, wie räumliche Interaktionstechniken die Erkundung von immersiven Mixed-Reality-Visualisierungen unterstützen können

Knowledge Communities in Online Education and (Visual) Knowledge Management: 19. Workshop GeNeMe‘16 as part of IFKAD 2016: Proceedings of 19th Conference GeNeMe

Communities in New Media started in 1998 as a workshop series at TU Dresden, and since then has annually dealt with online communities at the interface between several disciplines such as education and economics, computer science, social and communication sciences, and more. (See Köhler, Kahnwald & Schoop, 2015). The workshop is traditionally a forum for interdisciplinary dialogue between science and business and serves to share experiences and knowledge among participants from different disciplines, organisations, and institutions. In addition to the core themes of knowledge management and communities (in the chapters of the same name), the main focus of the conference is also on the support of knowledge and learning processes in the field of (media-assisted) higher education. This is complemented by an informational perspective when it comes to more functional and methodological approaches - use cases, workflows, and automation in knowledge management. In addition, systems and approaches for feedback, exchange, and ideas are presented. With the focus of knowledge media design and visual research as well as creative processes, this time there is also a highlight on visual aspects of knowledge management and mediation. For IFKAD 2016, three GeNeMe tracks were accepted which focus on the interface of knowledge communities and knowledge management as well as knowledge media design in science, business, or education. In this conference volume you will find detailed information about these three tracks: -- Knowledge Communities I: Knowledge Management -- Knowledge Communities II: Online Education -- Visual Knowledge Management [From the Preface.]:Preface IX Vorwort XIII Knowledge Communities I: Knowledge Management 1 Process Learning Environments 1 Two Steps to IT Transparency: A Practitioner’s Approach for a Knowledge Based Analysis of Existing IT Landscapes in SME 13 Social Media and Sustainable Communication. Rethinking the Role of Research and Innovation Networks 26 Consolidating eLearning in a Higher Education Institution: An Organisational Issue integrating Didactics, Technology, and People by the Means of an eLearning Strategy 39 How to treat the troll? An empirical analysis of counterproductive online behavior, personality traits and organizational behavior 51 Knowledge Communities II: Online Education 64 Sifa-Portfolio – a Continuing Education Concept for Specialists on Industrial Safety Combining Formal and Informal Learning 64 Analysing eCollaboration: Prioritisation of Monitoring Criteria for Learning Analytics in the Virtual Classroom 78 Gamifying Higher Education. Beyond Badges, Points and Leaderboards 93 Virtual International Learning Experience in Formal Higher Education – A Case Study from Jordan 105 Migration to the Flipped Classroom – Applying a Scalable Flipped Classroom Arrangement 117 MOOC@TU9 – Common MOOC Strategy of the Alliance of Nine Leading German Institutes of Technology 131 A Survey on Knowledge Management in Universities in the QS Rankings: E-learning and MOOCs 144 Visual Knowledge Media 157 Generating implications for design in practice: How different stimuli are retrieved and transformed to generate ideas 157 Behind the data – preservation of the knowledge in CH Visualisations 170 Building a Wiki resource on digital 3D reconstruction related knowledge assets 184 Visual media as a tool to acquire soft skills — cross-disciplinary teaching-learning project SUFUvet 196 Graphing Meeting Records - An Approach to Visualize Information in a Multi Meeting Context 209 HistStadt4D – A four dimensional access to history 221 Ideagrams: A digital tool for observing ideation processes 234 Adress- und Autorenverzeichnis 251Gemeinschaften in Neuen Medien hat 1998 als Workshop-Reihe an der TU Dresden begonnen und seither jährlich das Thema Online-Communities an der Schnittstelle mehrerer Disziplinen wie Informatik, Bildungs- und Wirtschaftswissenschaften, Informatik sowie Sozial-und Kommunikationswissenschaft u.a.m. thematisiert (vgl. Köhler, Kahnwald & Schoop, 2015). Der Workshop ist traditionell ein Forum für den interdisziplinären Dialog zwischen Wissenschaft und Wirtschaft und dient dazu, Erfahrungen und Wissen unter den Teilnehmern aus verschiedenen Disziplinen, Organisationen und Institutionen zu teilen. Die inhaltlichen Schwerpunkte der Konferenz widmen sich neben den Kernthemen Wissensmanagement und Communities (in den gleichnamigen Kapiteln) auch der Unterstützung von Wissens- und Lernprozessen im Bereich der (mediengestützten) Hochschullehre. Ergänzt wird diese eher organisationswissenschaftliche durch eine informatorische Perspektive, wenn es um stärker funktionale bzw. auch methodische Ansätze geht – Use Cases, Workflows und Automatisierung im Wissensmanagement. Darüber hinaus werden Systeme und Ansätze für Feedback, Austausch und Ideenfindung vorgestellt. Mit den Schwerpunkten der Wissensmediengestaltung und visuellen Forschungs- sowie Kreativprozessen wird diesmal auch ein Schlaglicht auf visuelle Aspekte von Wissensmanagement und -vermittlung geworfen. Für die IFKAD 2016 wurden drei GeNeMe-Tracks angenommen, die sich auf das Interface von Wissensgemeinschaften und Wissensmanagement sowie die Wissensmediengestaltung in Wissenschaft, Wirtschaft oder Bildung konzentrieren. Im vorliegenden Tagungsband finden Sie detaillierte Informationen zu diesen drei Tracks: -- Knowledge Communities I: Knowledge Management -- Knowledge Communities II: Online Education -- Visual Knowledge Management [Aus dem Vorwort.]:Preface IX Vorwort XIII Knowledge Communities I: Knowledge Management 1 Process Learning Environments 1 Two Steps to IT Transparency: A Practitioner’s Approach for a Knowledge Based Analysis of Existing IT Landscapes in SME 13 Social Media and Sustainable Communication. Rethinking the Role of Research and Innovation Networks 26 Consolidating eLearning in a Higher Education Institution: An Organisational Issue integrating Didactics, Technology, and People by the Means of an eLearning Strategy 39 How to treat the troll? An empirical analysis of counterproductive online behavior, personality traits and organizational behavior 51 Knowledge Communities II: Online Education 64 Sifa-Portfolio – a Continuing Education Concept for Specialists on Industrial Safety Combining Formal and Informal Learning 64 Analysing eCollaboration: Prioritisation of Monitoring Criteria for Learning Analytics in the Virtual Classroom 78 Gamifying Higher Education. Beyond Badges, Points and Leaderboards 93 Virtual International Learning Experience in Formal Higher Education – A Case Study from Jordan 105 Migration to the Flipped Classroom – Applying a Scalable Flipped Classroom Arrangement 117 MOOC@TU9 – Common MOOC Strategy of the Alliance of Nine Leading German Institutes of Technology 131 A Survey on Knowledge Management in Universities in the QS Rankings: E-learning and MOOCs 144 Visual Knowledge Media 157 Generating implications for design in practice: How different stimuli are retrieved and transformed to generate ideas 157 Behind the data – preservation of the knowledge in CH Visualisations 170 Building a Wiki resource on digital 3D reconstruction related knowledge assets 184 Visual media as a tool to acquire soft skills — cross-disciplinary teaching-learning project SUFUvet 196 Graphing Meeting Records - An Approach to Visualize Information in a Multi Meeting Context 209 HistStadt4D – A four dimensional access to history 221 Ideagrams: A digital tool for observing ideation processes 234 Adress- und Autorenverzeichnis 25