Spatial Interaction for Immersive Mixed-Reality Visualizations

Growing amounts of data, both in personal and professional settings, have caused an increased interest in data visualization and visual analytics. Especially for inherently three-dimensional data, immersive technologies such as virtual and augmented reality and advanced, natural interaction techniques have been shown to facilitate data analysis. Furthermore, in such use cases, the physical environment often plays an important role, both by directly influencing the data and by serving as context for the analysis. Therefore, there has been a trend to bring data visualization into new, immersive environments and to make use of the physical surroundings, leading to a surge in mixed-reality visualization research. One of the resulting challenges, however, is the design of user interaction for these often complex systems. In my thesis, I address this challenge by investigating interaction for immersive mixed-reality visualizations regarding three core research questions: 1) What are promising types of immersive mixed-reality visualizations, and how can advanced interaction concepts be applied to them? 2) How does spatial interaction benefit these visualizations and how should such interactions be designed? 3) How can spatial interaction in these immersive environments be analyzed and evaluated? To address the first question, I examine how various visualizations such as 3D node-link diagrams and volume visualizations can be adapted for immersive mixed-reality settings and how they stand to benefit from advanced interaction concepts. For the second question, I study how spatial interaction in particular can help to explore data in mixed reality. There, I look into spatial device interaction in comparison to touch input, the use of additional mobile devices as input controllers, and the potential of transparent interaction panels. Finally, to address the third question, I present my research on how user interaction in immersive mixed-reality environments can be analyzed directly in the original, real-world locations, and how this can provide new insights. Overall, with my research, I contribute interaction and visualization concepts, software prototypes, and findings from several user studies on how spatial interaction techniques can support the exploration of immersive mixed-reality visualizations.Zunehmende Datenmengen, sowohl im privaten als auch im beruflichen Umfeld, führen zu einem zunehmenden Interesse an Datenvisualisierung und visueller Analyse. Insbesondere bei inhärent dreidimensionalen Daten haben sich immersive Technologien wie Virtual und Augmented Reality sowie moderne, natürliche Interaktionstechniken als hilfreich für die Datenanalyse erwiesen. Darüber hinaus spielt in solchen Anwendungsfällen die physische Umgebung oft eine wichtige Rolle, da sie sowohl die Daten direkt beeinflusst als auch als Kontext für die Analyse dient. Daher gibt es einen Trend, die Datenvisualisierung in neue, immersive Umgebungen zu bringen und die physische Umgebung zu nutzen, was zu einem Anstieg der Forschung im Bereich Mixed-Reality-Visualisierung geführt hat. Eine der daraus resultierenden Herausforderungen ist jedoch die Gestaltung der Benutzerinteraktion für diese oft komplexen Systeme. In meiner Dissertation beschäftige ich mich mit dieser Herausforderung, indem ich die Interaktion für immersive Mixed-Reality-Visualisierungen im Hinblick auf drei zentrale Forschungsfragen untersuche: 1) Was sind vielversprechende Arten von immersiven Mixed-Reality-Visualisierungen, und wie können fortschrittliche Interaktionskonzepte auf sie angewendet werden? 2) Wie profitieren diese Visualisierungen von räumlicher Interaktion und wie sollten solche Interaktionen gestaltet werden? 3) Wie kann räumliche Interaktion in diesen immersiven Umgebungen analysiert und ausgewertet werden? Um die erste Frage zu beantworten, untersuche ich, wie verschiedene Visualisierungen wie 3D-Node-Link-Diagramme oder Volumenvisualisierungen für immersive Mixed-Reality-Umgebungen angepasst werden können und wie sie von fortgeschrittenen Interaktionskonzepten profitieren. Für die zweite Frage untersuche ich, wie insbesondere die räumliche Interaktion bei der Exploration von Daten in Mixed Reality helfen kann. Dabei betrachte ich die Interaktion mit räumlichen Geräten im Vergleich zur Touch-Eingabe, die Verwendung zusätzlicher mobiler Geräte als Controller und das Potenzial transparenter Interaktionspanels. Um die dritte Frage zu beantworten, stelle ich schließlich meine Forschung darüber vor, wie Benutzerinteraktion in immersiver Mixed-Reality direkt in der realen Umgebung analysiert werden kann und wie dies neue Erkenntnisse liefern kann. Insgesamt trage ich mit meiner Forschung durch Interaktions- und Visualisierungskonzepte, Software-Prototypen und Ergebnisse aus mehreren Nutzerstudien zu der Frage bei, wie räumliche Interaktionstechniken die Erkundung von immersiven Mixed-Reality-Visualisierungen unterstützen können

Exploring Slider vs. Categorical Response Formats in Web-Based Surveys

Web-based surveys have become a common mode of data collection for researchers in many fields, but there are many methodological questions that need to be answered. This article examines one such question—do the use of sliders to express numerical amounts and the use of the more traditional radio-button scales give the same, or different, measurements? First, we review the central debates surrounding the use of slider scales, including advantages and disadvantages. Second, we report findings from a controlled simple randomized design field experiment using a sample of business managers in Italy to compare the two response formats. Measures of topic sensitivity, topic interest, and likelihood of participation were obtained. No statistically significant differences were found between the response formats. The article concludes with suggestions for researchers who wish to use slider scales as a measurement device

Gesture, Rhetoric, and Digital Storytelling

This proposal seeks to develop new theory and technology for digital interactive narratives based on cross-cultural storytelling traditions that utilize non-verbal communication to convey meaning. Digital interactive narratives are of increasing cultural relevance including: new genres of electronic literature, virtual museums and memorials, educational computer games, and computer-assisted generation of scholarly documents. We propose a new form of storytelling that changes emotional tone, theme, perspective, other subjective elements based upon gestural input via touch-screen and motion-sensitive devices. This proposal builds upon the Principal Investigator's significant body of work in developing humanistically grounded interactive narrative and artificial intelligence. The Principal Investigator has found that literary, cultural, and performance studies theories of orature (oral literature) and accounts of non-verbal communication from animation studies provide models that can be adapted to interactive narratives

Phrasing Bimanual Interaction for Visual Design

Architects and other visual thinkers create external representations of their ideas to support early-stage design. They compose visual imagery with sketching to form abstract diagrams as representations. When working with digital media, they apply various visual operations to transform representations, often engaging in complex sequences. This research investigates how to build interactive capabilities to support designers in putting together, that is phrasing, sequences of operations using both hands. In particular, we examine how phrasing interactions with pen and multi-touch input can support modal switching among different visual operations that in many commercial design tools require using menus and tool palettes—techniques originally designed for the mouse, not pen and touch. We develop an interactive bimanual pen+touch diagramming environment and study its use in landscape architecture design studio education. We observe interesting forms of interaction that emerge, and how our bimanual interaction techniques support visual design processes. Based on the needs of architects, we develop LayerFish, a new bimanual technique for layering overlapping content. We conduct a controlled experiment to evaluate its efficacy. We explore the use of wearables to identify which user, and distinguish what hand, is touching to support phrasing together direct-touch interactions on large displays. From design and development of the environment and both field and controlled studies, we derive a set methods, based upon human bimanual specialization theory, for phrasing modal operations through bimanual interactions without menus or tool palettes

Creating mobile gesture-based interaction design patterns for older adults : a study of tap and swipe gestures with portuguese seniors

Tese de mestrado. Multimédia. Faculdade de Engenharia. Universidade do Porto. 201

Exploring Tablet Interfaces for Product Appearance Authoring in Spatial Augmented Reality

Users acceptance of innovative product appearance authoring tools based on Spatial Augmented Reality (SAR) is still limited due to their perception of a high technology complexity and a low performance/functionality of the current interaction systems. The integration of SAR technologies in professional design activities is still marginal, though many studies in this field have already proved their potential as supporting tools. To overcome this barrier, efficient means for interacting with the digital images projected onto the surfaces of real objects are essential. The aim of the current study is to respond to this demand by proposing and validating three UI configurations displayed by an unique and portable device embedded with a touch screen. These interface layouts, designed to cooperate with the output of the SAR system and to not affect the well-known benefits of its augmented environment, provide different types of visual feedback to the user by duplicating, extending or hiding the information already displayed by the projected mock-up. The experimental study reported here, performed with a panel of 41 subjects, revealed that accuracy, efficiency and perceived usability of the proposed solutions are comparable with each other and in comparison to standard desktop setups commonly used for design activities. According to these findings, the research simultaneously demonstrates (i) the high performances achieved by the touch device when coupled with a SAR system during the execution of authoring tasks, (ii) the capability of the projected mock-up to behave as an actual three-dimensional display for the real time rendering of the product appearance and (iii) the possibility to freely select - according to the users preference, the design task or the type of product - one of the three UI configurations without affecting the quality of the result.</p