Experimenting and improving perception of 3D rotation-based transitions between 2D visualizations

Part 1: Long and Short PapersInternational audienceExploring a multidimensional dataset with visualization requires to transition between points of view. In order to enable users to understand transitions, visualization can employ progressive 3D rotations. However, existing implementations of progressive 3D rotation exhibit some perception problems with visualization of cluttered scene. In this paper, we present a first experiment showing how existing 3D rotation is effective for tracking marks, and that cluttered scenes actually hinder perception of rotation. Then, we propose to set the axis of rotation on the graphical marks of interest, and ran a second experiment showing that focus-centered rotation improves perception of relative arrangement

A Visualization Framework for Designing Process Mining Diagrams

Sündmuslogid sisaldavad väärtuslikku informatsiooni äriprotsesside seisundi kohta. Informatsioonile ligi pääsemiseks peab andmestiku viima arusaadavale kujule. Protsissikaeve tööriistad kasutavad erinevaid diagramme, mis toetavad sündmuslogide visuaalset uurimist. Nende diagrammide kujundamine ei ole lihtne ülesanne, sest tihti ei tea arendaja ega kasutaja, kus huvipakkuv informatsioon võib asuda. Seepärast peavad diagrammid olema paindlikud, kuid samas lihtsad ja intuitiivsed, et nii analüütikud kui ka mitteasjatundjad saaksid tööriista kasutada. Antud töö uurib olemasolevate protsessikaeve diagrammide kujundusi ja kuidas need kujundused on autorite poolt põhjendatud. Töös tutvustatakse ka raamistikku, mis on välja töötatud selleks, et lihtsustada ja täiustada protsessikaeve diagrammide kujundamist. See põhineb andmete visualiseerimise teoorial ja visualiseerimise praktikatel protsessikaeves. Raamistiku tõhusust on katsetatud juhtumuuringus.Event logs hold valuable information about the health of business processes. In order to access this information, raw data must be transformed to a comprehensible format. Process mining tools use various diagrams to support visual exploration of process logs. Designing such diagrams is not an easy task because oftentimes neither the developer nor user know where interesting or intriguing information lays. Therefore, the diagrams require thoughtful designs that on the one hand allow flexible exploration, and on the other hand, are simple and intuitive to use for analysts as well as non-experts. This work takes a look into existing solutions of process mining visualizations and the design decisions the visualizations are based on. A framework is proposed to simplify and improve the design process for process mining diagrams. It is based on data visualization theory as well as visualization practices in process mining. The effectiveness of the framework is tested in a case study

Scene creation and exploration in outdoor augmented reality

This thesis investigates Outdoor Augmented Reality (AR) especially for scene creation and exploration aspects.We decompose a scene into several components: a) Device, b) Target Object(s), c) Task, and discuss their interrelations. Based on those relations we outline use-cases and workflows. The main contribution of this thesis is providing AR oriented workflows for selected professional fields specifically for scene creation and exploration purposes, through case studies as well as analyzing the relations between AR scene components. Our contributions inlude, but not limited to: i) analysis of scene components and factoring inherintly available errors, to create a transitional hybrid tracking scheme for multiple targets, ii) a novel image-based approach that uses building block analogy for modelling and introduces volumetric and temporal labeling for annotations, iii) an evaluation of the state of the art X-Ray visualization methods as well as our proposed multi-view method. AR technology and capabilities tend to change rapidly, however we believe the relation between scene components and the practical advantages their analysis provide are valuable. Moreover, we have chosen case studies as diverse as possible in order to cover a wide range of professional field studies. We believe our research is extendible to a variety of field studies for disciplines including but not limited to: Archaeology, architecture, cultural heritage, tourism, stratigraphy, civil engineering, and urban maintenance

Visualization of state transition graphs

State transition graphs are important in computer science and engineering where they are used to analyze the behavior of computer-based systems. In such a graph nodes represent states a system can be in. Links, or directed edges, represent transitions between states. Research in visualization investigates the application of interactive computer graphics to understand large and complex data sets. Large state transition graphs fall into this category. They often contain tens of thousands of nodes, or more, and tens to hundreds of thousands of edges. Also, they describe system behavior at a low abstraction level. This hinders analysis and insight. This dissertation presents a number of techniques for the interactive visualization of state transition graphs. Much of the work takes advantage of multivariate data associated with nodes and edges. Using an experimental approach, several new methods were developed in close collaboration with a number of users. The following approaches were pursued: • Selection and projection. This technique provides the user with visual support to select a subset of node attributes. Consequently, the state transition graph is projected to 2D and visualized in a second, correlated visualization. • Attribute-based clustering. By specifying subsets of node attributes and clustering based on these, the user generates simplified abstractions of a state transition graph. Clustering generates hierarchical, relational, and metric data, which are represented in a single visualization. • User-defined diagrams. With this technique the user investigates state transition graphs with custom diagrams. Diagrams are parameterized by linking their graphical properties to the data. Diagrams are integrated in a number of correlated visualizations. • Multiple views on traces. System traces are linear paths in state transition graphs. This technique provides the user with different perspectives on traces. • Querying nodes and edges. Direct manipulation enables the user to interactively inspect and query state transition graphs. In this way relations and patterns can be investigated based on data associated with nodes and edges. This dissertation shows that interactive visualization can play a role during the analysis of state transition graphs. The ability to interrogate visual representations of such graphs allows users to enhance their knowledge of the modeled systems. It is shown how the above techniques enable users to answer questions about their data. A number of case studies, developed in collaboration with system analysts, are presented. Finally, solutions to challenges encountered during the development of the visualization techniques are discussed. Insights generic to the field of visualization are considered and directions for future work are recommended

Understanding neuroanatomy in a virtual 3D environment: creation and use of a new survey tool to evaluate the effectiveness of 3D software in neuroanatomy education for understanding superficial and deep brain structures.

Studying cross-sections is a critical approach to learning and testing knowledge in neuroanatomy and the role of 3D technologies have been gradually increasing in medical education, especially after the COVID-19 pandemic. A study was conducted in a quasi-experimental one-group pre-post interventional design in an online setting by creating and evaluating the effectiveness of a virtual lab in neuroanatomy for all neuroscience students enrolled in the Fundamentals of Neuroscience course in our department at the University of Louisville. Study modules were created using the 2D resources used in previous years and 3D web applications of Visible Body and AnatomyLearning.com software. A newly developed 13-item Reaction-Relevance-Result survey measured the effectiveness of these resources, along with Confidence in topics surveys and test results. Results of the study confirmed the advantages of using 3D software for neuroanatomy, with mostly large effect sizes for the pre-post effects. The study also sheds some light on the social need and justice regarding the utility of 3D intervention to bring equitable learning among all genders and academic levels without any effects of earlier performances. The study also uncovered some bias in student perception of the advantages of 3D software for students with any previous neuroanatomy experience. 3D software increased understanding of superficial and deep structures but was more beneficial for deeper structures, thus bridging the difficulty gap between superficial and deep structures, male students being more successful in narrowing this difficulty gap

Expressive movement generation with machine learning

Movement is an essential aspect of our lives. Not only do we move to interact with our physical environment, but we also express ourselves and communicate with others through our movements. In an increasingly computerized world where various technologies and devices surround us, our movements are essential parts of our interaction with and consumption of computational devices and artifacts. In this context, incorporating an understanding of our movements within the design of the technologies surrounding us can significantly improve our daily experiences. This need has given rise to the field of movement computing – developing computational models of movement that can perceive, manipulate, and generate movements. In this thesis, we contribute to the field of movement computing by building machine-learning-based solutions for automatic movement generation. In particular, we focus on using machine learning techniques and motion capture data to create controllable, generative movement models. We also contribute to the field by creating datasets, tools, and libraries that we have developed during our research. We start our research by reviewing the works on building automatic movement generation systems using machine learning techniques and motion capture data. Our review covers background topics such as high-level movement characterization, training data, features representation, machine learning models, and evaluation methods. Building on our literature review, we present WalkNet, an interactive agent walking movement controller based on neural networks. The expressivity of virtual, animated agents plays an essential role in their believability. Therefore, WalkNet integrates controlling the expressive qualities of movement with the goal-oriented behaviour of an animated virtual agent. It allows us to control the generation based on the valence and arousal levels of affect, the movement’s walking direction, and the mover’s movement signature in real-time. Following WalkNet, we look at controlling movement generation using more complex stimuli such as music represented by audio signals (i.e., non-symbolic music). Music-driven dance generation involves a highly non-linear mapping between temporally dense stimuli (i.e., the audio signal) and movements, which renders a more challenging modelling movement problem. To this end, we present GrooveNet, a real-time machine learning model for music-driven dance generation

3D (embodied) projection mapping and sensing bodies : a study in interactive dance performance

This dissertation identifies the synergies between physical and virtual environments when designing for immersive experiences in interactive dance performances. The integration of virtual information in physical space is transforming our interactions and experiences with the world. By using the body and creative expression as the interface between real and virtual worlds, dance performance creates a privileged framework to research and design interactive mixed reality environments and immersive augmented architectures. The research is primarily situated in the fields of visual art and interaction design. It combines performance with transdisciplinary fields and intertwines practice with theory. The theoretical and conceptual implications involved in designing and experiencing immersive hybrid environments are analyzed using the reality–virtuality continuum. These theories helped frame the ways augmented reality architectures are achieved through the integration of dance performance with digital software and reception displays. They also helped identify the main artistic affordances and restrictions in the design of augmented reality and augmented virtuality environments for live performance. These pervasive media architectures were materialized in three field experiments, the live dance performances. Each performance was created in three different stages of conception, design and production. The first stage was to “digitize” the performer’s movement and brain activity to the virtual environment and our system. This was accomplished through the use of depth sensor cameras, 3D motion capture, and brain computer interfaces. The second stage was the creation of the computational architecture and software that aggregates the connections and mapping between the physical body and the spatial dynamics of the virtual environment. This process created real-time interactions between the performer’s behavior and motion and the real-time generative computer 3D graphics. Finally, the third stage consisted of the output modality: 3D projector based augmentation techniques were adopted in order to overlay the virtual environment onto physical space. This thesis proposes and lays out theoretical, technical, and artistic frameworks between 3D digital environments and moving bodies in dance performance. By sensing the body and the brain with the 3D virtual environments, new layers of augmentation and interactions are established, and ultimately this generates mixed reality environments for embodied improvisational self-expression.Radio-Television-Fil

Active and Physics-Based Human Pose Reconstruction

Perceiving humans is an important and complex problem within computervision. Its significance is derived from its numerous applications, suchas human-robot interaction, virtual reality, markerless motion capture,and human tracking for autonomous driving. The difficulty lies in thevariability in human appearance, physique, and plausible body poses. Inreal-world scenes, this is further exacerbated by difficult lightingconditions, partial occlusions, and the depth ambiguity stemming fromthe loss of information during the 3d to 2d projection. Despite thesechallenges, significant progress has been made in recent years,primarily due to the expressive power of deep neural networks trained onlarge datasets. However, creating large-scale datasets with 3dannotations is expensive, and capturing the vast diversity of the realworld is demanding. Traditionally, 3d ground truth is captured usingmotion capture laboratories that require large investments. Furthermore,many laboratories cannot easily accommodate athletic and dynamicmotions. This thesis studies three approaches to improving visualperception, with emphasis on human pose estimation, that can complementimprovements to the underlying predictor or training data.The first two papers present active human pose estimation, where areinforcement learning agent is tasked with selecting informativeviewpoints to reconstruct subjects efficiently. The papers discard thecommon assumption that the input is given and instead allow the agent tomove to observe subjects from desirable viewpoints, e.g., those whichavoid occlusions and for which the underlying pose estimator has a lowprediction error.The third paper introduces the task of embodied visual active learning,which goes further and assumes that the perceptual model is notpre-trained. Instead, the agent is tasked with exploring its environmentand requesting annotations to refine its visual model. Learning toexplore novel scenarios and efficiently request annotation for new datais a step towards life-long learning, where models can evolve beyondwhat they learned during the initial training phase. We study theproblem for segmentation, though the idea is applicable to otherperception tasks.Lastly, the final two papers propose improving human pose estimation byintegrating physical constraints. These regularize the reconstructedmotions to be physically plausible and serve as a complement to currentkinematic approaches. Whether a motion has been observed in the trainingdata or not, the predictions should obey the laws of physics. Throughintegration with a physical simulator, we demonstrate that we can reducereconstruction artifacts and enforce, e.g., contact constraints