An empirical comparison of interaction styles for map interfaces in immersive virtual environments

Geographical Information Systems (GIS) can be visualized using immersive technologies like Virtual Reality (VR). Before using this kind of technologies it is required to explore which interactions are affordable, efficient and satisfactory from the users' point of view. The purpose of this work is to provide insight on how to design efficient and natural interaction on GIS VR interfaces. This study presents a within-subjects comparative study that assesses the usability and performance of two popular interaction strategies: body-based interaction and device based interaction. In body-based interaction, participants use their hands and head orientation to control the VR map. In the second case, users interact with the Oculus Touch controller. Thirty two users participated in an experiment whose results suggest that interacting with the controller improves performance of the selection task, in terms of time spent and error rate. Also, the results show a preference of users for the controller in terms of perceived usability.This work is supported by the project PACE funded by the Spanish Ministry of Economy, Industry and Competitiveness (TIN2016-77690-R

An Overview of Enhancing Distance Learning Through Augmented and Virtual Reality Technologies

Although distance learning presents a number of interesting educational advantages as compared to in-person instruction, it is not without its downsides. We first assess the educational challenges presented by distance learning as a whole, and identify 4 main challenges that distance learning currently presents as compared to in-person instruction: the lack of social interaction, reduced student engagement and focus, reduced comprehension and information retention, and the lack of flexible and customizable instructor resources. After assessing each of these challenges in-depth, we examine how AR/VR technologies might serve to address each challenge along with their current shortcomings, and finally outline the further research that is required to fully understand the potential of AR/VR technologies as they apply to distance learning.Comment: 12 pages, 7 figures, submitted to TVC

Learning in a Mixed Reality System in the Context of ‚Industrie 4.0‘

This contribution in the field of innovative approaches to training and education in technical subjects focuses on the potential of modern teaching and learning environments. The contribution is based on a theoretical introduction to Mixed Reality Systems and virtual teaching and learning systems, and as such provides an overview of current research regarding modern learning environments. In particular, it takes a close look at motivational effects in the context of web-based learning structures, human-object interactions, gamification and immersion. The article discusses both technical, user-relevant and pedagogical aspects as well as suggestions for further research in the context of Ausbildung 4.0.Keywords: Industry 4.0, Vocational Training 4.0, Mixed Reality System, virtual learning AcknowledgementThe author would like to thank the ChinaScholarshipCouncil(CSC) for the financial support (No. 201406030091)

Designing 3D scenarios and interaction tasks for immersive environments

In the world of today, immersive reality such as virtual and mixed reality, is one of the most attractive research fields. Virtual Reality, also called VR, has a huge potential to be used in in scientific and educational domains by providing users with real-time interaction or manipulation. The key concept in immersive technologies to provide a high level of immersive sensation to the user, which is one of the main challenges in this field. Wearable technologies play a key role to enhance the immersive sensation and the degree of embodiment in virtual and mixed reality interaction tasks. This project report presents an application study where the user interacts with virtual objects, such as grabbing objects, open or close doors and drawers while wearing a sensory cyberglove developed in our lab (Cyberglove-HT). Furthermore, it presents the development of a methodology that provides inertial measurement unit(IMU)-based gesture recognition. The interaction tasks and 3D immersive scenarios were designed in Unity 3D. Additionally, we developed an inertial sensor-based gesture recognition by employing an Long short-term memory (LSTM) network. In order to distinguish the effect of wearable technologies in the user experience in immersive environments, we made an experimental study comparing the Cyberglove-HT to standard VR controllers (HTC Vive Controller). The quantitive and subjective results indicate that we were able to enhance the immersive sensation and self embodiment with the Cyberglove-HT. A publication resulted from this work [1] which has been developed in the framework of the R&D project Human Tracking and Perception in Dynamic Immersive Rooms (HTPDI

Development of a virtual audience concept for public speaking training in immersive virtual environments

Public speaking is common in today's work and study environment, yet many people dread giving presentations and sometimes even avoid them altogether. A frequent solution is to take up public speaking courses where speakers can learn presentation skills from speech trainers and rehearse in front of other trainees. By combining the opportunities of virtual reality with the characteristics and challenges of training sessions, the dissertation proposes a design concept for a virtual audience that can serve as speech training tool in immersive virtual environments. Through this approach the research addresses public speaking from a training angle - a novel view that wishes to complement the existing speech anxiety treatment studies in virtual reality. The work rests on a multidisciplinary ground and includes perspectives from interpersonal communication, virtual reality, human-computer interaction, and design of virtual learning environments.Die vorliegende Dissertation schlägt einen neuen Ansatz für Kommunikationstrainings und Sprechübungen vor, und führt ein Konzept für ein virtuelles Publikum ein. Das Publikum soll hierbei als Trainingsinstrument dienen und richtet sich an diejenigen, welche ihre rhetorischen Fähigkeiten und Präsentationstechniken verbessern möchten. Die Schlüssel-Komponente des Publikums bilden virtuelle Menschen, wobei diese möglichst realistisch aussehen und agieren sollen. Dies wirft eine zentrale Frage auf: Was sind die Merkmale eines realen Publikums und was machen die Menschen überhaupt, wenn sie vor einem Sprecher sitzen? Um diese Frage zu beantworten, wurden zwei Studien durchgeführt. Die erste explorative Studie beschäftigt sich mit Experten im Kommunikationstraining und VR Bereich. Sieben Experten wurden sowohl über die Rolle von Publikumsanpassung, Interaktion, und Gruppendynamik als auch über technische Anwendungsmöglichkeiten in der virtuellen Realität befragt. Die Ergebnisse zeigen, dass die Kommunikationstrainingsprogramme hohe Publikumsanpassungen und Interaktivitätsmöglichkeiten erfordern. Um die Ergebnisse zu komplementieren, wurde ein Trainingsseminar beobachtet. Die Resultate der Beobachtung verdeutlichen die Bedeutung von Sprechübungen im Rahmen eines Kommunikationsprogramms sowie die bevorzugten Feedbackmethoden. Die zweite Studie umfasst die Beobachtung eines studentischen Publikums im Rahmen einer Vorlesung an der Technischen Universität Ilmenau. Die nonverbalen Verhaltensweisen von 14 Studierenden wurden kodiert und in aufmerksame oder unaufmerksame Verhaltensweisen aufgeteilt. Weiterhin wurde die Frequenz, Komplexität und Dauer der ausgewiesenen Reaktionen analysiert. Die Ergebnisse dieser zwei Studien bilden das Fundament für die Entwicklung eines Designkonzepts für ein virtuelles Publikum. Dieses bezieht unterschiedliche Merkmale (z.B. demographische Charakteristika, Vorschläge für virtuelle Räumlichkeiten, etc.) ein. Zusätzlich wird eine Liste mit Indikatoren für Aufmerksamkeit und Unaufmerksamkeit eingebunden und ein fünfminütiges Szenario mit fünf virtuellen Menschen vorgestellt.The present dissertation proposes a novel approach to public speaking training and introduces a concept for a virtual audience in immersive virtual environments. The key component of such an audience are the virtual humans (VHs) whose role is to look and behave as closely as possible to a real public. Virtual audiences can then be used as training tools for people who wish to improve their public speaking and presentation skills. Two empirical studies were conducted to help identify relevant audience features and behaviors that occur in real life and that can be modeled for a virtual public. The first one is an exploratory study with experts in communication training and in fields related to virtual reality (VR). Seven experts were interviewed on the role and importance of audience customization, interaction, and group dynamic during training, as well as on technical possibilities to design virtual audiences with such features. The results show that audiences for communication training require extensive customization and interactivity options. To complement the findings, a speech practice session in a communication training seminar was observed and helped reveal the role of speech practice in the economy of the whole training seminar as well as the preferred feedback methods. In the second study, a video observation of a student audience during a lecture at Technische Universität Ilmenau was conducted. The nonverbal behaviors of 14 students were coded and divided into attentive and inattentive manifestations. The identified behaviors are further described and analyzed in terms of their frequency, complexity, and duration. The findings of both these studies helped create a design concept for a virtual audience with various characteristics (e.g., demographic features and virtual spaces they could inhabit) and a list with attentive and inattentive nonverbal behaviors the virtual humans could display. A five-minute scenario with virtual listeners is suggested as well

TREE-D-SEEK: A Framework for Retrieving Three-Dimensional Scenes

In this dissertation, a strategy and framework for retrieving 3D scenes is proposed. The strategy is to retrieve 3D scenes based on a unified approach for indexing content from disparate information sources and information levels. The TREE-D-SEEK framework implements the proposed strategy for retrieving 3D scenes and is capable of indexing content from a variety of corpora at distinct information levels. A semantic annotation model for indexing 3D scenes in the TREE-D-SEEK framework is also proposed. The semantic annotation model is based on an ontology for rapid prototyping of 3D virtual worlds. With ongoing improvements in computer hardware and 3D technology, the cost associated with the acquisition, production and deployment of 3D scenes is decreasing. As a consequence, there is a need for efficient 3D retrieval systems for the increasing number of 3D scenes in corpora. An efficient 3D retrieval system provides several benefits such as enhanced sharing and reuse of 3D scenes and 3D content. Existing 3D retrieval systems are closed systems and provide search solutions based on a predefined set of indexing and matching algorithms Existing 3D search systems and search solutions cannot be customized for specific requirements, type of information source and information level. In this research, TREE-D-SEEK—an open, extensible framework for retrieving 3D scenes—is proposed. The TREE-D-SEEK framework is capable of retrieving 3D scenes based on indexing low level content to high-level semantic metadata. The TREE-D-SEEK framework is discussed from a software architecture perspective. The architecture is based on a common process flow derived from indexing disparate information sources. Several indexing and matching algorithms are implemented. Experiments are conducted to evaluate the usability and performance of the framework. Retrieval performance of the framework is evaluated using benchmarks and manually collected corpora. A generic, semantic annotation model is proposed for indexing a 3D scene. The primary objective of using the semantic annotation model in the TREE-D-SEEK framework is to improve retrieval relevance and to support richer queries within a 3D scene. The semantic annotation model is driven by an ontology. The ontology is derived from a 3D rapid prototyping framework. The TREE-D-SEEK framework supports querying by example, keyword based and semantic annotation based query types for retrieving 3D scenes

Application of mixed and virtual reality in geoscience and engineering geology

Visual learning and efficient communication in mining and geotechnical practices is crucial, yet often challenging. With the advancement of Virtual Reality (VR) and Mixed Reality (MR) a new era of geovisualization has emerged. This thesis demonstrates the capabilities of a virtual continuum approach using varying scales of geoscience applications. An application that aids analyses of small-scale geological investigation was constructed using a 3D holographic drill core model. A virtual core logger was also developed to assist logging in the field and subsequent communication by visualizing the core in a complementary holographic environment. Enriched logging practices enhance interpretation with potential economic and safety benefits to mining and geotechnical infrastructure projects. A mine-scale model of the LKAB mine in Sweden was developed to improve communication on mining induced subsidence between geologists, engineers and the public. GPS, InSAR and micro-seismicity data were hosted in a single database, which was geovisualized through Virtual and Mixed Reality. The wide array of applications presented in this thesis illustrate the potential of Mixed and Virtual Reality and improvements gained on current conventional geological and geotechnical data collection, interpretation and communication at all scales from the micro- (e.g. thin section) to the macro- scale (e.g. mine)