The matrix revisited: A critical assessment of virtual reality technologies for modeling, simulation, and training

A convergence of affordable hardware, current events, and decades of research have advanced virtual reality (VR) from the research lab into the commercial marketplace. Since its inception in the 1960s, and over the next three decades, the technology was portrayed as a rarely used, high-end novelty for special applications. Despite the high cost, applications have expanded into defense, education, manufacturing, and medicine. The promise of VR for entertainment arose in the early 1990\u27s and by 2016 several consumer VR platforms were released. With VR now accessible in the home and the isolationist lifestyle adopted due to the COVID-19 global pandemic, VR is now viewed as a potential tool to enhance remote education. Drawing upon over 17 years of experience across numerous VR applications, this dissertation examines the optimal use of VR technologies in the areas of visualization, simulation, training, education, art, and entertainment. It will be demonstrated that VR is well suited for education and training applications, with modest advantages in simulation. Using this context, the case is made that VR can play a pivotal role in the future of education and training in a globally connected world

MAR-CPS: Measurable Augmented Reality for Prototyping Cyber-Physical Systems

Cyber-Physical Systems (CPSs) refer to engineering platforms that rely on the inte- gration of physical systems with control, computation, and communication technologies. Autonomous vehicles are instances of CPSs that are rapidly growing with applications in many domains. Due to the integration of physical systems with computational sens- ing, planning, and learning in CPSs, hardware-in-the-loop experiments are an essential step for transitioning from simulations to real-world experiments. This paper proposes an architecture for rapid prototyping of CPSs that has been developed in the Aerospace Controls Laboratory at the Massachusetts Institute of Technology. This system, referred to as MAR-CPS (Measurable Augmented Reality for Prototyping Cyber-Physical Systems), includes physical vehicles and sensors, a motion capture technology, a projection system, and a communication network. The role of the projection system is to augment a physical laboratory space with 1) autonomous vehicles' beliefs and 2) a simulated mission environ- ment, which in turn will be measured by physical sensors on the vehicles. The main focus of this method is on rapid design of planning, perception, and learning algorithms for au- tonomous single-agent or multi-agent systems. Moreover, the proposed architecture allows researchers to project a simulated counterpart of outdoor environments in a controlled, indoor space, which can be crucial when testing in outdoor environments is disfavored due to safety, regulatory, or monetary concerns. We discuss the issues related to the design and implementation of MAR-CPS and demonstrate its real-time behavior in a variety of problems in autonomy, such as motion planning, multi-robot coordination, and learning spatio-temporal fields.Boeing Compan

Proceedings of the 1993 Conference on Intelligent Computer-Aided Training and Virtual Environment Technology

The volume 2 proceedings from the 1993 Conference on Intelligent Computer-Aided Training and Virtual Environment Technology are presented. Topics discussed include intelligent computer assisted training (ICAT) systems architectures, ICAT educational and medical applications, virtual environment (VE) training and assessment, human factors engineering and VE, ICAT theory and natural language processing, ICAT military applications, VE engineering applications, ICAT knowledge acquisition processes and applications, and ICAT aerospace applications

Virtual reality as an educational tool in interior architecture

Ankara : The Department of Interior Architecture and Environmental Design and the Institute of Fine Arts of Bilkent Univ., 1997.Thesis (Master's) -- Bilkent University, 1997.Includes bibliographical references.This thesis discusses the use of virtual reality technology as an educational tool in interior architectural design. As a result of this discussion, it is proposed that virtual reality can be of use in aiding three-dimensional design and visualization, and may speed up the design process. It may also be of help in getting the designers/students more involved in their design projects. Virtual reality can enhance the capacity of designers to design in three dimensions. The virtual reality environment used in designing should be capable of aiding both the design and the presentation process. The tradeoffs of the technology, newly emerging trends and future directions in virtual reality are discussed.Aktaş, OrkunM.S

Need Finding for an Embodied Coding Platform: Educators’ Practices and Perspectives

Eight middle- and high-school Computer Science (CS) teachers in San Diego County were interviewed about the major challenges their students commonly encounter in learning computer programming. We identified strategic design opportunities -- that is, challenges and needs that can be addressed in innovative ways through the affordances of Augmented and Virtual Reality (AR/VR). Thematic Analysis of the interviews yielded six thematic clusters: Tools for Learning, Visualization and Representation, Pedagogical Approaches, Classroom Culture, Motivation, and Community Connections. Within the theme of visualization, focal clusters centered on visualizing problem spaces and using metaphors to explain computational concepts, indicating that an AR/VR coding system could help users to represent computational problems by allowing them to build from existing embodied experiences and knowledge. Additionally, codes clustered within the theme of learning tools reflected educators’ preference for web-based IDEs, which involve minimal start-up costs, as well as concern over the degree of transfer in learning between block- and text-based interfaces. Finally, themes related to motivation, community, and pedagogical practices indicated that the design of an AR coding platform should support collaboration, self-expression, and autonomy in learning. It should also foster selfefficacy and learners’ ability to address lived experience and real-world problems through computational means

3D Sensing Character Simulation using Game Engine Physics

Creating visual 3D sensing characters that interact with AI peers and the virtual envi- ronment can be a difficult task for those with less experience in using learning algorithms or creating visual environments to execute an agent-based simulation. In this thesis, the use of game engines was studied as a tool to create and execute vi- sual simulations with 3D sensing characters, and train game ready bots. The idea was to make use of the game engine’s available tools to create highly visual simulations without requiring much knowledge in modeling or animation, as well as integrating exterior agent simulation libraries to create sensing characters without needing expertise in learning algorithms. These sensing characters, were be 3D humanoid characters that can perform the basic functions of a game character such as moving, jumping, and interacting, but also have simulated different senses in them. The senses that these characters can have include: touch using collision detection, vision using ray casts, directional sound, smell, and other imaginable senses. These senses are obtained using different game develop- ment techniques available in the game engine and can be used as input for the learning algorithm to help the character learn. This allows the simulation of agents using off-the- shelf algorithms and using the game engine’s motor for the visualizations of these agents. We explored the use of these tools to create visual bots for games, and teach them how to play the game until they reach a level where they can serve as adversaries for real-life players in interactive games. This solution was tested using both reinforcement learning and imitation learning algorithms in an attempt to compare how efficient both learning methods can be when used to teach sensing game bots in different game scenarios. These scenarios varied in both objective and environment complexity as well as the number of bots to access how each solution behaves in different scenarios. In this document is presented a related work on the agent simulation and game engine areas, followed by a more detailed solution and its implementation ending with practical tests and its results.Criar visualizações de personagens 3D com sentidos que interagem com colegas de IA e com o ambiente virtual pode ser uma tarefa difícil para programadores com menos experiência no uso de algoritmos de aprendizagem automática ou na criação de ambientes visuais para executar simulações baseadas em agentes. Nesta tese foi estudado o uso de motores de jogos como ferramenta para criar e execu- tar simulações visuais com personagens 3D, e treinar bots para jogos. A ideia foi usar as ferramentas disponíveis do motor de jogos para criar simulações visuais sem exigir muito conhecimento em modelação ou animação, para além de integrar bibliotecas de simulação de agentes externas para criar personagens com sentidos sem precisar de conhecimentos em algoritmos de aprendizagem automática. Estas personagens 3D são humanoides que podem desempenhar as funções básicas de uma personagem de um jogo como mover, saltar e interagir, mas também terão simulados neles diferentes sentidos. Os sentidos que estas personagens podem ter inclui: o tato, colisões, visão, som direcional, olfato e outros sentidos imagináveis. Estes sentidos são obtidos usando diferentes técnicas de desenvol- vimento de jogos disponíveis no motor de jogos, e podem ser usados como inputs para os algoritmos de aprendizagem automática para ajudar as personagens a aprender. Esta solução foi testada usando algoritmos de Reinforcement Learning e Imitation Le- arning, com o intuito de comparar a eficiência de ambos os métodos de aprendizagem quando usados para ensinar bots de jogos em diferentes cenários. Estes cenários variaram em complexidade de objetivo e ambiente, e também no número de bots para que se possa visualizar como cada algoritmo se comporta em diferentes cenários. Neste documento será apresentado um estado da arte nas áreas de simulação de agentes e motores de jogos, seguido de uma proposta de solução mais detalhada para este problema

Proceedings of the 1993 Conference on Intelligent Computer-Aided Training and Virtual Environment Technology, Volume 1

These proceedings are organized in the same manner as the conference's contributed sessions, with the papers grouped by topic area. These areas are as follows: VE (virtual environment) training for Space Flight, Virtual Environment Hardware, Knowledge Aquisition for ICAT (Intelligent Computer-Aided Training) & VE, Multimedia in ICAT Systems, VE in Training & Education (1 & 2), Virtual Environment Software (1 & 2), Models in ICAT systems, ICAT Commercial Applications, ICAT Architectures & Authoring Systems, ICAT Education & Medical Applications, Assessing VE for Training, VE & Human Systems (1 & 2), ICAT Theory & Natural Language, ICAT Applications in the Military, VE Applications in Engineering, Knowledge Acquisition for ICAT, and ICAT Applications in Aerospace

Reconciling the dissonance between Historic Preservation and Virtual Reality through a Place-based Virtual Heritage system.

This study explores a problematic disconnect associated with virtual heritage and the immersive 3D computer modeling of cultural heritage. The products of virtual heritage often fail to adhere to long-standing principles and recent international conventions associated with historic preservation, heritage recording, designation, and interpretation. By drawing upon the geographic concepts of space, landscape, and place, along with advances in Geographic Information Systems, first-person serious games, and head-mounted Virtual Reality platforms this study envisions, designs, implements, and evaluates a virtual heritage system that seeks to reconcile the dissonance between Virtual Reality and historic preservation. Finally, the dissertation examines the contributions and future directions of such a Place-based Virtual Heritage system in human geography and historic preservation planning and interpretation