Using Synthetic Worlds for Work and Learning

Synthetic worlds [Castronova 2005] are graphically-rich, three-dimensional (3D), electronic environments where members assume an embodied persona (i.e., avatars) and engage in socializing, competitive quests, and economic transactions with globally distributed others. Frequently categorized as technologies of play, synthetic worlds range from massively multiplayer online games (MMOGs) such as World of Warcraft, to virtual reality environments such as Second Life. Increasingly, educators, researchers and corporations are recognizing these 3D online spaces as legitimate communication media, thereby blurring the lines between work and play, and between reality and virtuality. In this panel, presented at the 2007 International Conference on Information Systems, we explore how the fluid work-play and reality-virtuality boundaries are negotiated and managed in practice. The panelists will rely on their research, conducted in educational, corporate and game environments, to address questions about learning, working and playing in these new media spaces

Synthetic worlds, synthetic strategies: attaining creativity in the metaverse

This text will attempt to delineate the underlying theoretical premises and the definition of the output of an immersive learning approach pertaining to the visual arts to be implemented in online, three dimensional synthetic worlds. Deviating from the prevalent practice of the replication of physical art studio teaching strategies within a virtual environment, the author proposes instead to apply the fundamental tenets of Roy Ascott’s “Groundcourse”, in combination with recent educational approaches such as “Transformative Learning” and “Constructionism”. In an amalgamation of these educational approaches with findings drawn from the fields of Metanomics, Ludology, Cyberpsychology and Presence Studies, as well as an examination of creative practices manifest in the metaverse today, the formulation of a learning strategy for creative enablement unique to online, three dimensional synthetic worlds; one which will focus upon “Play” as well as Role Play, virtual Assemblage and the visual identity of the avatar within the pursuits, is being proposed in this chapter

Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications

We present an overview and evaluation of a new, systematic approach for generation of highly realistic, annotated synthetic data for training of deep neural networks in computer vision tasks. The main contribution is a procedural world modeling approach enabling high variability coupled with physically accurate image synthesis, and is a departure from the hand-modeled virtual worlds and approximate image synthesis methods used in real-time applications. The benefits of our approach include flexible, physically accurate and scalable image synthesis, implicit wide coverage of classes and features, and complete data introspection for annotations, which all contribute to quality and cost efficiency. To evaluate our approach and the efficacy of the resulting data, we use semantic segmentation for autonomous vehicles and robotic navigation as the main application, and we train multiple deep learning architectures using synthetic data with and without fine tuning on organic (i.e. real-world) data. The evaluation shows that our approach improves the neural network's performance and that even modest implementation efforts produce state-of-the-art results.Comment: The project web page at http://vcl.itn.liu.se/publications/2017/TKWU17/ contains a version of the paper with high-resolution images as well as additional materia

Spatio-Temporal Image Boundary Extrapolation

Boundary prediction in images as well as video has been a very active topic of research and organizing visual information into boundaries and segments is believed to be a corner stone of visual perception. While prior work has focused on predicting boundaries for observed frames, our work aims at predicting boundaries of future unobserved frames. This requires our model to learn about the fate of boundaries and extrapolate motion patterns. We experiment on established real-world video segmentation dataset, which provides a testbed for this new task. We show for the first time spatio-temporal boundary extrapolation in this challenging scenario. Furthermore, we show long-term prediction of boundaries in situations where the motion is governed by the laws of physics. We successfully predict boundaries in a billiard scenario without any assumptions of a strong parametric model or any object notion. We argue that our model has with minimalistic model assumptions derived a notion of 'intuitive physics' that can be applied to novel scenes

Long-Term Image Boundary Prediction

Boundary estimation in images and videos has been a very active topic of research, and organizing visual information into boundaries and segments is believed to be a corner stone of visual perception. While prior work has focused on estimating boundaries for observed frames, our work aims at predicting boundaries of future unobserved frames. This requires our model to learn about the fate of boundaries and corresponding motion patterns -- including a notion of "intuitive physics". We experiment on natural video sequences along with synthetic sequences with deterministic physics-based and agent-based motions. While not being our primary goal, we also show that fusion of RGB and boundary prediction leads to improved RGB predictions.Comment: Accepted in the AAAI Conference for Artificial Intelligence, 201

Synthetic worlds, synthetic strategies: attaining creativity in the metaverse

This text will attempt to delineate the underlying theoretical premises and the definition of the output of an immersive learning approach pertaining to the visual arts to be implemented in online, three dimensional synthetic worlds. Deviating from the prevalent practice of the replication of physical art studio teaching strategies within a virtual environment, the author proposes instead to apply the fundamental tenets of Roy Ascott’s “Groundcourse”, in combination with recent educational approaches such as “Transformative Learning” and “Constructionism”. In an amalgamation of these educational approaches with findings drawn from the fields of Metanomics, Ludology, Cyberpsychology and Presence Studies, as well as an examination of creative practices manifest in the metaverse today, the formulation of a learning strategy for creative enablement unique to online, three dimensional synthetic worlds; one which will focus upon “Play” as well as Role Play, virtual Assemblage and the visual identity of the avatar within the pursuits, is being proposed in this chapter

Explore, Exploit or Listen: Combining Human Feedback and Policy Model to Speed up Deep Reinforcement Learning in 3D Worlds

We describe a method to use discrete human feedback to enhance the performance of deep learning agents in virtual three-dimensional environments by extending deep-reinforcement learning to model the confidence and consistency of human feedback. This enables deep reinforcement learning algorithms to determine the most appropriate time to listen to the human feedback, exploit the current policy model, or explore the agent's environment. Managing the trade-off between these three strategies allows DRL agents to be robust to inconsistent or intermittent human feedback. Through experimentation using a synthetic oracle, we show that our technique improves the training speed and overall performance of deep reinforcement learning in navigating three-dimensional environments using Minecraft. We further show that our technique is robust to highly innacurate human feedback and can also operate when no human feedback is given