Substitutional reality:using the physical environment to design virtual reality experiences

Experiencing Virtual Reality in domestic and other uncontrolled settings is challenging due to the presence of physical objects and furniture that are not usually defined in the Virtual Environment. To address this challenge, we explore the concept of Substitutional Reality in the context of Virtual Reality: a class of Virtual Environments where every physical object surrounding a user is paired, with some degree of discrepancy, to a virtual counterpart. We present a model of potential substitutions and validate it in two user studies. In the first study we investigated factors that affect participants' suspension of disbelief and ease of use. We systematically altered the virtual representation of a physical object and recorded responses from 20 participants. The second study investigated users' levels of engagement as the physical proxy for a virtual object varied. From the results, we derive a set of guidelines for the design of future Substitutional Reality experiences

Development of a Physics-Aware Dead Reckoning Mechanism for Distributed Interactive Applications

Distributed Interactive Applications (DIAs) are a class of software that allow geographically remote users to interact within a shared virtual environment. Many DIAs seek to present a rich and realistic virtual world to users, both on a visual and behavioural level. A relatively recent addition to virtual environments (both distributed and single user) to achieve the latter has been the simulation of realistic physical phenomena between objects in the environment. However, the application of physics simulation to virtual environments in DIAs currently lags that of single user environments. This is primarily due to the unavailability of entity state update mechanisms which can maintain consistency in such physics-rich environments. The difference is particularly evident in applications built on a peer-to-peer architecture, as a lack of a single authority presents additional challenges in synchronising the state of shared objects while also presenting a responsive simulation. This thesis proposes a novel state maintenance mechanism for physics-rich environments in peer-to-peer DIAs composed of two parts: a dynamic authority scheme for shared objects, and a physics-aware dead reckoning model with an adaptive error threshold. The first part is intended to place a bound on the overall inconsistency present in shared objects, while the second is implemented to minimise the instantaneous inconsistency during users’ interactions with shared objects. A testbed application is also described, which is used to validate the performance of the proposed mechanism. The state maintenance mechanism is implemented for a single type of physicsaware application, and demonstrates a marked improvement in consistency for that application. However, several flexible terms are described in its implementation, as well as their potential relevance to alternative applications. Finally, it should be noted that the physics-aware dead reckoning model does not depend on the authority scheme, and can therefore be employed with alternative authority scheme

MSL-RAPTOR: A 6DoF Relative Pose Tracker for Onboard Robotic Perception

Determining the relative position and orientation of objects in an environment is a fundamental building block for a wide range of robotics applications. To accomplish this task efficiently in practical settings, a method must be fast, use common sensors, and generalize easily to new objects and environments. We present MSL-RAPTOR, a two-stage algorithm for tracking a rigid body with a monocular camera. The image is first processed by an efficient neural network-based front-end to detect new objects and track 2D bounding boxes between frames. The class label and bounding box is passed to the back-end that updates the object's pose using an unscented Kalman filter (UKF). The measurement posterior is fed back to the 2D tracker to improve robustness. The object's class is identified so a class-specific UKF can be used if custom dynamics and constraints are known. Adapting to track the pose of new classes only requires providing a trained 2D object detector or labeled 2D bounding box data, as well as the approximate size of the objects. The performance of MSL-RAPTOR is first verified on the NOCS-REAL275 dataset, achieving results comparable to RGB-D approaches despite not using depth measurements. When tracking a flying drone from onboard another drone, it outperforms the fastest comparable method in speed by a factor of 3, while giving lower translation and rotation median errors by 66% and 23% respectively.Comment: 12 pages, 6 figures, to be published in 2020 International Symposium on Experimental Robotics (ISER

Molecular Inventories and Chemical Evolution of Low-mass Protostellar Envelopes

This paper presents the first substantial study of the chemistry of the envelopes around a sample of 18 low-mass pre- and protostellar objects for which physical properties have previously been derived from radiative transfer modeling of their dust continuum emission. Single-dish line observations of 24 transitions of 9 molecular species (not counting isotopes) including HCO+, N2H+, CS, SO, SO2, HCN, HNC, HC3N and CN are reported. The line intensities are used to constrain the molecular abundances by comparison to Monte Carlo radiative transfer modeling of the line strengths. An empirical chemical network is constructed on the basis of correlations between the abundances of various species. For example, it is seen that the HCO+ and CO abundances are linearly correlated, both increasing with decreasing envelope mass. Species such as CS, SO and HCN show no trend with envelope mass. In particular no trend is seen between ``evolutionary stage'' of the objects and the abundances of the main sulfur- or nitrogen-containing species. Among the nitrogen-bearing species abundances of CN, HNC and HC3N are found to be closely correlated, which can be understood from considerations of the chemical network. The CS/SO abundance ratio is found to correlate with the abundances of CN and HC3N, which may reflect a dependence on the atomic carbon abundance. An anti-correlation is found between the deuteration of HCO+ and HCN, reflecting different temperature dependences for gas-phase deuteration mechanisms. The abundances are compared to other protostellar environments. In particular it is found that the abundances in the cold outer envelope of the previously studied class 0 protostar IRAS16293-2422 are in good agreement with the average abundances for the presented sample of class 0 objects.Comment: Accepted for publication in A&A. 29 pages, 23 figures. Abstract abridge

Object-Oriented Dynamics Learning through Multi-Level Abstraction

Object-based approaches for learning action-conditioned dynamics has demonstrated promise for generalization and interpretability. However, existing approaches suffer from structural limitations and optimization difficulties for common environments with multiple dynamic objects. In this paper, we present a novel self-supervised learning framework, called Multi-level Abstraction Object-oriented Predictor (MAOP), which employs a three-level learning architecture that enables efficient object-based dynamics learning from raw visual observations. We also design a spatial-temporal relational reasoning mechanism for MAOP to support instance-level dynamics learning and handle partial observability. Our results show that MAOP significantly outperforms previous methods in terms of sample efficiency and generalization over novel environments for learning environment models. We also demonstrate that learned dynamics models enable efficient planning in unseen environments, comparable to true environment models. In addition, MAOP learns semantically and visually interpretable disentangled representations.Comment: Accepted to the Thirthy-Fourth AAAI Conference On Artificial Intelligence (AAAI), 202

User interface development and software environments : the Chiron-1 system

User interface development systems for software environments have to cope with the broad, extensible and dynamic character of such environments, must support internal and external integration, and should enable various software development strategies. The Chiron-1 system adapts and extends key ideas from current research in user interface development systems to address the particular demands of software environments. Important Chiron-1 concepts are: separation of concerns, dynamism, and open architecture. We discuss the requirements on such user interface development systems, present the Chiron-1 architecture and a scenario of its usage, detail the concepts it embodies, and report on its design and prototype implementation

A Framework for Designing 3d Virtual Environments

The process of design and development of virtual environments can be supported by tools and frameworks, to save time in technical aspects and focusing on the content. In this paper we present an academic framework which provides several levels of abstraction to ease this work. It includes state-of-the-art components we devised or integrated adopting open-source solutions in order to face specific problems. Its architecture is modular and customizable, the code is open-source.\u