Collaboration with agents in VR environments

Virtual reality is gaining on importance in many fields – scientific simulation, training, therapy and also more and more in entertainment. All these applications require the human user to interact with virtual worlds inhabited by intelligent characters and to solve simulated or real problems. This thesis will present an integrated approach to simulated problem solving in virtual reality environments, with the emphasis on teamwork and the ability to control the simulations. A simulation framework satisfying these goals will be presented. A unified approach to the representation of semantic information in virtual environments based on predicate calculus will be introduced, including the representation of the world state, action semantics and basic axioms holding in the simulated world. Afterwards, the focus will be on the collaboration model based on task delegation and facilitator-centric architecture. A simple but efficient facilitator design will be presented. The issues of the collaborative problem solving will be examined. A new technique using propositional (STRIPS-like) planning with delegated actions and object-specific planning will be described. A control technique for virtual characters/objects will be detailed, enabling run-time exchange of control and control sharing over a virtual entity between multiple autonomous agents and/or human users. Finally, a set of case studies will be shown, illustrating the possible applications of the techniques developed and described in this dissertation

Dynamic Simulation for Zero-Gravity Activities

Working and training for space activities is difficult in terrestrial environments. We approach this crucial aspect of space human factors through 3D computer graphics dynamics simulation of crewmembers, their tasks, and physics-based movement modeling. Such virtual crewmembers may be used to design tasks and analyze their physical workload to maximize success and safety without expensive physical mockups or partially realistic neutral-buoyancy tanks. Among the software tools we have developed are methods for fully articulated 3D human models and dynamic simulation. We are developing a fast recursive dynamics algorithm for dynamically simulating articulated 3D human models, which comprises kinematic chains - serial, closed-loop, and tree-structure - as well as the inertial properties of the segments. Motion planning is done by first solving the inverse kinematic problem to generate possible trajectories, and then by solving the resulting nonlinear optimal control problem. For example, the minimization of the torques during a simulation under certain constraints is usually applied and has its origin in the biomechanics literature. Examples of space activities shown are zero-gravity self orientation and ladder traversal. Energy expenditure is computed for the traversal task

Neurodiverse Human-Machine Interaction and Collaborative Problem-Solving in Social VR

Social motor coordination is an important mechanism responsible for creating shared understanding but can be a challenge for Autistic individuals. Social virtual reality (VR) provides an opportunity to create a safe and inclusive environment for which interactions can be augmented to promote social interactivity. Due to the bi-directional nature of social interaction and adaptation, we created a framework to explore social motor coordination with a virtual artificial agent which can exhibit human-like behaviors. In this experiment, we assessed the interactive behaviors of participants completing a collaborative problem-solving task with the agent using multidimensional cross-recurrence quantification analysis (mdCRQA). Our results show that participants who discovered novel solutions to the task exhibited greater coupling to the artificial agent regardless of participant characteristics. Future work will explore how social VR environments can be augmented to promote social coordination

Gaming Business Communities: Developing online learning organisations to foster communities, develop leadership, and grow interpersonal education

This paper explores, through observation and testing, what possibilities from gaming can be extended into other realms of human interaction to help bring people together, extend education, and grow business. It uses through action learning within the safety of the virtual world within Massively Multiplayer Online Games. Further, I explore how the world of online gaming provides opportunity to train a wide range of skills through extending Revans’ (1980) learning equation and action inquiry methodology. This equation and methodology are deployed in relation to a gaming community to see if the theories could produce strong relationships within organisations and examine what learning, if any, is achievable. I also investigate the potential for changes in business (e.g., employee and customer relationships) through involvement in the gaming community as a unique place to implement action learning. The thesis also asks the following questions on a range of extended possibilities in the world of online gaming: What if the world opened up to a social environment where people could discuss their successes and failures? What if people could take a real world issue and re‐create it in the safe virtual world to test ways of dealing with it? What education answers can the world of online gaming provide

The virtual playground: an educational virtual reality environment for evaluating interactivity and conceptual learning

The research presented in this paper aims at investigating user interaction in immersive virtual learning environments (VLEs), focusing on the role and the effect of interactivity on conceptual learning. The goal has been to examine if the learning of young users improves through interacting in (i.e. exploring, reacting to, and acting upon) an immersive virtual environment (VE) compared to non interactive or non-immersive environments. Empirical work was carried out with more than 55 primary school students between the ages of 8 and 12, in different between-group experiments: an exploratory study, a pilot study, and a large-scale experiment. The latter was conducted in a virtual environment designed to simulate a playground. In this ‘Virtual Playground’, each participant was asked to complete a set of tasks designed to address arithmetical ‘fractions’ problems. Three different conditions, two experimental virtual reality (VR) conditions and a non-VR condition, that varied the levels of activity and interactivity, were designed to evaluate how children accomplish the various tasks. Pre-tests, post-tests, interviews, video, audio, and log files were collected for each participant, and analyzed both quantitatively and qualitatively. This paper presents a selection of case studies extracted from the qualitative analysis, which illustrate the variety of approaches taken by children in the VEs in response to visual cues and system feedback. Results suggest that the fully interactive VE aided children in problem solving but did not provide as strong evidence of conceptual change as expected; rather, it was the passive VR environment, where activity was guided by a virtual robot, that seemed to support student reflection and recall, leading to indications of conceptual change

Supporting Memorization and Problem Solving with Spatial Information Presentations in Virtual Environments

While it has been suggested that immersive virtual environments could provide benefits for educational applications, few studies have formally evaluated how the enhanced perceptual displays of such systems might improve learning. Using simplified memorization and problem-solving tasks as representative approximations of more advanced types of learning, we are investigating the effects of providing supplemental spatial information on the performance of learning-based activities within virtual environments. We performed two experiments to investigate whether users can take advantage of a spatial information presentation to improve performance on cognitive processing activities. In both experiments, information was presented either directly in front of the participant or wrapped around the participant along the walls of a surround display. In our first experiment, we found that the spatial presentation caused better performance on a memorization and recall task. To investigate whether the advantages of spatial information presentation extend beyond memorization to higher level cognitive activities, our second experiment employed a puzzle-like task that required critical thinking using the presented information. The results indicate that no performance improvements or mental workload reductions were gained from the spatial presentation method compared to a non-spatial layout for our problem-solving task. The results of these two experiments suggest that supplemental spatial information can support performance improvements for cognitive processing and learning-based activities, but its effectiveness is dependent on the nature of the task and a meaningful use of space

Modelling virtual urban environments

In this paper, we explore the way in which virtual reality (VR) systems are being broadened to encompass a wide array of virtual worlds, many of which have immediate applicability to understanding urban issues through geocomputation. Wesketch distinctions between immersive, semi-immersive and remote environments in which single and multiple users interact in a variety of ways. We show how suchenvironments might be modelled in terms of ways of navigating within, processes of decision-making which link users to one another, analytic functions that users have to make sense of the environment, and functions through which users can manipulate, change, or design their world. We illustrate these ideas using four exemplars that we have under construction: a multi-user internet GIS for Londonwith extensive links to 3-d, video, text and related media, an exploration of optimal retail location using a semi-immersive visualisation in which experts can explore such problems, a virtual urban world in which remote users as avatars can manipulate urban designs, and an approach to simulating such virtual worlds through morphological modelling based on the digital record of the entire decision-making process through which such worlds are built