Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Exploring Robot Teleoperation in Virtual Reality

This thesis presents research on VR-based robot teleoperation with a focus on remote environment visualisation in virtual reality, the effects of remote environment reconstruction scale in virtual reality on the human-operator's ability to control the robot and human-operator's visual attention patterns when teleoperating a robot from virtual reality. A VR-based robot teleoperation framework was developed, it is compatible with various robotic systems and cameras, allowing for teleoperation and supervised control with any ROS-compatible robot and visualisation of the environment through any ROS-compatible RGB and RGBD cameras. The framework includes mapping, segmentation, tactile exploration, and non-physically demanding VR interface navigation and controls through any Unity-compatible VR headset and controllers or haptic devices. Point clouds are a common way to visualise remote environments in 3D, but they often have distortions and occlusions, making it difficult to accurately represent objects' textures. This can lead to poor decision-making during teleoperation if objects are inaccurately represented in the VR reconstruction. A study using an end-effector-mounted RGBD camera with OctoMap mapping of the remote environment was conducted to explore the remote environment with fewer point cloud distortions and occlusions while using a relatively small bandwidth. Additionally, a tactile exploration study proposed a novel method for visually presenting information about objects' materials in the VR interface, to improve the operator's decision-making and address the challenges of point cloud visualisation. Two studies have been conducted to understand the effect of virtual world dynamic scaling on teleoperation flow. The first study investigated the use of rate mode control with constant and variable mapping of the operator's joystick position to the speed (rate) of the robot's end-effector, depending on the virtual world scale. The results showed that variable mapping allowed participants to teleoperate the robot more effectively but at the cost of increased perceived workload. The second study compared how operators used a virtual world scale in supervised control, comparing the virtual world scale of participants at the beginning and end of a 3-day experiment. The results showed that as operators got better at the task they as a group used a different virtual world scale, and participants' prior video gaming experience also affected the virtual world scale chosen by operators. Similarly, the human-operator's visual attention study has investigated how their visual attention changes as they become better at teleoperating a robot using the framework. The results revealed the most important objects in the VR reconstructed remote environment as indicated by operators' visual attention patterns as well as their visual priorities shifts as they got better at teleoperating the robot. The study also demonstrated that operators’ prior video gaming experience affects their ability to teleoperate the robot and their visual attention behaviours

HI-DWA:human influenced dynamic window approach for shared control of a telepresence robot

Abstract. This thesis introduces a shared control mechanism, which allows the user to modify the path of a telepresence robot. The robot is capable of autonomously navigating to a goal predefined by the user, but the user might still want to adjust the path, for example, to go further away from other people or robots, steer away from a wall or go closer to interesting landmarks they want to see on the way. The thesis proposes Human-Influenced Dynamic Window Approach (HI-DWA), a shared control method aimed for immersive telepresence robots based on Dynamic Window Approach (DWA). The proposed method was compared with switching between autonomous navigation and manual control in a user study (N=32), where participants controlled a simulated telepresence robot in Virtual Reality (VR) using various control methods. Results showed that the users reached their goal faster using HI-DWA controller and found it less demanding and easier to use. However, preference between the two methods was split equally. Qualitative analysis revealed that a major reason for the participants that preferred switching between the two modes was the feeling of control. The study also analyzed the effect of different input methods using the HI-DWA, joystick and gesture, on the preference and perceived workload. Even though the gesture based HI-DWA was found significantly harder and more demanding to use, it was preferred more than the proposed method using joystick, or the switching.Käyttäjävaikutteinen DWA-pohjainen etäläsnäolorobotin jaettu ohjaus. Tiivistelmä. Tämä diplomityö esittelee jaetun ohjauksen mekanismin, joka mahdollistaa käyttäjän muuttamaan etäläsnäolorobotin kulkemaa reittiä. Robotti kykenee navigoimaan käyttäjän määrittelemään määränpäähän, mutta käyttäjä saattaa haluta vaikuttaa sen kulkemaan reittiin. Esimerkiksi käyttäjä saatta haluta mennä kauemmaksi muista ihmisistä tai roboteista, kääntää kauemmaksi seinästä tai mennä lähemmäksi mielenkiintoisia kiintopisteitä, joita he haluavat nähdä reitin varrella. Diplomityö esittelee jaettuun ohjaukseen tarkoitetun HI-DWA-menetelmän, joka on suunniteltu immersiivisille etäläsnäolo roboteille ja joka pohjautuu DWA-menetelmään. Ehdotettua menetelmää verrataan menetelmää, jossa käyttäjä voi vaihdella autonomisen ja manuaalisen ohjauksen välillä. Vertailu tehtiin käyttäjäkokeessa (N=32), jossa käyttäjät ohjasivat simuloitua etäläsnäolorobottia virtuaalitodellisuudessa erilaisia ohjausmenetelmiä käyttäen. Tutkimuksen tulokset näyttivät, että käyttäjät pääsivät tavoitepaikkaansa nopeammin käyttäen HI-DWA-metodia ja kokivat sen helpommaksi käyttää ja vähemmän vaativaksi. Kuitenkin, kysymykseen "kumpaa menetelmää suosit"vastaukset jakaantuivat tasan. Kvalitatiivinen analyysi osoitti merkittävimmäksi syyksi autonomisen ja manuaalisen ohjauksen välillä vaihtelun suosioon käyttäjien kokema täysi hallinta robotin liikkeisiin. Tutkimus tutki myös erilaisten ohjausmenetelmien, ohjaussauvan sekä eleohjauksen, vaikutusta HI-DWA-menetelmää käyttäessä suosioon sekä koettuun työtaakkaan. Vaikka eleohjaus koettiin merkittävän verran hankalammaksi sekä vaativammaksi käyttää, melkein puolet käyttäjistä suosivat sitä verrattaessa ohjaussauvaan tai autonomian tason vaihteluun

A control architecture and human interface for agile, reconfigurable micro aerial vehicle formations

This thesis considers the problem of controlling a group of micro aerial vehicles for agile maneuvering cooperatively, or distributively. We first introduce the background and motivation for micro aerial vehicles, especially for the popular multi-rotor aerial vehicle platform. Then, we discuss the dynamics of quadrotor helicopters. A quadrotor is a specific kind of multi-rotor aerial vehicle with a special property called differential flatness, which simplifies the algorithm of trajectory planning, such that, instead of planning a trajectory in a 12-dimensional state space and 4-dimensional input space, we only need to plan the trajectory in 4-dimensional, so called, flat output space, while the 12-dimensional state and 4-dimensional input can be recovered from a mapping called endogenous transformation. We propose a series of approaches to achieve agile maneuvering of a dynamic quadrotor formation, from controlling a single quadrotor in an artificial vector field, to controlling a group of quadrotors in a Virtual Rigid Body (VRB) framework, to balancing the effect between the human control and autonomy for collision avoidance, and to fast on-line distributed collision avoidance with Buffered Voronoi Cells (BVC). In the vector field method, we generate velocity, acceleration, jerk and snap fields, depending on the tasks, or the positions of obstacles, such that a single quadrotor can easily find its required state and input from the endogenous transformation in order to track the artificial vector field. Next, with a Virtual Rigid Body framework, we let a group of quadrotors follow a single control command while also keeping a required formation, or even reconfigure from one formation to another. The Virtual Rigid Body framework decouples the trajectory planning problem into two sub-problems. Then we consider the problem of collision avoidance of the quadrotor formation when it is meanwhile tele-operated by a single human operator. The autonomy with collision avoidance algorithm, based on the vector field methods for a single quadrotor, is an assistive portion of the quadrotor formation controller, such that the human operator can focus on his/her high-level tasks, leaving the low-level collision avoidance task be handled automatically. We also consider the full autonomy problem of quadrotor formations when reconfiguring from one formation to another by developing a fast, on-line distributed collision avoidance algorithm using Buffered Voronoi Cells (BVCs). Our BVC based collision avoidance algorithm only requires sensed relative position, rather than relative position and velocity, while the computational complexity is comparable to other methods like velocity obstacles. At last, we introduce our experimental quadrotor platform which is built from PixHawk flight controller and Odroid-XU4 single-board computer. The hardware and software architecture of this multiple-quadrotor platform is described in detail so that our platform can easily be adopted and extended with different purposes. Our conclusion remark and discussion of future work are also given in this thesi

Navigating Through Virtual Worlds: From Single Characters to Large Crowds

With the rise and success of digital games over the past few decades, path planning algorithms have become an important aspect in modern game development for all types of genres. Indirectly-controlled playable characters as well as non-player characters have to find their way through the game's environment to reach their goal destinations. Modern gaming hardware and new algorithms enable the simulation of large crowds with thousands of individual characters. Still, the task of generating feasible and believable paths in a time- and storage-efficient way is a big challenge in this emerging and exciting research field. In this chapter, the authors describe classical algorithms and data structures, as well as recent approaches that enable the simulation of new and immersive features related to path planning and crowd simulation in modern games. The authors discuss the pros and cons of such algorithms, give an overview of current research questions and show why graph-based methods will soon be replaced by novel approaches that work on a surface-based representation of the environment

Just Around the Corner: The Impact of Instruction Method and Corner Geometry on Teleoperation of Virtual Unmanned Ground Vehicles

Teleoperated robots have proven useful across various domains, as they can more readily search for survivors, survey collapsed and structurally unsound buildings, map out safe routes for rescue workers, and monitor rescue environments. A significant drawback of these robots is that they require the operator to perceive the environment indirectly. As such, camera angles, uneven terrain, lighting, and other environmental conditions can result in robots colliding with obstacles, getting stuck in rubble, and falling over (Casper & Murphy, 2003). To better understand how operators remotely perceive and navigate unmanned ground vehicles, the present work investigated operators’ abilities to negotiate corners of varying widths. In Experiment 1, we evaluated how instruction method impacts cornering time and collisions, looking specifically at the speed-accuracy tradeoff for negotiating corners. Participants navigated a virtual vehicle around corners under the instruction to focus on accuracy (i.e., avoiding collisions) or speed (i.e., negotiating the corners as quickly as possible). We found that as the task became more difficult, subjects’ cornering times increased, and their probability of successful cornering decreased. We also demonstrated that the Fitts’ law speed-accuracy tradeoff could be extended to a cornering task. In Experiment 2, we challenged two of the assumptions of Pastel et al.’s (2007) cornering law and assessed how corner angle and differences in path widths impacted cornering time. Participants navigated a virtual vehicle around corners of varying angles (45°, 90°, and 135°) and varying path widths. We found that increases in corner angle resulted in increased cornering times and a decreased probability of successful cornering. The findings from these experiments are applicable to contexts where an individual is tasked with remotely navigating around corners (e.g., video gaming, USAR, surveillance, military operations, training)

Behavior based autonomous mobile Robot for industrial logistics

The design of robot behaviors to meet the requirements of the new industrial era - Industry 4.0 - has grown significantly in recent years. Especially the demand for flexible and adaptable systems has increased exponentially since intelligent robots started to be integrated into assembly lines and replace human activities. Tools such as Finite State Machines have proven to be an understandable and quick way to solve high-level problems in robotics; however, unmanageable when complexity rises. They become confusing and unreadable, making their modification and mainte- nance a problem. New tools, such as Behavior Trees, have emerged, creating modular, flexible, and adaptable systems without sacrificing readability with the increased com- plexity. The proposed architecture follows a hierarchical layered approach taking advantage of Behavior Trees, developing modular robot skills and system interfaces to create an autonomous behavior-based system. The software was implemented and tested in an Autonomous Mobile Robot capable of navigating complex environments and executing basic tasks. The results showed real advantages in using the layer-based approach, particularly giving the system modularity and increased flexibility capable of being easily improved and used in other systems. It was also concluded that Behavior Trees are an adequate tool for reactive systems in highly dynamic environments.Nos últimos anos, tem-se verificado um crescimentos na modelação de comportamen- tos robóticos com o objetivo de satisfazer necessidades dos novos paradigmas da indústria. Em particular, na indústria 4.0, com a integração de robôs nas linhas de produção e a subs- tituição dos humanos em diversas atividades, tem-se verificado um aumento na exigência de sistemas mais adaptáveis e flexíveis. Ferramentas tais como as máquinas de estado provaram ser percetíveis e de fácil uti- lização na resolução de problemas na área da robótica. No entanto, com o aumento da complexidade, tornam-se problemáticas pela sua desorganização e ilegibilidade. Por con- seguinte, emergiram novas estruturas, tais como as árvores de comportamento, capazes de tornar os sistemas mais modulares e flexíveis. A arquitetura por hierarquisação de camadas proposta, tira partido das vantagens das árvores de comportamento, com o desenvolvimento de comportamentos e interfaces de modo a criar um sistema reativo e autónomo. O software foi implementado e testado num robô móvel autónomo, capaz de navegar em ambientes complexos e de executar tarefas basicas. Os resultados mostraram vantagens na utilização da arquitetura proposta, em parti- cular, trazendo modularidade e flexibilidade ao sistema robótico, permitindo uma futura melhoria de cada um dos módulos, tal como, a sua utilização noutros sistemas

Autonomous navigation in interaction-based environments - a case of non-signalised roundabouts

To reduce the number of collision fatalities at crossroads intersections many countries have started replacing intersections with non-signalised roundabouts, forcing the drivers to be more situationally aware and to adapt their behaviours according to the scenario. A non-signalised roundabout adds to the autonomous vehicle planning challenge, as navigating such interaction dependent scenarios safely, efficiently and comfortably has been a challenge even for human drivers. Unlike traffic signal controlled roundabouts where the merging order is centrally controlled, driving a non-signalised roundabout requires the individual actor to make the decision to merge based on the movement of other interacting actors. Most traditional autonomous planning approaches use rule-based speed assignment for generating admissible motion trajectories, which work successfully in non-interaction-based driving scenarios. They, however, are less effective in interaction-based scenarios as they lack the necessary ability to adapt the vehicle's motion according to the evolving driving scenario. In this paper, we demonstrate an Adaptive Tactical Behaviour Planner (ATBP) for an autonomous vehicle that is capable of planning human-like motion behaviours for navigating a non-signalised roundabout, combining naturalistic behaviour planning and tactical decision-making algorithm. The human driving simulator experiment used to learn the behaviour planning approach and ATBP design are described in the paper

Master of Science

thesisThe objective of this research is to improve the ability of a human operator to drive an omnidirectional robot by using omnidirectional force-feedback. Omnidirectional vehicles offer improved mobility over conventional vehicles and can potentially benefit people requiring motorized transportation and industries where vehicles must operate in confined spaces. However, omnidirectional vehicles require more skill to control due to the additional degrees of freedom inherent in the vehicle’s design. We hypothesize that providing force-feedback to the driver through an omnidirectional joystick will allow the robot to assist the driver in navigating and avoiding collisions with obstacles in a manner that is natural to the operator. This research is the first attempt to use true omnidirectional 3-DOF (degree of freedom) force-feedback to provide navigational assistance for a human to drive an omnidirectional vehicle. While 2-DOF force-feedback has been used in a limited capacity for obstacle avoidance on omnidirectional vehicles, this is the first study to include a third rotational axis of force-feedback and use it to guide a driver along planar collision-avoiding trajectories with a natural coordination of orientation. Unique intellectual merits put forth by this research include use of a novel omnidirectional haptic device and force-feedback strategies to guide operators and experiments to quantify the ability of force-feedback to improve omnidirectional driving performance and driver experience in a real time scenario