Human-robot Interaction For Multi-robot Systems

Designing an effective human-robot interaction paradigm is particularly important for complex tasks such as multi-robot manipulation that require the human and robot to work together in a tightly coupled fashion. Although increasing the number of robots can expand the area that the robots can cover within a bounded period of time, a poor human-robot interface will ultimately compromise the performance of the team of robots. However, introducing a human operator to the team of robots, does not automatically improve performance due to the difficulty of teleoperating mobile robots with manipulators. The human operator’s concentration is divided not only among multiple robots but also between controlling each robot’s base and arm. This complexity substantially increases the potential neglect time, since the operator’s inability to effectively attend to each robot during a critical phase of the task leads to a significant degradation in task performance. There are several proven paradigms for increasing the efficacy of human-robot interaction: 1) multimodal interfaces in which the user controls the robots using voice and gesture; 2) configurable interfaces which allow the user to create new commands by demonstrating them; 3) adaptive interfaces which reduce the operator’s workload as necessary through increasing robot autonomy. This dissertation presents an evaluation of the relative benefits of different types of user interfaces for multi-robot systems composed of robots with wheeled bases and three degree of freedom arms. It describes a design for constructing low-cost multi-robot manipulation systems from off the shelf parts. User expertise was measured along three axes (navigation, manipulation, and coordination), and participants who performed above threshold on two out of three dimensions on a calibration task were rated as expert. Our experiments reveal that the relative expertise of the user was the key determinant of the best performing interface paradigm for that user, indicating that good user modiii eling is essential for designing a human-robot interaction system that will be used for an extended period of time. The contributions of the dissertation include: 1) a model for detecting operator distraction from robot motion trajectories; 2) adjustable autonomy paradigms for reducing operator workload; 3) a method for creating coordinated multi-robot behaviors from demonstrations with a single robot; 4) a user modeling approach for identifying expert-novice differences from short teleoperation traces

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

An efficacious method to assemble a modern multi-modal robotic team: dilemmas, challenges, possibilities and solutions

A modern multiagent robotic platform consists of a cooperative team of humans which develop a collaborative team of robots. The multi-modal nature of both the system and the team causes a complex problem which needs to be solved for optimum performance. Both the management and the technical aspect of a modern robotic team are explored in this Chapter in the platform of the RoboCup Competition. RoboCup is an example of such an environment where researchers from different disciplines join to develop a robotic team for completion as an evaluation challenge (Robocup, 2011). RoboCup competitions were first proposed by Mackworth in 1993. The main goal of this scientific competition is to exploit, improve and integrate the methods and techniques from robotics, machine vision and artificial intelligence disciplines to create an autonomous team of soccer playing robots(Kitano, 1997a; Kitano, 1997b; Kitano et al., 1997). Such experiment includes several challenges, from inviting an expert of specific field to the team to choosing bolts and nuts for each part of the robots. Usually each challenge has several possible solutions and choosing the best one is often challenging. We have participated in several world wide RoboCup competitions (Abdollahi, Samani et al. 2002, 2003 & 2004) and share our experience as an extensive instruction for setting up a modern robotic team including management and technical issues.Peer ReviewedPostprint (published version

Tennis expertise reduces costs in cognition but not in motor skills in a cognitive-motor dual-task condition

Dual-process theories predict performance reductions under dual-task situations (= situations where two tasks have to be processed and executed simultaneously), because limited cognitive resources have to be shared between concurrent tasks. Increases in expertise should reduce the attentional resources needed to perform a motor task, leading to reduced dual-task costs. The current studies investigated whether expert tennis players (performance ratings of 1 to 14 in the German system) show smaller costs compared to intermediate players (performance ratings of 15 to 23). Two studies assessed single- and dual-task performance in a within-subject design in the same tennis task, returning balls into a target field. Two different cognitive tasks were used, a 3-back working memory task in study 1, and a vocabulary-learning task (episodic memory) in study 2. As predicted, performance in both cognitive tasks was reduced during dual-tasking, while the accuracy of tennis returns remained stable under cognitive challenge. These findings indicate that skilled tennis players show a task-prioritization strategy in favor of the tennis task in a dual-task situation. In study 1, intermediate players showed higher overall dual-task costs than experts, but the group differences in dual-task costs did not reach significance in study 2. This may have been due to less pronounced expertise-differences between the groups in study 2. The findings replicate and extend previous expertise studies in sports to the domain of tennis. We argue that an athlete's ability to keep up cognitive and motor performances in challenging dual-task situations may be a valid indicator of skill level

Testing the construct validity of a soccer-specific virtual reality simulator using novice, academy and professional soccer players

Virtual Reality (VR) provides the potential for immersive and engaging training solutions for improving sport performance. However, if VR training is to be adopted and used in an effective and evidence-based fashion, a more rigorous assessment of the validity of the simulation is required. Construct validity is the degree to which the simulation provides an accurate representation of core features of the task. In the context of sport, if the training drills in the VR environment are a true representation of the skills needed in the real world, then those that excel at the sport in the real world should also excel in the virtual one. In this experiment, we examined the construct validity of a soccer-specific VR simulator by recruiting professional, academy, and novice players. Seventeen participants in each group completed four VR soccer drills and the VR software provided scores relating to performance and process (e.g., passing accuracy, composure, reaction time, and adaptability). Based on these scores, an algorithm gave a diagnostic score relating to the predicted ability of the player. Results showed that this VR platform successfully differentiated between participants of differing skill levels. These results provide some support for the construct validity of this VR simulator and suggest at least partial overlap between the perceptual-cognitive and motor skills needed to perform well across ‘real’ and virtual environments. Further work is needed to explore the validity and fidelity of the simulation before its adoption as a training device can be fully endorsed

Embodied Cognition and Sport

Successful athletic performance requires precision in many respects. A batter stands behind home plate awaiting the arrival of a ball that is less than three inches in diameter and moving close to 100 mph. His goal is to hit it with a ba­­t that is also less than three inches in diameter. This impressive feat requires extraordinary temporal and spatial coordination. The sweet spot of the bat must be at the same place, at the same time, as the ball. A basketball player must keep a ball bouncing as she speeds from one end of the court to another, evading defensive players. She may never break pace as she lifts from the ground, throwing the ball fifteen feet toward a hoop that is eighteen inches in diameter. One task facing a psychologist involves explaining how the body does such things within the sometimes very demanding spatial and temporal constraints that a given task imposes. Part of the goal of this chapter is to sketch the commitments of an embodied approach to such an explanation. We shall see that an embodied account of motor skills draws concepts that depart radically from more traditional cognitivist theories of motor activity. Similarly, because an embodied approach to cognition introduces new ways to understand the human capacity for social interaction, it also promises to shed new light on how athletes coordinate their actions with each other

Non-Monotonic Reasoning on Board a Sony AIBO

Griffith Sciences, School of Information and Communication TechnologyFull Tex

Probabilistic Human-Robot Information Fusion

This thesis is concerned with combining the perceptual abilities of mobile robots and human operators to execute tasks cooperatively. It is generally agreed that a synergy of human and robotic skills offers an opportunity to enhance the capabilities of today’s robotic systems, while also increasing their robustness and reliability. Systems which incorporate both human and robotic information sources have the potential to build complex world models, essential for both automated and human decision making. In this work, humans and robots are regarded as equal team members who interact and communicate on a peer-to-peer basis. Human-robot communication is addressed using probabilistic representations common in robotics. While communication can in general be bidirectional, this work focuses primarily on human-to-robot information flow. More specifically, the approach advocated in this thesis is to let robots fuse their sensor observations with observations obtained from human operators. While robotic perception is well-suited for lower level world descriptions such as geometric properties, humans are able to contribute perceptual information on higher abstraction levels. Human input is translated into the machine representation via Human Sensor Models. A common mathematical framework for humans and robots reinforces the notion of true peer-to-peer interaction. Human-robot information fusion is demonstrated in two application domains: (1) scalable information gathering, and (2) cooperative decision making. Scalable information gathering is experimentally demonstrated on a system comprised of a ground vehicle, an unmanned air vehicle, and two human operators in a natural environment. Information from humans and robots was fused in a fully decentralised manner to build a shared environment representation on multiple abstraction levels. Results are presented in the form of information exchange patterns, qualitatively demonstrating the benefits of human-robot information fusion. The second application domain adds decision making to the human-robot task. Rational decisions are made based on the robots’ current beliefs which are generated by fusing human and robotic observations. Since humans are considered a valuable resource in this context, operators are only queried for input when the expected benefit of an observation exceeds the cost of obtaining it. The system can be seen as adjusting its autonomy at run-time based on the uncertainty in the robots’ beliefs. A navigation task is used to demonstrate the adjustable autonomy system experimentally. Results from two experiments are reported: a quantitative evaluation of human-robot team effectiveness, and a user study to compare the system to classical teleoperation. Results show the superiority of the system with respect to performance, operator workload, and usability

Intuitive Human-Machine Interfaces for Non-Anthropomorphic Robotic Hands

As robots become more prevalent in our everyday lives, both in our workplaces and in our homes, it becomes increasingly likely that people who are not experts in robotics will be asked to interface with robotic devices. It is therefore important to develop robotic controls that are intuitive and easy for novices to use. Robotic hands, in particular, are very useful, but their high dimensionality makes creating intuitive human-machine interfaces for them complex. In this dissertation, we study the control of non-anthropomorphic robotic hands by non-roboticists in two contexts: collaborative manipulation and assistive robotics. In the field of collaborative manipulation, the human and the robot work side by side as independent agents. Teleoperation allows the human to assist the robot when autonomous grasping is not able to deal sufficiently well with corner cases or cannot operate fast enough. Using the teleoperator’s hand as an input device can provide an intuitive control method, but finding a mapping between a human hand and a non-anthropomorphic robot hand can be difficult, due to the hands’ dissimilar kinematics. In this dissertation, we seek to create a mapping between the human hand and a fully actuated, non-anthropomorphic robot hand that is intuitive enough to enable effective real-time teleoperation, even for novice users. We propose a low-dimensional and continuous teleoperation subspace which can be used as an intermediary for mapping between different hand pose spaces. We first propose the general concept of the subspace, its properties and the variables needed to map from the human hand to a robot hand. We then propose three ways to populate the teleoperation subspace mapping. Two of our mappings use a dataglove to harvest information about the user's hand. We define the mapping between joint space and teleoperation subspace with an empirical definition, which requires a person to define hand motions in an intuitive, hand-specific way, and with an algorithmic definition, which is kinematically independent, and uses objects to define the subspace. Our third mapping for the teleoperation subspace uses forearm electromyography (EMG) as a control input. Assistive orthotics is another area of robotics where human-machine interfaces are critical, since, in this field, the robot is attached to the hand of the human user. In this case, the goal is for the robot to assist the human with movements they would not otherwise be able to achieve. Orthotics can improve the quality of life of people who do not have full use of their hands. Human-machine interfaces for assistive hand orthotics that use EMG signals from the affected forearm as input are intuitive and repeated use can strengthen the muscles of the user's affected arm. In this dissertation, we seek to create an EMG based control for an orthotic device used by people who have had a stroke. We would like our control to enable functional motions when used in conjunction with a orthosis and to be robust to changes in the input signal. We propose a control for a wearable hand orthosis which uses an easy to don, commodity forearm EMG band. We develop an supervised algorithm to detect a user’s intent to open and close their hand, and pair this algorithm with a training protocol which makes our intent detection robust to changes in the input signal. We show that this algorithm, when used in conjunction with an orthosis over several weeks, can improve distal function in users. Additionally, we propose two semi-supervised intent detection algorithms designed to keep our control robust to changes in the input data while reducing the length and frequency of our training protocol

The efficacy of virtual reality in professional soccer

Professional soccer clubs have taken an interest to virtual reality, however, only a paucity of evidence exists to support its use in the soccer training ground environment. Further, several soccer virtual reality companies have begun providing solutions to teams, claiming to test specific characteristics of players, yet supportive evidence for certain measurement properties remain absent from the literature. The aims of this thesis were to explore the efficacy of virtual reality being used in the professional football training ground environment. To do so, this thesis looked to explore the fundamental measurement properties of soccer specific virtual reality tests, along with the perceptions of professional coaches, backroom staff, and players that could use virtual reality. The first research study (Chapter 3) aimed to quantify the learning effect during familiarisation trials of a soccer-specific virtual reality task. Thirty-four professional soccer players age, stature, and body mass: mean (SD) 20 (3.4) years; 180 (7) cm; 79 (8) kg, participated in six trials of a virtual reality soccer passing task. The task required participants to receive and pass 30 virtual soccer balls into highlighted mini-goals that surrounded the participant. The number of successful passes were recorded in each trial. The one-sided Bayesian paired samples t-test indicated very strong evidence in favour of the alternative hypothesis (H1)(BF10 = 46.5, d = 0.56 [95% CI = 0.2 to 0.92]) for improvements in total goals scored between trial 1: 13.6 (3.3) and trial 2: 16 (3.3). Further, the Bayesian paired-samples equivalence t-tests indicated strong evidence in favour of H1 (BF10 = 10.2, d = 0.24 [95% CI = -0.09 to 0.57]) for equivalence between trial 4: 16.7 (3.7) and trial 5: 18.2 (4.7); extreme evidence in favour of H1 (BF10 = 132, d = -0.02 [95% CI = -0.34 to 0.30]) for equivalence between trials 5 and 6: 18.1 (3.5); and moderate evidence in favour of H1 (BF10 = 8.4, d = 0.26 [95% CI = -0.08 to 0.59]) for equivalence between trials 4 and 6. Sufficient evidence indicated that a learning effect took place between the first two trials, and that up to five trials might be necessary for performance to plateau in a specific virtual reality soccer passing task.The second research study (Chapter 4) aimed to assess the validity of a soccer passing task by comparing passing ability between virtual reality and real-world conditions. A previously validated soccer passing test was replicated into a virtual reality environment. Twenty-nine soccer players participated in the study which required them to complete as many passes as possible between two rebound boards within 45 s. Counterbalancing determined the condition order, and then for each condition, participants completed four familiarisation trials and two recorded trials, with the best score being used for analysis. Sense of presence and fidelity were also assessed via questionnaires to understand how representative the virtual environments were compared to the real-world. Results showed that between conditions a difference was observed (EMM = -3.9, 95% HDI = -5.1 to -2.7) with the number of passes being greater in the real-world (EMM = 19.7, 95% HDI = 18.6 to 20.7) than in virtual reality (EMM = 15.7, 95% HDI = 14.7 to 16.8). Further, several subjective differences for fidelity between the two conditions were reported, notably the ability to control the ball in virtual reality which was suggested to have been more difficult than in the real-world. The last research study (Chapter 5) aimed to compare and quantify the perceptions of virtual reality use in soccer, and to model behavioural intentions to use this technology. This study surveyed the perceptions of coaches, support staff, and players in relation to their knowledge, expectations, influences, and barriers of using virtual reality via an internet-based questionnaire. To model behavioural intention, modified questions and constructs from the Unified Theory of Acceptance and Use of Technology were used, and the model was analysed through partial least squares structural equation modelling. Respondents represented coaches and support staff (n = 134) and players (n = 64). All respondents generally agreed that virtual reality should be used to improve tactical awareness and cognition, with its use primarily in performance analysis and rehabilitation settings. Generally, coaches and support staff agreed that monetary cost, coach buy-in and limited evidence base were barriers towards its use. In a sub-sample of coaches and support staff without access to virtual reality (n = 123), performance expectancy was the strongest construct in explaining behavioural intention to use virtual reality, followed by facilitating conditions (i.e., barriers) construct which had a negative association with behavioural intention. This thesis aimed to explore the measurement properties of soccer specific virtual reality tests, and the perceptions of staff and players who might use the technology. The key findings from exploring the measurement properties were (1) evidence of a learning curve, suggesting the need for multiple familiarisation trials before collecting data, and (2) a lack of evidence to support the validity of a virtual reality soccer passing test as evident by a lack of agreement to a real-world equivalent. This finding raises questions on the suitability for virtual reality being used to measure passing skill related performance. The key findings from investigating the perceptions of users included, using the technology to improve cognition and tactical awareness, and using it in rehabilitation and performance analysis settings. Future intention to use was generally positive, and driven by performance related factors, yet several barriers exist that may prevent its widespread use. In Chapter 7 of the thesis, a reflective account is presented for the reader, detailing some of the interactions made with coaches, support staff and players in relation to the personal, moral, and ethical challenges faced as a practitioner-researcher, working and studying, respectively, in a professional soccer club