Knowing when to assist: Developmental issues in lifelong assistive robotics

Children and adults with sensorimotor disabilities can significantly increase their autonomy through the use of assistive robots. As the field progresses from short-term, task-specific solutions to long-term, adaptive ones, new challenges are emerging. In this paper a lifelong methodological approach is presented, that attempts to balance the immediate context-specific needs of the user, with the long-term effects that the robots assistance can potentially have on the users developmental trajectory

Anticipation in Human-Robot Cooperation: A Recurrent Neural Network Approach for Multiple Action Sequences Prediction

Close human-robot cooperation is a key enabler for new developments in advanced manufacturing and assistive applications. Close cooperation require robots that can predict human actions and intent, and understand human non-verbal cues. Recent approaches based on neural networks have led to encouraging results in the human action prediction problem both in continuous and discrete spaces. Our approach extends the research in this direction. Our contributions are three-fold. First, we validate the use of gaze and body pose cues as a means of predicting human action through a feature selection method. Next, we address two shortcomings of existing literature: predicting multiple and variable-length action sequences. This is achieved by introducing an encoder-decoder recurrent neural network topology in the discrete action prediction problem. In addition, we theoretically demonstrate the importance of predicting multiple action sequences as a means of estimating the stochastic reward in a human robot cooperation scenario. Finally, we show the ability to effectively train the prediction model on a action prediction dataset, involving human motion data, and explore the influence of the model's parameters on its performance. Source code repository: https://github.com/pschydlo/ActionAnticipationComment: IEEE International Conference on Robotics and Automation (ICRA) 2018, Accepte

Rebellion and Obedience: The Effects of Intention Prediction in Cooperative Handheld Robots

Within this work, we explore intention inference for user actions in the context of a handheld robot setup. Handheld robots share the shape and properties of handheld tools while being able to process task information and aid manipulation. Here, we propose an intention prediction model to enhance cooperative task solving. The model derives intention from the user's gaze pattern which is captured using a robot-mounted remote eye tracker. The proposed model yields real-time capabilities and reliable accuracy up to 1.5s prior to predicted actions being executed. We assess the model in an assisted pick and place task and show how the robot's intention obedience or rebellion affects the cooperation with the robot.Comment: submitted to iROS 2019. arXiv admin note: substantial text overlap with arXiv:1810.0646

Explainable shared control in assistive robotics

Shared control plays a pivotal role in designing assistive robots to complement human capabilities during everyday tasks. However, traditional shared control relies on users forming an accurate mental model of expected robot behaviour. Without this accurate mental image, users may encounter confusion or frustration whenever their actions do not elicit the intended system response, forming a misalignment between the respective internal models of the robot and human. The Explainable Shared Control paradigm introduced in this thesis attempts to resolve such model misalignment by jointly considering assistance and transparency. There are two perspectives of transparency to Explainable Shared Control: the human's and the robot's. Augmented reality is presented as an integral component that addresses the human viewpoint by visually unveiling the robot's internal mechanisms. Whilst the robot perspective requires an awareness of human "intent", and so a clustering framework composed of a deep generative model is developed for human intention inference. Both transparency constructs are implemented atop a real assistive robotic wheelchair and tested with human users. An augmented reality headset is incorporated into the robotic wheelchair and different interface options are evaluated across two user studies to explore their influence on mental model accuracy. Experimental results indicate that this setup facilitates transparent assistance by improving recovery times from adverse events associated with model misalignment. As for human intention inference, the clustering framework is applied to a dataset collected from users operating the robotic wheelchair. Findings from this experiment demonstrate that the learnt clusters are interpretable and meaningful representations of human intent. This thesis serves as a first step in the interdisciplinary area of Explainable Shared Control. The contributions to shared control, augmented reality and representation learning contained within this thesis are likely to help future research advance the proposed paradigm, and thus bolster the prevalence of assistive robots.Open Acces

Opportunities for using eye tracking technology in manufacturing and logistics: Systematic literature review and research agenda

Workers play essential roles in manufacturing and logistics. Releasing workers from routine tasks and enabling them to focus on creative, value-adding activities can enhance their performance and wellbeing, and it is also key to the successful implementation of Industry 4.0. One technology that can help identify patterns of worker-system interaction is Eye Tracking (ET), which is a non-intrusive technology for measuring human eye movements. ET can provide moment-by-moment insights into the cognitive state of the subject during task execution, which can improve our understanding of how humans behave and make decisions within complex systems. It also enables explorations of the subject’s interaction mode with the working environment. Earlier research has investigated the use of ET in manufacturing and logistics, but the literature is fragmented and has not yet been discussed in a literature review yet. This article therefore conducts a systematic literature review to explore the applications of ET, summarise its benefits, and outline future research opportunities of using ET in manufacturing and logistics. We first propose a conceptual framework to guide our study and then conduct a systematic literature search in scholarly databases, obtaining 71 relevant papers. Building on the proposed framework, we systematically review the use of ET and categorize the identified papers according to their application in manufacturing (product development, production, quality inspection) and logistics. Our results reveal that ET has several use cases in the manufacturing sector, but that its application in logistics has not been studied extensively so far. We summarize the benefits of using ET in terms of process performance, human performance, and work environment and safety, and also discuss the methodological characteristics of the ET literature as well as typical ET measures used. We conclude by illustrating future avenues for ET research in manufacturing and logistics