A review on manipulation skill acquisition through teleoperation-based learning from demonstration

Manipulation skill learning and generalization have gained increasing attention due to the wide applications of robot manipulators and the spurt of robot learning techniques. Especially, the learning from demonstration method has been exploited widely and successfully in the robotic community, and it is regarded as a promising direction to realize the manipulation skill learning and generalization. In addition to the learning techniques, the immersive teleoperation enables the human to operate a remote robot with an intuitive interface and achieve the telepresence. Thus, it is a promising way to transfer manipulation skills from humans to robots by combining the learning methods and the teleoperation, and adapting the learned skills to different tasks in new situations. This review, therefore, aims to provide an overview of immersive teleoperation for skill learning and generalization to deal with complex manipulation tasks. To this end, the key technologies, e.g. manipulation skill learning, multimodal interfacing for teleoperation and telerobotic control, are introduced. Then, an overview is given in terms of the most important applications of immersive teleoperation platform for robot skill learning. Finally, this survey discusses the remaining open challenges and promising research topics

WEHST: Wearable Engine for Human-Mediated Telepresence

This dissertation reports on the industrial design of a wearable computational device created to enable better emergency medical intervention for situations where electronic remote assistance is necessary. The design created for this doctoral project, which assists practices by paramedics with mandates for search-and-rescue (SAR) in hazardous environments, contributes to the field of human-mediated teleparamedicine (HMTPM). Ethnographic and industrial design aspects of this research considered the intricate relationships at play in search-and-rescue operations, which lead to the design of the system created for this project known as WEHST: Wearable Engine for Human-Mediated Telepresence. Three case studies of different teams were carried out, each focusing on making improvements to the practices of teams of paramedics and search-and-rescue technicians who use combinations of ambulance, airplane, and helicopter transport in specific chemical, biological, radioactive, nuclear and explosive (CBRNE) scenarios. The three paramedicine groups included are the Canadian Air Force 442 Rescue Squadron, Nelson Search and Rescue, and the British Columbia Ambulance Service Infant Transport Team. Data was gathered over a seven-year period through a variety of methods including observation, interviews, examination of documents, and industrial design. The data collected included physiological, social, technical, and ecological information about the rescuers. Actor-network theory guided the research design, data analysis, and design synthesis. All of this leads to the creation of the WEHST system. As identified, the WEHST design created in this dissertation project addresses the difficulty case-study participants found in using their radios in hazardous settings. As the research identified, a means of controlling these radios without depending on hands, voice, or speech would greatly improve communication, as would wearing sensors and other computing resources better linking operators, radios, and environments. WEHST responds to this need. WEHST is an instance of industrial design for a wearable “engine” for human-situated telepresence that includes eight interoperable families of wearable electronic modules and accompanying textiles. These make up a platform technology for modular, scalable and adaptable toolsets for field practice, pedagogy, or research. This document details the considerations that went into the creation of the WEHST design

A Broad View on Robot Self-Defense: Rapid Scoping Review and Cultural Comparison

With power comes responsibility: as robots become more advanced and prevalent, the role they will play in human society becomes increasingly important. Given that violence is an important problem, the question emerges if robots could defend people, even if doing so might cause harm to someone. The current study explores the broad context of how people perceive the acceptability of such robot self-defense (RSD) in terms of (1) theory, via a rapid scoping review, and (2) public opinion in two countries. As a result, we summarize and discuss: increasing usage of robots capable of wielding force by law enforcement and military, negativity toward robots, ethics and legal questions (including differences to the well-known trolley problem), control in the presence of potential failures, and practical capabilities that such robots might require. Furthermore, a survey was conducted, indicating that participants accepted the idea of RSD, with some cultural differences. We believe that, while substantial obstacles will need to be overcome to realize RSD, society stands to gain from exploring its possibilities over the longer term, toward supporting human well-being in difficult times

Modeling and Improving Teleoperation Performance of Semi-Autonomous Wheeled Robots

Robotics and unmanned vehicles have allowed us to interact with environments in ways that were impossible decades ago. As perception, decision making, and control improve, it becomes possible to automate more parts of robot operation. However, humans will remain a critical part of robot control based on preference, ethical, and technical reasons. An ongoing question will be when and how to pair humans and automation to create semi-autonomous systems. The answer to this question depends on numerous factors such as the robot's task, platform, environment conditions, and the user. The work in this dissertation focuses on modeling the impact of these factors on performance and developing improved semi-autonomous control schemes, so that robot systems can be better designed. Experiments and analysis focus on wheeled robots, however the approach taken and many of the trends could be applied to a variety of platforms. Wheeled robots are often teleoperated over wireless communication networks. While this arrangement may be convenient, it introduces many challenges including time-varying delays and poor perception of the robot's environment that can lead to the robot colliding with objects or rolling over. With regards to semi-autonomous control, rollover prevention and obstacle avoidance behaviors are considered. In this area, two contributions are presented. The first is a rollover prevention method that uses an existing manipulator arm on-board a wheeled robot. The second is a method of approximating convex obstacle free regions for use in optimal control path planning problems. Teleoperation conditions, including communication delays, automation, and environment layout, are considered in modeling robot operation performance. From these considerations stem three contributions. The first is a method of relating driving performance among different communication delay distributions. The second parameterizes how driving through different arrangements of obstacles relates to performance. Lastly, based on user studies, teleoperation performance is related to different conditions of communication delay, automation level, and environment arrangement. The contributions of this dissertation will assist roboticists to implement better automation and understand when to use automation.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/136951/1/jgstorms_1.pd

Robotic Maintenance and ROS - Appearance Based SLAM and Navigation With a Mobile Robot Prototype

Robotic maintenance has been a topic in several master's theses and specialization projects at the Department of Engineering Cybernetics (ITK) at NTNU over many years. This thesis continues on the same topic, with special focus on camera-based mapping and navigation in conjunction with automated maintenance, and automated maintenance in general. The objective of this thesis is to implement one or more functionalities based on camera-based sensors in a mobile autonomous robot. This is accomplished by acquiring knowledge of existing solutions and future requirements within automated maintenance. A mobile robot prototype has been configured to run ROS (Robot Operating System), a middleware framework that is suited to the development of robotic systems. The system uses RTAB-Map (Real-Time Appearance Based Mapping) to survey the surroundings and a built navigation stack in ROS to navigate autonomously against easy targets in the map. The method uses a Kinect for Xbox 360 as the main sensor and a 2D laser scanner to the surveying and odometry. It is also developed functional concepts for two support functions, an Android application for remote control over Bluetooth and a remote central (OCS) developed in Qt. Remote Central is a skeletal implementation that is able to remotely control the robot via WiFi, as well as to display video from the robot's camera. Test results, obtained from both live and simulated trials, indicate that the robot is able to form 3D and 2D map of the surroundings. The method has weaknesses that are related to the ability to find visual features. Laser Based odometry can be tricked when the environment is changing, and when there are few unique features. Further testing has demonstrated that the robot can navigate autonomously, but there is still room for improvement. Better results can be achieved with a new movable platform and further tuning of the system. In conclusion, ROS works well as a development tools for robots, and the current system is suitable for further development. RTAB-Maps suitability for use on an industrial installation is still uncertain and requires further testing

Robot manipulator skill learning and generalising through teleoperation

Robot manipulators have been widely used for simple repetitive, and accurate tasks in industrial plants, such as pick and place, assembly and welding etc., but it is still hard to deploy in human-centred environments for dexterous manipulation tasks, such as medical examination and robot-assisted healthcare. These tasks are not only related to motion planning and control but also to the compliant interaction behaviour of robots, e.g. motion control, force regulation and impedance adaptation simultaneously under dynamic and unknown environments. Recently, with the development of collaborative robotics (cobots) and machine learning, robot skill learning and generalising have attained increasing attention from robotics, machine learning and neuroscience communities. Nevertheless, learning complex and compliant manipulation skills, such as manipulating deformable objects, scanning the human body and folding clothes, is still challenging for robots. On the other hand, teleoperation, also namely remote operation or telerobotics, has been an old research area since 1950, and there have been a number of applications such as space exploration, telemedicine, marine vehicles and emergency response etc. One of its advantages is to combine the precise control of robots with human intelligence to perform dexterous and safety-critical tasks from a distance. In addition, telepresence allows remote operators could feel the actual interaction between the robot and the environment, including the vision, sound and haptic feedback etc. Especially under the development of various augmented reality (AR), virtual reality (VR) and wearable devices, intuitive and immersive teleoperation have received increasing attention from robotics and computer science communities. Thus, various human-robot collaboration (HRC) interfaces based on the above technologies were developed to integrate robot control and telemanipulation by human operators for robot skills learning from human beings. In this context, robot skill learning could benefit teleoperation by automating repetitive and tedious tasks, and teleoperation demonstration and interaction by human teachers also allow the robot to learn progressively and interactively. Therefore, in this dissertation, we study human-robot skill transfer and generalising through intuitive teleoperation interfaces for contact-rich manipulation tasks, including medical examination, manipulating deformable objects, grasping soft objects and composite layup in manufacturing. The introduction, motivation and objectives of this thesis are introduced in Chapter 1. In Chapter 2, a literature review on manipulation skills acquisition through teleoperation is carried out, and the motivation and objectives of this thesis are discussed subsequently. Overall, the main contents of this thesis have three parts: Part 1 (Chapter 3) introduces the development and controller design of teleoperation systems with multimodal feedback, which is the foundation of this project for robot learning from human demonstration and interaction. In Part 2 (Chapters 4, 5, 6 and 7), we studied primitive skill library theory, behaviour tree-based modular method, and perception-enhanced method to improve the generalisation capability of learning from the human demonstrations. And several applications were employed to evaluate the effectiveness of these methods.In Part 3 (Chapter 8), we studied the deep multimodal neural networks to encode the manipulation skill, especially the multimodal perception information. This part conducted physical experiments on robot-assisted ultrasound scanning applications.Chapter 9 summarises the contributions and potential directions of this thesis. Keywords: Learning from demonstration; Teleoperation; Multimodal interface; Human-in-the-loop; Compliant control; Human-robot interaction; Robot-assisted sonography

Bio-Inspired Robotics

Modern robotic technologies have enabled robots to operate in a variety of unstructured and dynamically-changing environments, in addition to traditional structured environments. Robots have, thus, become an important element in our everyday lives. One key approach to develop such intelligent and autonomous robots is to draw inspiration from biological systems. Biological structure, mechanisms, and underlying principles have the potential to provide new ideas to support the improvement of conventional robotic designs and control. Such biological principles usually originate from animal or even plant models, for robots, which can sense, think, walk, swim, crawl, jump or even fly. Thus, it is believed that these bio-inspired methods are becoming increasingly important in the face of complex applications. Bio-inspired robotics is leading to the study of innovative structures and computing with sensory–motor coordination and learning to achieve intelligence, flexibility, stability, and adaptation for emergent robotic applications, such as manipulation, learning, and control. This Special Issue invites original papers of innovative ideas and concepts, new discoveries and improvements, and novel applications and business models relevant to the selected topics of ``Bio-Inspired Robotics''. Bio-Inspired Robotics is a broad topic and an ongoing expanding field. This Special Issue collates 30 papers that address some of the important challenges and opportunities in this broad and expanding field

Robotics 2010

Without a doubt, robotics has made an incredible progress over the last decades. The vision of developing, designing and creating technical systems that help humans to achieve hard and complex tasks, has intelligently led to an incredible variety of solutions. There are barely technical fields that could exhibit more interdisciplinary interconnections like robotics. This fact is generated by highly complex challenges imposed by robotic systems, especially the requirement on intelligent and autonomous operation. This book tries to give an insight into the evolutionary process that takes place in robotics. It provides articles covering a wide range of this exciting area. The progress of technical challenges and concepts may illuminate the relationship between developments that seem to be completely different at first sight. The robotics remains an exciting scientific and engineering field. The community looks optimistically ahead and also looks forward for the future challenges and new development