An Architecture for Online Affordance-based Perception and Whole-body Planning

The DARPA Robotics Challenge Trials held in December 2013 provided a landmark demonstration of dexterous mobile robots executing a variety of tasks aided by a remote human operator using only data from the robot's sensor suite transmitted over a constrained, field-realistic communications link. We describe the design considerations, architecture, implementation and performance of the software that Team MIT developed to command and control an Atlas humanoid robot. Our design emphasized human interaction with an efficient motion planner, where operators expressed desired robot actions in terms of affordances fit using perception and manipulated in a custom user interface. We highlight several important lessons we learned while developing our system on a highly compressed schedule

DARPA Robotics Challenge

In this paper we discuss Worcester Polytechnic Institute(WPI) and Carnegie Mellon University(CMU) team approach for competing on the DARPA Robotics Challenge (DRC) using the humanoid Boston Dynamics Atlas robot. An overview and analysis of the hardware and software architecture is described with emphasis on two of the challenges tasks, Wall and Drill. The realization of a double stance Inverse Kinematics(IK) full-body controller used for manipulation and its overall performance is given. Moreover, an analysis for using Dynamic Programming optimization for robotic humanoid path planning is developed

Importance and applications of robotic and autonomous systems (RAS) in railway maintenance sector: a review

Maintenance, which is critical for safe, reliable, quality, and cost-effective service, plays a dominant role in the railway industry. Therefore, this paper examines the importance and applications of Robotic and Autonomous Systems (RAS) in railway maintenance. More than 70 research publications, which are either in practice or under investigation describing RAS developments in the railway maintenance, are analysed. It has been found that the majority of RAS developed are for rolling-stock maintenance, followed by railway track maintenance. Further, it has been found that there is growing interest and demand for robotics and autonomous systems in the railway maintenance sector, which is largely due to the increased competition, rapid expansion and ever-increasing expense

An Object Template Approach to Manipulation for Semi-autonomous Avatar Robots

Nowadays, the first steps towards the use of mobile robots to perform manipulation tasks in remote environments have been made possible. This opens new possibilities for research and development, since robots can help humans to perform tasks in many scenarios. A remote robot can be used as avatar in applications such as for medical or industrial use, in rescue and disaster recovery tasks which might be hazardous environments for human beings to enter, as well as for more distant scenarios like planetary explorations. Among the most typical applications in recent years, research towards the deployment of robots to mitigate disaster scenarios has been of great interest in the robotics field. Disaster scenarios present challenges that need to be tackled. Their unstructured nature makes them difficult to predict and even though some assumptions can be made for human-designed scenarios, there is no certainty on the expected conditions. Communications with a robot inside these scenarios might also be challenged; wired communications limit reachability and wireless communications are limited by bandwidth. Despite the great progress in the robotics research field, these difficulties have prevented the current autonomous robotic approaches to perform efficiently in unstructured remote scenarios. On one side, acquiring physical and abstract information from unknown objects in a full autonomous way in uncontrolled environmental conditions is still an unsolved problem. Several challenges have to be overcome such as object recognition, grasp planning, manipulation, and mission planning among others. On the other side, purely teleoperated robots require a reliable communication link robust to reachability, bandwidth, and latency which can provide all the necessary feedback that a human operator needs in order to achieve sufficiently good situational awareness, e.g., worldmodel, robot state, forces, and torques exerted. Processing this amount of information plus the necessary training to perform joint motions with the robot represent a high mental workload for the operator which results in very low execution times. Additionally, a pure teleoperated approach is error-prone given that the success in a manipulation task strongly depends on the ability and expertise of the human operating the robot. Both, autonomous and teleoperated robotic approaches have pros and cons, for this reason a middle ground approach has emerged. In an approach where a human supervises a semi-autonomous remote robot, strengths from both, full autonomous and purely teleoperated approaches can be combined while at the same time their weaknesses can be tackled. A remote manipulation task can be divided into sub-tasks such as planning, perception, action, and evaluation. A proper distribution of these sub-tasks between the human operator and the remote robot can increase the efficiency and potential of success in a manipulation task. On the one hand, a human operator can trivially plan a task (planning), identify objects in the sensor data acquired by the robot (perception), and verify the completion of a task (evaluation). On the other hand, it is challenging to remotely control in joint space a robotic system like a humanoid robot that can easily have over 25 degrees of freedom (DOF). For this reason, in this approach the complex sub-tasks such as motion planning, motion execution, and obstacle avoidance (action) are performed autonomously by the remote robot. With this distribution of tasks, the challenge of converting the operator intent into a robot action arises. This thesis investigates concepts of how to efficiently provide a remote robot with the operator intent in a flexible means of interaction. While current approaches focus on an object-grasp-centered means of interaction, this thesis aims at providing physical and abstract properties of the objects of interest. With this information, the robot can perform autonomous subtasks like locomotion through the environment, grasping objects, and manipulating them at an affordance-level avoiding collisions with the environment in order to efficiently accomplish the manipulation task needed. For this purpose, the concept of Object Template (OT) has been developed in this thesis. An OT is a virtual representation of an object of interest that contains information that a remote robot can use to manipulate such object or other similar objects. The object template concept presented here goes beyond state-of-the-art related concepts by extending the robot capabilities to use affordance information of the object. This concept includes physical information (mass, center of mass, inertia tensor) as well as abstract information (potential grasps, affordances, and usabilities). Because humans are very good at analysing a situation, planning new ways of how to solve a task, even using objects for different purposes, it is important to allow communicating the planning and perception performed by the operator such that the robot can execute the action based on the information contained in the OT. This leverages human intelligence with robot capabilities. For example, as an implementation in a 3D environment, an OT can be visualized as a 3D geometry mesh that simulates an object of interest. A human operator can manipulate the OT and move it so that it overlaps with the visualized sensor data of the real object. Information of the object template type and its pose can be compressed and sent using low bandwidth communication. Then, the remote robot can use the information of the OT to approach, grasp, and manipulate the real object. The use of remote humanoid robots as avatars is expected to be intuitive to operators (or potential human response forces) since the kinematic chains and degrees of freedom are similar to humans. This allows operators to visualize themselves in the remote environment and think how to solve a task, however, task requirements such as special tools might not be found. For this reason, a flexible means of interaction that can account for allowing improvisation from the operator is also needed. In this approach, improvisation is described as "a change of a plan on how to achieve a certain task, depending on the current situation". A human operator can then improvise by adapting the affordances of known objects into new unknown objects. For example, by utilizing the affordances defined in an OT on a new object that has similar physical properties or which manipulation skills belong to the same class. The experimental results presented in this thesis validate the proposed approach by demonstrating the successful achievement of several manipulation tasks using object templates. Systematic laboratory experimentation has been performed to evaluate the individual aspects of this approach. The performance of the approach has been tested in three different humanoid robotic systems (one of these robots belongs to another research laboratory). These three robotic platforms also participated in the renowned international competition DARPA Robotics Challenge (DRC) which between 2012 and 2015 was considered the most ambitious and challenging robotic competition

Negotiating Large Obstacles with a Humanoid Robot via Multi-Contact Motion Planning

Incremental progress in humanoid robot locomotion over the years has achieved essential capabilities such as navigation over at or uneven terrain, stepping over small obstacles and imbing stairls. However, the locomotion research has mostly been limited to using only bipedal gait and only foot contacts with the environment, using the upper body for balancing without considering additional external contacts. As a result, challenging locomotion tasks like climbing over large obstacles relative to the size of the robot have remained unsolved. In this paper, we address this class of open problems with an approach based on multi-contact motion planning, guided by physical human demonstrations. Our goal is to make humanoid locomotion problem more tractable by taking advantage of objects in the surrounding environment instead of avoiding them. We propose a multi-contact motion planning algorithm for humanoid robot locomotion which exploits the multi-contacts at the upper and lower body limbs. We propose a contact stability measure, which simplies the contact search from demonstration and contact transition motion generation for the multi-contact motion planning algorithm. The algorithm uses the whole-body motions generated via Quadratic Programming (QP) based solver methods. The multi-contact motion planning algorithm is applied for a challenging task of climbing over a relatively larger obstacle compared to the robot. We validate our planning approach with simulations and experiments for climbing over a large wooden obstacle with COMAN, which is a complaint humanoid robot with 23 degrees of freedom (DOF). We also propose a generalization method, the \Policy-Contraction Learning Method" to extend the algorithm for generating new multi-contact plans for our multi-contact motion planner, that can adapt to changes in the environment. The method learns a general policy and the multi-contact behavior from the human demonstrations, for generating new multi-contact plans for the obstacle-negotiation

Human-Robot Collaboration Enabled By Real-Time Vision Tracking

The number of robotic systems in the world is growing rapidly. However, most industrial robots are isolated in caged environments for the safety of users. There is an urgent need for human-in-the-loop collaborative robotic systems since robots are very good at performing precise and repetitive tasks but lack the cognitive ability and soft skills of humans. To fill this need, a key challenge is how to enable a robot to interpret its human co-worker’s motion and intention. This research addresses this challenge by developing a collaborative human-robot interface via innovations in computer vision, robotics, and system integration techniques. Specifically, this work integrates a holistic framework of cameras, motion sensors, and a 7-degree-of-freedom robotic manipulator controlled by vision data processing and motion planning algorithms implemented in the open-source robotics middleware Robot Operating System (ROS)

Real-time Control of Robot Arm Based on Hand Tracking Using Leap Motion Sensor Technology

Leap Motion is an example of ground-breaking technology that has the potential to change the way we control machines and therefore, how we control our world! Leap Motion is a sensor that is currently used to navigate through a personal computer with just hand gestures. For our MQP, the team has used this technology to control a physical robot arm. For the second part of this project, the team used SolidWorks to design a six-degree-of–freedom robot arm with five human-like fingers. The robot was designed to be controlled by Leap Motion, with human hand gestures as the input. It utilizes all of the sensor’s features, including the simultaneous control of all five fingers. The robot could be used for virtually any application, including service in the medical or military fields

Scaled Autonomy for Networked Humanoids

Humanoid robots have been developed with the intention of aiding in environments designed for humans. As such, the control of humanoid morphology and effectiveness of human robot interaction form the two principal research issues for deploying these robots in the real world. In this thesis work, the issue of humanoid control is coupled with human robot interaction under the framework of scaled autonomy, where the human and robot exchange levels of control depending on the environment and task at hand. This scaled autonomy is approached with control algorithms for reactive stabilization of human commands and planned trajectories that encode semantically meaningful motion preferences in a sequential convex optimization framework. The control and planning algorithms have been extensively tested in the field for robustness and system verification. The RoboCup competition provides a benchmark competition for autonomous agents that are trained with a human supervisor. The kid-sized and adult-sized humanoid robots coordinate over a noisy network in a known environment with adversarial opponents, and the software and routines in this work allowed for five consecutive championships. Furthermore, the motion planning and user interfaces developed in the work have been tested in the noisy network of the DARPA Robotics Challenge (DRC) Trials and Finals in an unknown environment. Overall, the ability to extend simplified locomotion models to aid in semi-autonomous manipulation allows untrained humans to operate complex, high dimensional robots. This represents another step in the path to deploying humanoids in the real world, based on the low dimensional motion abstractions and proven performance in real world tasks like RoboCup and the DRC