World Robot Challenge 2020 -- Partner Robot: A Data-Driven Approach for Room Tidying with Mobile Manipulator

Tidying up a household environment using a mobile manipulator poses various challenges in robotics, such as adaptation to large real-world environmental variations, and safe and robust deployment in the presence of humans.The Partner Robot Challenge in World Robot Challenge (WRC) 2020, a global competition held in September 2021, benchmarked tidying tasks in the real home environments, and importantly, tested for full system performances.For this challenge, we developed an entire household service robot system, which leverages a data-driven approach to adapt to numerous edge cases that occur during the execution, instead of classical manual pre-programmed solutions. In this paper, we describe the core ingredients of the proposed robot system, including visual recognition, object manipulation, and motion planning. Our robot system won the second prize, verifying the effectiveness and potential of data-driven robot systems for mobile manipulation in home environments

Investigation on the mobile robot navigation in an unknown environment

Mobile robots could be used to search, find, and relocate objects in many types of manufacturing operations and environments. In this scenario, the target objects might reside with equal probability at any location in the environment and, therefore, the robot must navigate and search the whole area autonomously, and be equipped with specific sensors to detect objects. Novel challenges exist in developing a control system, which helps a mobile robot achieve such tasks, including constructing enhanced systems for navigation, and vision-based object recognition. The latter is important for undertaking the exploration task that requires an optimal object recognition technique. In this thesis, these challenges, for an indoor environment, were divided into three sub-problems. In the first, the navigation task involved discovering an appropriate exploration path for the entire environment, with minimal sensing requirements. The Bug algorithm strategies were adapted for modelling the environment and implementing the exploration path. The second was a visual-search process, which consisted of employing appropriate image-processing techniques, and choosing a suitable viewpoint field for the camera. This study placed more emphasis on colour segmentation, template matching and Speeded-Up Robust Features (SURF) for object detection. The third problem was the relocating process, which involved using a robot’s gripper to grasp the detected, desired object and then move it to the assigned, final location. This also included approaching both the target and the delivery site, using a visual tracking technique. All codes were developed using C++ and C programming, and some libraries that included OpenCV and OpenSURF were utilized for image processing. Each control system function was tested both separately, and then in combination as a whole control program. The system performance was evaluated using two types of mobile robots: legged and wheeled. In this study, it was necessary to develop a wheeled search robot with a high performance processor. The experimental results demonstrated that the methodology used for the search robots was highly efficient provided the processor was adequate. It was concluded that it is possible to implement a navigation system within a minimum number of sensors if they are located and used effectively on the robot’s body. The main challenge within a visual-search process is that the environmental conditions are difficult to control, because the search robot executes its tasks in dynamic environments. The additional challenges of scaling these small robots up to useful industrial capabilities were also explored

Towards safe human-to-robot handovers of unknown containers

Safe human-to-robot handovers of unknown objects require accurate estimation of hand poses and object properties, such as shape, trajectory, and weight. Accurately estimating these properties requires the use of scanned 3D object models or expensive equipment, such as motion capture systems and markers, or both. However, testing handover algorithms with robots may be dangerous for the human and, when the object is an open container with liquids, for the robot. In this paper, we propose a real-to-simulation framework to develop safe human-to-robot handovers with estimations of the physical properties of unknown cups or drinking glasses and estimations of the human hands from videos of a human manipulating the container. We complete the handover in simulation, and we estimate a region that is not occluded by the hand of the human holding the container. We also quantify the safeness of the human and object in simulation. We validate the framework using public recordings of containers manipulated before a handover and show the safeness of the handover when using noisy estimates from a range of perceptual algorithms

Towards Robust Autonomous Grasping with Reflexes Using High-Bandwidth Sensing and Actuation

Modern robotic manipulation systems fall short of human manipulation skills partly because they rely on closing feedback loops exclusively around vision data, which reduces system bandwidth and speed. By developing autonomous grasping reflexes that rely on high-bandwidth force, contact, and proximity data, the overall system speed and robustness can be increased while reducing reliance on vision data. We are developing a new system built around a low-inertia, high-speed arm with nimble fingers that combines a high-level trajectory planner operating at less than 1 Hz with low-level autonomous reflex controllers running upwards of 300 Hz. We characterize the reflex system by comparing the volume of the set of successful grasps for a naive baseline controller and variations of our reflexive grasping controller, finding that our controller expands the set of successful grasps by 55% relative to the baseline. We also deploy our reflexive grasping controller with a simple vision-based planner in an autonomous clutter clearing task, achieving a grasp success rate above 90% while clearing over 100 items.Comment: 6 pages, 1 page of references, supplementary video at https://youtu.be/f8Coo02Jvdg. Submitted to ICRA 202