Design and Motion Planning for a Reconfigurable Robotic Base

A robotic platform for mobile manipulation needs to satisfy two contradicting requirements for many real-world applications: A compact base is required to navigate through cluttered indoor environments, while the support needs to be large enough to prevent tumbling or tip over, especially during fast manipulation operations with heavy payloads or forceful interaction with the environment. This paper proposes a novel robot design that fulfills both requirements through a versatile footprint. It can reconfigure its footprint to a narrow configuration when navigating through tight spaces and to a wide stance when manipulating heavy objects. Furthermore, its triangular configuration allows for high-precision tasks on uneven ground by preventing support switches. A model predictive control strategy is presented that unifies planning and control for simultaneous navigation, reconfiguration, and manipulation. It converts task-space goals into whole-body motion plans for the new robot. The proposed design has been tested extensively with a hardware prototype. The footprint reconfiguration allows to almost completely remove manipulation-induced vibrations. The control strategy proves effective in both lab experiment and during a real-world construction task.Comment: 8 pages, accepted for RA-L and IROS 202

Mobile Manipulation for Industrial Inspection and Building Construction

Industrial robots have revolutionized the manufacturing industries, mass-producing goods at affordable prices in controlled factory environments. In recent years, mobile robots were introduced that can move around unstructured environments like construction sites, but they mostly observe their surrounding and do not interact. This thesis aims to combine the two developments, building mobile manipulating robots that automate tasks in unstructured environments like construction sites or infrastructure plants by interaction. We identify robot motion planning and task-related perception as the key challenges and propose solutions to those problems. Traditionally, mobile manipulators separate the locomotion and the manipulation problem by first driving to a job site and then executing a task with the manipulator. We propose a control framework that combines locomotion and manipulation with whole-body control to solve continuous manipulation tasks that exceed the workspace of a fixed-base manipulator. During task execution, unintended collisions are avoided, and forces at intentional contact are controlled. The framework respects tip-over stability and joint limit constraints and can be deployed on a broad class of mobile manipulators. A mobile manipulator's base is its locomotion system. Its shape creates conflicting operational constraints, such as being small enough to fit through narrow passages or being large enough to ensure stability. We propose a base design that can reconfigure its footprint to satisfy those conflicting requirements. Besides the wide footprint for stability and the narrow footprint for navigation, the robot can also assume a triangular configuration for high-precision manipulation tasks. The robot can switch between the different configurations while it is locomoting using its swerve-steering driving units and no additional actuators. A hardware prototype is built and extensively tested in lab experiments and a field deployment on a construction site. A mobile manipulator has to perceive its environment to interact with it meaningfully. Different sensing modalities like vision and touch have unique advantages and can contribute manipulation-relevant information. We propose two state estimation methods for the two modalities with the same internal belief representation, a particle filter. The first method processes images from a depth camera and infers weights for the measurement update of the particle filter. The update rule is learned from data collected in a simulator and can be used for various problems such as object localization or articulation state estimation. The second method is a contact-based state estimator that leverages the high accuracy of modern manipulators' kinematics to refine the state estimate from a centimeter-level accuracy to the submillimeter level. A reinforcement learning agent decides how a robot should engage with the environment to decrease the uncertainty about its state. The highly accurate state estimate enables the robot to execute a fuse-inertion task with tight tolerances. All proposed methods are put to work in two application studies. First, a mobile manipulator should inspect switchboard cabinets in a train tunnel and replace fuses. The second use case aims to automate the plastering process in the building construction industry

Perceptive Model Predictive Control for Continuous Mobile Manipulation

A mobile robot needs to be aware of its environment to interact with it safely. We propose a receding horizon control scheme for mobile manipulators that tracks task space reference trajectories. It uses visual information to avoid obstacles and haptic sensing to control interaction forces. Additional constraints for mechanical stability and joint limits are met. The proposed method is faster than state of the art sampling based planners, available as opensource and can be implemented on a broad class of robots. We validate the method both in simulation and through extensive hardware experiments with a multitude of mobile manipulation platforms. The resulting software package is released with this paper.ISSN:2377-376

Learning Contact-Based State Estimation for Assembly Tasks

Robotic object manipulation requires knowledge of the environment’s state. In particular, the object poses of fixed elements in the environment relative to the robot and the in-hand poses of grasped objects are of interest. For insertion tasks with tight tolerances, the accuracy of vision systems to estimate the object and in-hand pose is not high enough. This work proposes a state estimation system that delivers precise estimates for both estimation problems. It uses contact detections and the precise forward kinematics that robot arms provide thanks to their high-resolution joint encoders. We propose a reinforcement-learning-based exploration strategy that decides how the robot should engage with the environment to reduce state uncertainty. The system is evaluated in several simulation and hardware experiments. We show that the learned policy can propose meaningful actions for object localization. In hardware experiments with precision-milled objects, sub-millimeter accuracy is achieved for the in-hand pose estimation task. With objects relevant to industrial tasks, i.e., a melting fuse and a fuse box, millimeter-level accuracy can be reached for both in-hand pose estimation and fixed object localization. In an integrated experiment, we show how a robot grasps a fuse, estimates the in-hand pose, and inserts it into a fuse box

Deep Measurement Updates for Bayes Filters

Measurement update rules for Bayes filters often contain hand-crafted heuristics to compute observation probabilities for high-dimensional sensor data, like images. In this work, we propose the novel approach Deep Measurement Update (DMU) as a general update rule for a wide range of systems. DMU has a conditional encoder-decoder neural network structure to process depth images as raw inputs. Even though the network is trained only on synthetic data, the model shows good performance at evaluation time on real-world data. With our proposed training scheme primed data training , we demonstrate how the DMU models can be trained efficiently to be sensitive to condition variables without having to rely on a stochastic information bottleneck. We validate the proposed methods in multiple scenarios of increasing complexity, beginning with the pose estimation of a single object to the joint estimation of the pose and the internal state of an articulated system. Moreover, we provide a benchmark against Articulated Signed Distance Functions(A-SDF) on the RBO dataset as a baseline comparison for articulation state estimation.ISSN:2377-376

PLASTR: Planning for Autonomous Sampling-based Trowelling

Plaster is commonly used in the construction industry to finish walls and ceilings, but the application is labor-intensive and physically strenuous, which motivates the need for automation. We present PLASTR, a receding horizon optimization-based planning algorithm for robotic plaster trowelling. It samples trowelling sequence rollouts from a new plaster simulator and weights them according to the flatness of the finished wall.\\ The proposed simulator approximates the real-world plaster-trowel interaction adequately while allowing execution orders of magnitude faster than real-time. We evaluate PLASTR in simulation and on a real-world test setup and compare it to two handcrafted heuristic baseline algorithms. PLASTR performs equal to or better than the best heuristic in terms of material coverage for both simulated and real-world experiments while being 50% more efficient in terms of trowelled distance.ISSN:2377-376

Design and Motion Planning for a Reconfigurable Robotic Base

ISSN:2377-376

A Fully-Integrated Sensing and Control System for High-Accuracy Mobile Robotic Building Construction

We present a fully-integrated sensing and control system which enables mobile manipulator robots to execute building tasks with millimeter-scale accuracy on building construction sites. The approach leverages multi-modal sensing capabilities for state estimation, tight integration with digital building models, and integrated trajectory planning and whole-body motion control. A novel method for high-accuracy localization updates relative to the known building structure is proposed. The approach is implemented on a real platform and tested under realistic construction conditions. We show that the system can achieve sub-cm end-effector positioning accuracy during fully autonomous operation using solely on-board sensing