Learning Driven Coarse-to-Fine Articulated Robot Tracking

In this work we present an articulated tracking approach for robotic manipulators, which relies only on visual cues from colour and depth images to estimate the robot’s state when interacting with or being occluded by its environment. We hypothesise that articulated model fitting approaches can only achieve accurate tracking if subpixel-level accurate correspondences between observed and estimated state can be established. Previous work in this area has exclusively relied on either discriminative depth information or colour edge correspondences as tracking objective and required initialisation from joint encoders. In this paper we propose a coarse-to-fine articulated state estimator, which relies only on visual cues from colour edges and learned depth keypoints, and which is initialised from a robot state distribution predicted from a depth image. We evaluate our approach on four RGB-D sequences showing a KUICA LWR arm with a Schunk SDH2 hand interacting with its environment and demonstrate that this combined keypoint and edge tracking objective can estimate the palm position with an average error of 2. 5cm without using any joint encoder sensing

Articulated pose estimation via over-parametrization and noise projection

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2014.Cataloged from PDF version of thesis.Includes bibliographical references (pages 155-161).Outside the factory, robots will often encounter mechanical systems with which they need to interact. The robot may need to open and unload a kitchen dishwasher or move around heavy construction equipment. Many of the mechanical systems encountered can be described as a series of rigid segments connected by joints. The pose of a segment places constraints on adjacent segments because they are mechanically 'connected. When modeling or perceiving the motion of such an articulated system, it is beneficial to make use of these constraints to reduce uncertainty. In this thesis, we examine two aspects of perception related to articulated structures. First, we examine the special case of a single segment and recover the rigid body transformation between two sensors mounted on it. Second, we consider the task of tracking the configuration of a multi-segment structure, given some knowledge of its kinematics. First, we develop an algorithm to recover the rigid body transformation, or extrinsic calibration, between two sensors on a link of a mobile robot. The single link, a degenerate articulated object, is often encountered in practice. The algorithm requires only a set of sensor observations made as the robot moves along a suitable path. Over-parametrization of poses avoids degeneracies and the corresponding Lie algebra enables noise projection to and from the over-parametrized space. We demonstrate and validate the end-to-end calibration procedure, achieving Cramer-Rao Lower Bounds. The parameters are accurate to millimeters and milliradians in the case of planar LIDARs data and about 1 cm and 1 degree for 6-DOF RGB-D cameras. Second, we develop a particle filter to track an articulated object. Unlike most previous work, the algorithm accepts a kinematic description as input and is not specific to a particular object. A potentially incomplete series of observations of the object's links are used to form an on-line estimate of the object's configuration (i.e., the pose of one link and the joint positions). The particle filter does not require a reliable state transition model, since observations are incorporated during particle proposal. Noise is modeled in the observation space, an over-parametrization of the state space, reducing the dependency on the kinematic description. We compare our method to several alternative implementations and demonstrate lower tracking error for fixed observation noise.by Jonathan David Brookshire.Ph. D

Camera Marker Networks for Pose Estimation and Scene Understanding in Construction Automation and Robotics.

The construction industry faces challenges that include high workplace injuries and fatalities, stagnant productivity, and skill shortage. Automation and Robotics in Construction (ARC) has been proposed in the literature as a potential solution that makes machinery easier to collaborate with, facilitates better decision-making, or enables autonomous behavior. However, there are two primary technical challenges in ARC: 1) unstructured and featureless environments; and 2) differences between the as-designed and the as-built. It is therefore impossible to directly replicate conventional automation methods adopted in industries such as manufacturing on construction sites. In particular, two fundamental problems, pose estimation and scene understanding, must be addressed to realize the full potential of ARC. This dissertation proposes a pose estimation and scene understanding framework that addresses the identified research gaps by exploiting cameras, markers, and planar structures to mitigate the identified technical challenges. A fast plane extraction algorithm is developed for efficient modeling and understanding of built environments. A marker registration algorithm is designed for robust, accurate, cost-efficient, and rapidly reconfigurable pose estimation in unstructured and featureless environments. Camera marker networks are then established for unified and systematic design, estimation, and uncertainty analysis in larger scale applications. The proposed algorithms' efficiency has been validated through comprehensive experiments. Specifically, the speed, accuracy and robustness of the fast plane extraction and the marker registration have been demonstrated to be superior to existing state-of-the-art algorithms. These algorithms have also been implemented in two groups of ARC applications to demonstrate the proposed framework's effectiveness, wherein the applications themselves have significant social and economic value. The first group is related to in-situ robotic machinery, including an autonomous manipulator for assembling digital architecture designs on construction sites to help improve productivity and quality; and an intelligent guidance and monitoring system for articulated machinery such as excavators to help improve safety. The second group emphasizes human-machine interaction to make ARC more effective, including a mobile Building Information Modeling and way-finding platform with discrete location recognition to increase indoor facility management efficiency; and a 3D scanning and modeling solution for rapid and cost-efficient dimension checking and concise as-built modeling.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113481/1/cforrest_1.pd