Pose Estimation of Free-form Objects: Theory and Experiments

In this report we present geometric foundations and an algorithmic approach to deal with the 2D-3D pose estimation problem for free-form surface models. This work is an extension to earlier studies presented in [29]. The discussion of 1D contour models in [29] is extended to 2D free-form surface models. We use a parametric representation of surfaces and apply Fourier transformations to gain low-pass descriptions of objects. We present an algorithm for pose estimation, which uses the silhouette of the object as pictorial information and recovers the 3D pose of the object even for changing aspects of the object during image sequences. We further present extensions to couple surface and contour information on objects and show the potential of our chosen approach for complex objects and scenes

Pose Estimation 3D Free-form Contours

In this report we discuss the 2D-3D pose estimation problem of 3D free-form contours. In our scenario we observe objects of any 3D shape in an image of a calibrated camera. Pose estimation means to estimate the relative position and orientation (containing a rotation $R$ and translation $T$) of the 3D object to the reference camera system. The fusion of modeling free-form contours within the pose estimation problem is achieved by using the conformal geometric algebra. The conformal geometric algebra is a geometric algebra which models entities as stereographic projected entities in an homogeneous model. This leads to a linear description of kinematics on the one hand and projective geometry on the other hand. To model free-form contours in the conformal framework we use twists to model cycloidal curves as twist-depending functions and interpret $n$-times nested cycloidal curves as functions generated by 3D Fourier descriptors. This means, we use the twist concept to apply a spectral domain representation of 3D contours within the pose estimation problem. We will show that twist representations of objects can numerically efficient and easily be applied to the pose estimation problem. The pose problem itself is formalized as implicit problem and we gain constraint equations, which have to be fulfilled with respect to the unknown rigid body motion. Several experiments visualize the robustness and real-time performance of our algorithms

Pose Estimation Revisited

The presented thesis deals with the 2D-3D pose estimation problem. Pose estimation means to estimate the relative position and orientation of a 3D object with respect to a reference camera system. The main focus concentrates on the geometric modeling and application of the pose problem. To deal with the different geometric spaces (Euclidean, affine and projective ones), a homogeneous model for conformal geometry is applied in the geometric algebra framework. It allows for a compact and linear modeling of the pose scenario. In the chosen embedding of the pose problem, a rigid body motion is represented as an orthogonal transformation whose parameters can be estimated efficiently in the corresponding Lie algebra. In addition, the chosen algebraic embedding allows the modeling of extended features derived from sphere concepts in contrast to point concepts used in classical vector calculus. For pose estimation, 3D object models are further treated two-fold, feature based and free-form based: While the feature based pose scenarios provide constraint equations to link different image and object entities, the free-form approach for pose estimation is achieved by applying extracted image silhouettes from objects on 3D free-form contours modeled by 3D Fourier descriptors. In conformal geometric algebra an extended scenario is derived, which deals beside point features with higher-order features such as lines, planes, circles, spheres, kinematic chains or cycloidal curves. This scenario is extended to general free-form contours by interpreting contours generated with 3D Fourier descriptors as n-times nested cycloidal curves. The introduced method for shape modeling links signal theory, geometry and kinematics and is applied advantageously for 2D-3D silhouette based free-form pose estimation. The experiments show the real-time capability and noise stability of the algorithms. Experiments of a running navigation system with visual self-localization are also presented

Point Pair Feature based Object Detection for Random Bin Picking

Point pair features are a popular representation for free form 3D object detection and pose estimation. In this paper, their performance in an industrial random bin picking context is investigated. A new method to generate representative synthetic datasets is proposed. This allows to investigate the influence of a high degree of clutter and the presence of self similar features, which are typical to our application. We provide an overview of solutions proposed in literature and discuss their strengths and weaknesses. A simple heuristic method to drastically reduce the computational complexity is introduced, which results in improved robustness, speed and accuracy compared to the naive approach

Dimensional Measurement of Objects in Single Images Independent from Restrictive Camera Parameters

Recent advances in microelectronics have produced new generations of digital cameras with variable focal lengths and pixel sizes which facilitate automatic and high-quality imaging. However, without knowing the values of these critical camera parameters, it is difficult to measure objects in images using existing algorithms. This work investigates this important problem aiming at dimensional measurements (e.g., diameter, length, width and height) of regularly shaped physical objects in a single 2-D image free from restrictive camera parameters. Traditionally, such measurements usually require determinations of the poses of a certain reference feature, i.e., the location and orientation of the feature relative to the camera, in order to establish a geometric model for the dimensional calculation. Points or lines associated with certain shapes (including triangles and rectangles) are often used as reference features for the pose estimation. However, with only a single image as the input, these methods assume the availability of 3-D spatial relationships of the points or lines, which limits the applications of these methods to practical problems where this knowledge is unavailable or difficult to estimate, such as in the problem of image-based food portion size estimation in dietary assessment. In addition to points and lines, the circle has also been used as a reference feature because it has a single elliptic perspective projection in images. However, almost all the existing approaches treat the parameters of focal length and pixel size as the necessary prior information. Here, we propose a new approach to dimensional estimation based on single image input using the circular reference feature and a pin-hole model without considering camera distortion. Without knowing the focal length and pixel size, our approach provides a closed-form solution for the orientation estimation of the circular feature. With additional information provided, such as the size of the circular reference feature, analytical solutions are provided for physical length estimation between an arbitrary pair of points on the reference plane. Studies using both synthetic and actual objects have been conducted to evaluate this new method, which exhibited satisfactory results. This method has also been applied to the measurement of food dimensions based on digital pictures of foods in circular dining plates

Recovering 6D Object Pose: A Review and Multi-modal Analysis

A large number of studies analyse object detection and pose estimation at visual level in 2D, discussing the effects of challenges such as occlusion, clutter, texture, etc., on the performances of the methods, which work in the context of RGB modality. Interpreting the depth data, the study in this paper presents thorough multi-modal analyses. It discusses the above-mentioned challenges for full 6D object pose estimation in RGB-D images comparing the performances of several 6D detectors in order to answer the following questions: What is the current position of the computer vision community for maintaining "automation" in robotic manipulation? What next steps should the community take for improving "autonomy" in robotics while handling objects? Our findings include: (i) reasonably accurate results are obtained on textured-objects at varying viewpoints with cluttered backgrounds. (ii) Heavy existence of occlusion and clutter severely affects the detectors, and similar-looking distractors is the biggest challenge in recovering instances' 6D. (iii) Template-based methods and random forest-based learning algorithms underlie object detection and 6D pose estimation. Recent paradigm is to learn deep discriminative feature representations and to adopt CNNs taking RGB images as input. (iv) Depending on the availability of large-scale 6D annotated depth datasets, feature representations can be learnt on these datasets, and then the learnt representations can be customized for the 6D problem

Hybrid Bayesian Eigenobjects: Combining Linear Subspace and Deep Network Methods for 3D Robot Vision

We introduce Hybrid Bayesian Eigenobjects (HBEOs), a novel representation for 3D objects designed to allow a robot to jointly estimate the pose, class, and full 3D geometry of a novel object observed from a single viewpoint in a single practical framework. By combining both linear subspace methods and deep convolutional prediction, HBEOs efficiently learn nonlinear object representations without directly regressing into high-dimensional space. HBEOs also remove the onerous and generally impractical necessity of input data voxelization prior to inference. We experimentally evaluate the suitability of HBEOs to the challenging task of joint pose, class, and shape inference on novel objects and show that, compared to preceding work, HBEOs offer dramatically improved performance in all three tasks along with several orders of magnitude faster runtime performance.Comment: To appear in the International Conference on Intelligent Robots (IROS) - Madrid, 201