Estimation of the camera pose from image point correspondences through the essential matrix and convex optimization

Estimating the camera pose in stereo vision systems is an important issue in computer vision and robotics. One popular way to handle this problem consists of determining the essential matrix which minimizes the algebraic error obtained from image point correspondences. Unfortunately, this search amounts to solving a nonconvex optimization, and the existing methods either rely on some approximations in order to get rid of the non-convexity or provide a solution that may be affected by the presence of local minima. This paper proposes a new approach to address this search without presenting such problems. In particular, we show that the sought essential matrix can be obtained by solving a convex optimization built through a suitable reformulation of the considered minimization via appropriate techniques for representing polynomials. Numerical results show the proposed approach compares favorably with some standard methods in both cases of synthetic data and real data. © 2009 IEEE.published_or_final_versio

Optimal object configurations to minimize the positioning error in visual servoing

Image noise unavoidably affects the available image points that are used in visual-servoing schemes to steer a robot end-effector toward a desired location. As a consequence, letting the image points in the current view converge to those in the desired view does not ensure that the camera converges accurately to the desired location. This paper investigates the selection of object configurations to minimize the worst-case positioning error due to the presence of image noise. In particular, a strategy based on linear matrix inequalities (LMIs) and barrier functions is proposed to compute upper and lower bounds of this error for a given maximum error of the image points. This strategy can be applied to problems such as selecting an optimal subset of object points or determining an optimal position of an object in the scene. Some examples illustrate the use of the proposed strategy in such problems. © 2010 IEEE.published_or_final_versio

Conferring robustness to path-planning for image-based control

Path-planning has been proposed in visual servoing for reaching the desired location while fulfilling various constraints. Unfortunately, the real trajectory can be significantly different from the reference trajectory due to the presence of uncertainties on the model used, with the consequence that some constraints may not be fulfilled hence leading to a failure of the visual servoing task. This paper proposes a new strategy for addressing this problem, where the idea consists of conferring robustness to the path-planning scheme by considering families of admissible models. In order to obtain these families, uncertainty in the form of random variables is introduced on the available image points and intrinsic parameters. Two families are considered, one by generating a given number of admissible models corresponding to extreme values of the uncertainty, and one by estimating the extreme values of the components of the admissible models. Each model of these families identifies a reference trajectory, which is parametrized by design variables that are common to all the models. The design variables are hence determined by imposing that all the reference trajectories fulfill the required constraints. Discussions on the convergence and robustness of the proposed strategy are provided, in particular showing that the satisfaction of the visibility and workspace constraints for the second family ensures the satisfaction of these constraints for all models bounded by this family. The proposed strategy is illustrated through simulations and experiments. © 2011 IEEE.published_or_final_versio

A visual servoing path-planning strategy for cameras obeying the unified model

Part of 2010 IEEE Multi-Conference on Systems and ControlRecently, a unified camera model has been introduced in visual control systems in order to describe through a unique mathematical model conventional perspective cameras, fisheye cameras, and catadioptric systems. In this paper, a path-planning strategy for visual servoing is proposed for any camera obeying this unified model. The proposed strategy is based on the projection onto a virtual plane of the available image projections. This has two benefits. First, it allows one to perform camera pose estimation and 3D object reconstruction by using methods for conventional camera that are not valid for other cameras. Second, it allows one to perform image pathplanning for multi-constraint satisfaction by using a simplified but equivalent projection model, that in this paper is addressed by introducing polynomial parametrizations of the rotation and translation. The planned image trajectory is hence tracked by using an IBVS controller. The proposed strategy is validated through simulations with image noise and calibration errors typical of real experiments. It is worth remarking that visual servoing path-planning for non conventional perspective cameras has not been proposed yet in the literature. © 2010 IEEE.published_or_final_versionThe 2010 IEEE International Symposium on Computer-Aided Control System Design (CACSD), Yokohama, Japan, 8-10 September 2010. In Proceedings of CACSD, 2010, p. 1795-180

Position and motion estimation for visual robot control with planar targets

This paper addresses two problems in visually-controlled robots. The first consists of positioning the end-effector of a robot manipulator on a plane of interest by using a monocular vision system. The problem amounts to estimating the transformation between the coordinates of an image point and its three-dimensional location supposing that only the camera intrinsic parameters are known. The second problem consists of positioning the robot end-effector with respect to an object of interest free to move on a plane, and amounts to estimating the camera displacement in a stereo vision system in the presence of motion constraints. For these problems, some solutions are proposed through dedicated optimizations based on decoupling the effects of rotation and translation and based on an a-priori imposition of the degrees of freedom of the system. These solutions are illustrated via simulations and experiments. ©2009 ACA.published_or_final_versionThe 7th Asian Control Conference (ASCC 2009), Hong Kong, China, 27-29 August 2009. In Proceedings of the Asian Control Conference, 2009, p. 372-37

Position and motion estimation for visual robot control with planar targets

This paper addresses two problems in visually-controlled robots. The first consists of positioning the end-effector of a robot manipulator on a plane of interest by using a monocular vision system. The problem amounts to estimating the transformation between the coordinates of an image point and its three-dimensional location supposing that only the camera intrinsic parameters are known. The second problem consists of positioning the robot end-effector with respect to an object of interest free to move on a plane, and amounts to estimating the camera displacement in a stereo vision system in the presence of motion constraints. For these problems, some solutions are proposed through dedicated optimizations based on decoupling the effects of rotation and translation and based on an a-priori imposition of the degrees of freedom of the system. These solutions are illustrated via simulations and experiments. ©2009 ACA.published_or_final_versionThe 7th Asian Control Conference (ASCC 2009), Hong Kong, China, 27-29 August 2009. In Proceedings of the Asian Control Conference, 2009, p. 372-37

Visual servoing path planning for cameras obeying the unified model

This paper proposes a path planning visual servoing strategy for a class of cameras that includes conventional perspective cameras, fisheye cameras and catadioptric cameras as special cases. Specifically, these cameras are modeled by adopting a unified model recently proposed in the literature and the strategy consists of designing image trajectories for eye-in-hand robotic systems that allow the robot to reach a desired location while satisfying typical visual servoing constraints. To this end, the proposed strategy introduces the projection of the available image features onto a virtual plane and the computation of a feasible image trajectory through polynomial programming. Then, the computed image trajectory is tracked by using an image-based visual servoing controller. Experimental results with a fisheye camera mounted on a 6-d.o.f. robot arm are presented in order to illustrate the proposed strategy. © 2012 Copyright Taylor & Francis and The Robotics Society of Japan.postprin

Depth adaptive zooming visual servoing for a robot with a zooming camera

To solve the view visibility problem and keep the observed object in the field of view (FOV) during the visual servoing, a depth adaptive zooming visual servoing strategy for a manipulator robot with a zooming camera is proposed. Firstly, a zoom control mechanism is introduced into the robot visual servoing system. It can dynamically adjust the camera's field of view to keep all the feature points on the object in the field of view of the camera and get high object local resolution at the end of visual servoing. Secondly, an invariant visual servoing method is employed to control the robot to the desired position under the changing intrinsic parameters of the camera. Finally, a nonlinear depth adaptive estimation scheme in the invariant space using Lyapunov stability theory is proposed to estimate adaptively the depth of the image features on the object. Three kinds of robot 4DOF visual positioning simulation experiments are conducted. The simulation experiment results show that the proposed approach has higher positioning precision. © 2013 Xin et al

Robust and Cooperative Image-Based Visual Servoing System Using a Redundant Architecture

The reliability and robustness of image-based visual servoing systems is still unsolved by the moment. In order to address this issue, a redundant and cooperative 2D visual servoing system based on the information provided by two cameras in eye-in-hand/eye-to-hand configurations is proposed. Its control law has been defined to assure that the whole system is stable if each subsystem is stable and to allow avoiding typical problems of image-based visual servoing systems like task singularities, features extraction errors, disappearance of image features, local minima, etc. Experimental results with an industrial robot manipulator based on Schunk modular motors to demonstrate the stability, performance and robustness of the proposed system are presented

Applying BAT Evolutionary Optimization to Image-Based Visual Servoing

This paper presents a predictive control strategy for an image-based visual servoing scheme that employs evolutionary optimization. The visual control task is approached as a nonlinear optimization problem that naturally handles relevant visual servoing constraints such as workspace limitations and visibility restrictions. As the predictive scheme requires a reliable model, this paper uses a local model that is based on the visual interaction matrix and a global model that employs 3D trajectory data extracted from a quaternion-based interpolator. The work assumes a free-flying camera with 6-DOF simulation whose results support the discussion on the constraint handling and the image prediction scheme