2 1/2 D Visual servoing with respect to unknown objects through a new estimation scheme of camera displacement

Abstract. Classical visual servoing techniques need a strong a priori knowledge of the shape and the dimensions of the observed objects. In this paper, we present how the 2 1/2 D visual servoing scheme we have recently developed, can be used with unknown objects characterized by a set of points. Our scheme is based on the estimation of the camera displacement from two views, given by the current and desired images. Since vision-based robotics tasks generally necessitate to be performed at video rate, we focus only on linear algorithms. Classical linear methods are based on the computation of the essential matrix. In this paper, we propose a different method, based on the estimation of the homography matrix related to a virtual plane attached to the object. We show that our method provides a more stable estimation when the epipolar geometry degenerates. This is particularly important in visual servoing to obtain a stable control law, especially near the convergence of the system. Finally, experimental results confirm the improvement in the stability, robustness, and behaviour of our scheme with respect to classical methods. Keywords: visual servoing, projective geometry, homography 1

Estimation of the camera pose from image point correspondences through the essential matrix and convex optimization

Estimating the camera pose in stereo vision systems is an important issue in computer vision and robotics. One popular way to handle this problem consists of determining the essential matrix which minimizes the algebraic error obtained from image point correspondences. Unfortunately, this search amounts to solving a nonconvex optimization, and the existing methods either rely on some approximations in order to get rid of the non-convexity or provide a solution that may be affected by the presence of local minima. This paper proposes a new approach to address this search without presenting such problems. In particular, we show that the sought essential matrix can be obtained by solving a convex optimization built through a suitable reformulation of the considered minimization via appropriate techniques for representing polynomials. Numerical results show the proposed approach compares favorably with some standard methods in both cases of synthetic data and real data. © 2009 IEEE.published_or_final_versio

Designing image trajectories in the presence of uncertain data for robust visual servoing path-planning

Path-planning allows one to steer a camera to a desired location while taking into account the presence of constraints such as visibility, workspace, and joint limits. Unfortunately, the planned path can be significantly different from the real path due to the presence of uncertainty on the available data, with the consequence that some constraints may be not fulfilled by the real path even if they are satisfied by the planned path. In this paper we address the problem of performing robust path-planning, i.e. computing a path that satisfies the required constraints not only for the nominal model as in traditional path-planning but rather for a family of admissible models. Specifically, we consider an uncertain model where the point correspondences between the initial and desired views and the camera intrinsic parameters are affected by unknown random uncertainties with known bounds. The difficulty we have to face is that traditional path-planning schemes applied to different models lead to different paths rather than to a common and robust path. To solve this problem we propose a technique based on polynomial optimization where the required constraints are imposed on a number of trajectories corresponding to admissible camera poses and parameterized by a common design variable. The planned image trajectory is then followed by using an IBVS controller. Simulations carried out with all typical uncertainties that characterize a real experiment illustrate the proposed strategy and provide promising results. © 2009 IEEE.published_or_final_versio

Euclidean Calculation of Feature Points of a Rotating Satellite: A Daisy Chaining Approach

The occlusion of feature points and/or feature points leaving the field of view of a camera is a significant practical problem that can lead to degraded performance or instability of visual servo control and vision-based estimation algorithms. By assuming that one knownEuclidean distance between two feature points in an initial view is available, homography relationships and image geometry are used in this paper to determine the Euclidean coordinates of feature points in the field of view. A new daisy-chainingmethod is then used to relate the position and orientation of a plane defined by the feature points to other feature-point planes that are rigidly connected. Through these relationships, the Euclidean coordinates of the original feature points can be tracked even as they leave the field of view. This objective is motivated by the desire to track the Euclidean coordinates of feature points on one face of a satellite as it continually rotates and feature points become self-occluded. A numerical simulation is included to demonstrate that the Euclidean coordinates can be tracked even when they leave the field of view. However, the results indicate the need for amethod to reconcile any accumulated errorwhen the feature points return to thefield of view. Nomenclature A = intrinsic camera-calibration matrix dj = distance to j plane along nj F j, Fj = frames attached to the j and j planes Gj = projective homography matrix of the jth frame Hj = Euclidean homography matrix of the jth frame I = fixed coordinate frame attached to the camera mji, m ji = normalized Euclidean coordinate of the ith feature point of the j and j planes expressed in

ViSP for visual servoing: a generic software platform with a wide class of robot control skills

Special issue on Software Packages for Vision-Based Control of Motion, P. Oh, D. Burschka (Eds.)International audienceViSP (Visual Servoing Platform), a fully functional modular architecture that allows fast development of visual servoing applications, is described. The platform takes the form of a library which can be divided in three main modules: control processes, canonical vision-based tasks that contain the most classical linkages, and real-time tracking. ViSP software environment features independence with respect to the hardware, simplicity, extendibility, and portability. ViSP also features a large library of elementary tasks with various visual features that can be combined together, an image processing library that allows the tracking of visual cues at video rate, a simulator, an interface with various classical framegrabbers, a virtual 6-DOF robot that allows the simulation of visual servoing experiments, etc. The platform is implemented in C++ under Linux

An Investigation of Nonlinear Estimation and System Design for Mechatronic Systems

This thesis is a collection of two projects in which the author was involved during his master\u27s degree program. The first project involves the estimation of 3D Euclidean coordinates of features from 2D images. A 3D Euclidean position estimation strategy is developed for a static object using a single moving camera whose motion is known. This Euclidean depth estimator has a very simple mathematical structure and is easy to implement. Numerical simulations and experimental results using a mobile robot in an indoor environment are presented to illustrate the performance of the algorithm. The second section describes the design of a test system for the Argon Environment Electrical Study (AEES) conducted by the Department of Energy (DOE). The initial research proposal, safety review, and literature review are presented. Additionally, the test plan and system design are highlighted

High-Performance Control of an On-Board Missile Seeker Using Vision Information

학위논문 (석사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2016. 2. 하인중.본 논문에서는 미사일 탐색기에 영상센서를 도입하여 고성능, 고효율의 제어 성능을 보장할 수 있는 탐색기 제어기를 설계한다. 특히 표적의 깊이 정보 없이도 조준선 오차를 빠르게 0으로 수렴시켜 표적을 추적할 수 있음을 보였다. 제안하고 있는 새로운 탐색기 제어기는 표적 운동에 대한 선형 시불변 추정기를 도입하였다. 먼저 영상센서로부터 획득한 정보를 이용하여 이동 표적에 대한 호모그래피 행렬을 유도하였다. 또한 호모그래피 행렬을 분해하여 표적과 탐색기의 운동에 대한 정보를 획득한다. 획득한 운동 정보를 바탕으로 병진 운동 표적의 동역학 방정식과 측정 방정식을 선형 시불변 시스템으로 나타낸다. 이러한 병진 운동 표적 동역학 방정식에 기반하여 표적의 크기에 대한 불확실성을 고려한 루엔버거 관측기 형태의 표적 병진 운동 정보 추정기를 사용한다. 도입한 표적 병진 운동 정보 추정기는 기존의 기법들과 달리 영상 센서의 움직임과 관계없이 항상 수렴성을 보장할 수 있다. 이는 미사일 탐색기 제어기에 활용하기 매우 적합한 형태이다. 또한 영상 센서를 활용한 탐색기의 동역학 방정식을 유도하였다. 탐색기의 동역학 방정식과 표적 병진 운동 추정기에서 획득한 표적의 운동 정보를 이용하여 시선 변화율 추정기를 설계하였다. 더 나아가 설계한 추정기를 바탕으로 시선 변화율을 보상한 탐색기 제어 명령을 생성하도록 한다. 마지막으로 본 논문에서 제안한 탐색기 제어기가 표적을 추적할 수 있음을 증명하기 위해 수학적으로 엄밀한 분석을 제공한다. 또한 모의 실험을 실행하여 기존의 탐색기 제어기와 성능을 비교하고 제안한 기법의 실용성을 입증하도록 한다.This dissertation proposes a high-performance controller of an on-board missile seeker using vision information. The seeker controller can approach to a moving target without knowing the information of the target depth. Our approach consists of two parts: 1) an innovative time invariant linear estimator of the target motion, 2) a nonlinear seeker controller. First, by using the parameters of the homography matrix for a moving target, we derive the dynamic equation of a moving target as a time invariant system. This equation was derived under the assumption that the velocities of both seeker and the target are varying slowly. Based on the derived dynamic equation of the target motion, an innovative time invariant linear estimator is constructed, which could provide the information of target velocity. Different from the previous works, the proposed estimator does not require any motion of the seeker, such as snaking or accelerating of the seeker, for estimation convergence. Besides, it can guarantee the convergence even without knowing the information of the target depth. Next, a nonlinear seeker controller to bring the boresight error down to zero is proposed. We present some rigorous mathematical convergence analysis to demonstrate that the proposed seeker controller can track the moving target even when the information of the target depth is not given. Furthermore, we present the simulation result of conventional seeker controller to clarify the practicability of the proposed seeker controller. Thus, the proposed approach should be used and applied widely in industries and military applications.1. 서 론 1 1.1 연구 배경 1 1.2 연구 목표 5 2. 기존 탐색기 제어기법 7 2.1 표적의 운동을 고려한 호모그래피 행렬 8 2.2 이동 표적을 고려한 탐색기 제어 기법 12 2.2.1 미사일 탐색기의 동역학 방정식 12 2.2.2 표적 6자유도 운동 정보 추정 기법 19 2.3 표적 병진 운동 추정 기법 23 3. 새로운 탐색기 제어기 설계 40 3.1 새로운 제어기 설계 및 분석 40 3.2 모의 실험 결과 51 4. 결론 및 향후 연구 과제 57 참고문헌 59 Abstract 67Maste