Visual Servoing

The goal of this book is to introduce the visional application by excellent researchers in the world currently and offer the knowledge that can also be applied to another field widely. This book collects the main studies about machine vision currently in the world, and has a powerful persuasion in the applications employed in the machine vision. The contents, which demonstrate that the machine vision theory, are realized in different field. For the beginner, it is easy to understand the development in the vision servoing. For engineer, professor and researcher, they can study and learn the chapters, and then employ another application method

Enhanced Image-Based Visual Servoing Dealing with Uncertainties

Nowadays, the applications of robots in industrial automation have been considerably increased. There is increasing demand for the dexterous and intelligent robots that can work in unstructured environment. Visual servoing has been developed to meet this need by integration of vision sensors into robotic systems. Although there has been significant development in visual servoing, there still exist some challenges in making it fully functional in the industry environment. The nonlinear nature of visual servoing and also system uncertainties are part of the problems affecting the control performance of visual servoing. The projection of 3D image to 2D image which occurs in the camera creates a source of uncertainty in the system. Another source of uncertainty lies in the camera and robot manipulator's parameters. Moreover, limited field of view (FOV) of the camera is another issues influencing the control performance. There are two main types of visual servoing: position-based and image-based. This project aims to develop a series of new methods of image-based visual servoing (IBVS) which can address the nonlinearity and uncertainty issues and improve the visual servoing performance of industrial robots. The first method is an adaptive switch IBVS controller for industrial robots in which the adaptive law deals with the uncertainties of the monocular camera in eye-in-hand configuration. The proposed switch control algorithm decouples the rotational and translational camera motions and decomposes the IBVS control into three separate stages with different gains. This method can increase the system response speed and improve the tracking performance of IBVS while dealing with camera uncertainties. The second method is an image feature reconstruction algorithm based on the Kalman filter which is proposed to handle the situation where the image features go outside the camera's FOV. The combination of the switch controller and the feature reconstruction algorithm can not only improve the system response speed and tracking performance of IBVS, but also can ensure the success of servoing in the case of the feature loss. Next, in order to deal with the external disturbance and uncertainties due to the depth of the features, the third new control method is designed to combine proportional derivative (PD) control with sliding mode control (SMC) on a 6-DOF manipulator. The properly tuned PD controller can ensure the fast tracking performance and SMC can deal with the external disturbance and depth uncertainties. In the last stage of the thesis, the fourth new semi off-line trajectory planning method is developed to perform IBVS tasks for a 6-DOF robotic manipulator system. In this method, the camera's velocity screw is parametrized using time-based profiles. The parameters of the velocity profile are then determined such that the velocity profile takes the robot to its desired position. This is done by minimizing the error between the initial and desired features. The algorithm for planning the orientation of the robot is decoupled from the position planning of the robot. This allows a convex optimization problem which lead to a faster and more efficient algorithm. The merit of the proposed method is that it respects all of the system constraints. This method also considers the limitation caused by camera's FOV. All the developed algorithms in the thesis are validated via tests on a 6-DOF Denso robot in an eye-in-hand configuration

Survey of Visual and Force/Tactile Control of Robots for Physical Interaction in Spain

Sensors provide robotic systems with the information required to perceive the changes that happen in unstructured environments and modify their actions accordingly. The robotic controllers which process and analyze this sensory information are usually based on three types of sensors (visual, force/torque and tactile) which identify the most widespread robotic control strategies: visual servoing control, force control and tactile control. This paper presents a detailed review on the sensor architectures, algorithmic techniques and applications which have been developed by Spanish researchers in order to implement these mono-sensor and multi-sensor controllers which combine several sensors

High-Speed Vision and Force Feedback for Motion-Controlled Industrial Manipulators

Over the last decades, both force sensors and cameras have emerged as useful sensors for different applications in robotics. This thesis considers a number of dynamic visual tracking and control problems, as well as the integration of these techniques with contact force control. Different topics ranging from basic theory to system implementation and applications are treated. A new interface developed for external sensor control is presented, designed by making non-intrusive extensions to a standard industrial robot control system. The structure of these extensions are presented, the system properties are modeled and experimentally verified, and results from force-controlled stub grinding and deburring experiments are presented. A novel system for force-controlled drilling using a standard industrial robot is also demonstrated. The solution is based on the use of force feedback to control the contact forces and the sliding motions of the pressure foot, which would otherwise occur during the drilling phase. Basic methods for feature-based tracking and servoing are presented, together with an extension for constrained motion estimation based on a dual quaternion pose parametrization. A method for multi-camera real-time rigid body tracking with time constraints is also presented, based on an optimal selection of the measured features. The developed tracking methods are used as the basis for two different approaches to vision/force control, which are illustrated in experiments. Intensity-based techniques for tracking and vision-based control are also developed. A dynamic visual tracking technique based directly on the image intensity measurements is presented, together with new stability-based methods suitable for dynamic tracking and feedback problems. The stability-based methods outperform the previous methods in many situations, as shown in simulations and experiments

Unfalsified visual servoing for simultaneous object recognition and pose tracking

In a complex environment, simultaneous object recognition and tracking has been one of the challenging topics in computer vision and robotics. Current approaches are usually fragile due to spurious feature matching and local convergence for pose determination. Once a failure happens, these approaches lack a mechanism to recover automatically. In this paper, data-driven unfalsified control is proposed for solving this problem in visual servoing. It recognizes a target through matching image features with a 3-D model and then tracks them through dynamic visual servoing. The features can be falsified or unfalsified by a supervisory mechanism according to their tracking performance. Supervisory visual servoing is repeated until a consensus between the model and the selected features is reached, so that model recognition and object tracking are accomplished. Experiments show the effectiveness and robustness of the proposed algorithm to deal with matching and tracking failures caused by various disturbances, such as fast motion, occlusions, and illumination variation

相対座標における高速視覚フィードバックに基づくダイナミックコンペンセーション

学位の種別:課程博士University of Tokyo(東京大学

Image Based Visual Servoing Using Trajectory Planning and Augmented Visual Servoing Controller

Robots and automation manufacturing machineries have become an inseparable part of industry, nowadays. However, robotic systems are generally limited to operate in highly structured environments. Although, sensors such as laser tracker, indoor GPS, 3D metrology and tracking systems are used for positioning and tracking in manufacturing and assembly tasks, these devices are highly limited to the working environment and the speed of operation and they are generally very expensive. Thus, integration of vision sensors with robotic systems and generally visual servoing system allows the robots to work in unstructured spaces, by producing non-contact measurements of the working area. However, projecting a 3D space into a 2D space, which happens in the camera, causes the loss of one dimension data. This initiates the challenges in vision based control. Moreover, the nonlinearities and complex structure of a manipulator robot make the problem more challenging. This project aims to develop new reliable visual servoing methods that allow its use in real robotic tasks. The main contributions of this project are in two parts; the visual servoing controller and trajectory planning algorithm. In the first part of the project, a new image based visual servoing controller called Augmented Image Based Visual Servoing (AIBVS) is presented. A proportional derivative (PD) controller is developed to generate acceleration as the controlling command of the robot. The stability analysis of the controller is conducted using Lyapanov theory. The developed controller has been tested on a 6 DOF Denso robot. The experimental results on point features and image moment features demonstrate the performance of the proposed AIBVS. Experimental results show that a damped response could be achieved using a PD controller with acceleration output. Moreover, smoother feature and robot trajectories are observed compared to those in conventional IBVS controllers. Later on, this controller is used on a moving object catching process. Visual servoing controllers have shown difficulty in stabilizing the system in global space. Hence, in the second part of the project, a trajectory planning algorithm is developed to achieve the global stability of the system. The trajectory planning is carried out by parameterizing the camera's velocity screw. The camera's velocity screw is parameterized using time-based profiles. The parameters of the velocity profile are then determined such that the velocity profile guides the robot to its desired position. This is done by minimizing the error between the initial and desired features. This method provides a reliable path for the robot considering all robotic constraints. The developed algorithm is tested on a Denso robot. The results show that the trajectory planning algorithm is able to perform visual servoing tasks which are unstable when performed using visual servoing controllers

Vision-Based Control of Flexible Robot Systems

This thesis covers the controlling of flexible robot systems by using a camera as a measurement device. To accomplish the purpose of the study, the estimation process of dynamic state variables of flexible link robot has been examined based on camera measurements. For the purpose of testing two application examples for flexible link have been applied, an algorithm for the dynamic state variables estimation is proposed. Flexible robots can have very complex dynamic behavior during their operations, which can lead to induced vibrations. Since the vibrations and its derivative are not all measurable, therefore the estimation of state variables plays a significant role in the state feedback control of flexible link robots. A vision sensor (i.e. camera) realizing a contact-less measurement sensor can be used to measure the deflection of flexible robot arm. Using a vision sensor, however, would generate new effects such as limited accuracy and time delay, which are the main inherent problems of the application of vision sensors within the context. These effects and related compensation approaches are studied in this thesis. An indirect method for link deflection (i.e. system states) sensing is presented. It uses a vision system consisting of a CCD camera and an image processing unit. The main purpose of this thesis is to develop an estimation approach combining suitable measurement devices which are easy to realize with improved reliability. It includes designing two state estimators; the first one for the traditional sensor type (negligible noise and time delay) and the second one is for the camera measurement which account for the dynamic error due to the time delay. The estimation approach is applied first using a single link flexible robot; the dynamic model of the flexible link is derived using a finite element method. Based on the suggested estimation approach, the first observer estimates the vibrations using strain gauge (fast and complete dynamics), and the second observer estimates the vibrations using vision data (slow dynamical parts). In order to achieve an optimal estimation, a proper combination process of the two estimated dynamical parts of the system dynamics is described. The simulation results for the estimations based on vision measurements show that the slow dynamical states can be estimated and the observer can compensate the time delay dynamic errors. It is also observed that an optimal estimation can be attained by combining slow dynamical estimated states with those of fast observer-based on strain gauge measurement. Based on suggested estimation approach a vision-based control for elastic shipmounted crane is designed to regulate the motion of the payload. For the observers and the controller design, a linear dynamic model of elastic-ship mounted crane incorporating a finite element technique for modeling flexible link is employed. In order to estimate the dynamic states variables and the unknown disturbance two state observers are designed. The first one estimates the state variables using camera measurement (augmented Kalman filter). The second one used potentiometers measurement (PI-Observer). To realize a multi-model approach of elastic-ship mounted crane, a variable gain controller and variable gain observers are designed. The variable gain controller is used to generate the required damping to control the system based on the estimated states and the roll angle. Simulation results show that the variable gain observers can adequately estimate the states and the unknown disturbance acting on the payload. It is further observed that the variable gain controller can effectively reduce the payload pendulations. Experiments are conducted using the camera to measure the link deflection of scaled elastic ship-mounted crane system. The results shown that the variable gain controller based on the combined states observers mitigated the vibrations of the system and the swinging of the payload. The presented material above is embedded into an interrelated thesis. A concise introduction to the vision-based control and state estimation problems is attached in the first chapter. An extensive survey of available visual servoing algorithms that include the rigid robot system and the flexible robot system is also presented. The conclusions of the work and suggestions for the future research are provided at the last chapter of this thesis