Positioning and trajectory following tasks in microsystems using model free visual servoing

In this paper, we explore model free visual servoing algorithms by experimentally evaluating their performances for various tasks performed on a microassembly workstation developed in our lab. Model free or so called uncalibrated visual servoing does not need the system calibration (microscope-camera-micromanipulator) and the model of the observed scene. It is robust to parameter changes and disturbances. We tested its performance in point-to-point positioning and various trajectory following tasks. Experimental results validate the utility of model free visual servoing in microassembly tasks

Visual servoing of aerial manipulators

The final publication is available at link.springer.comThis chapter describes the classical techniques to control an aerial manipulator by means of visual information and presents an uncalibrated image-based visual servo method to drive the aerial vehicle. The proposed technique has the advantage that it contains mild assumptions about the principal point and skew values of the camera, and it does not require prior knowledge of the focal length, in contrast to traditional image-based approaches.Peer ReviewedPostprint (author's final draft

Image based visual servoing using bitangent points applied to planar shape alignment

We present visual servoing strategies based on bitangents for aligning planar shapes. In order to acquire bitangents we use convex-hull of a curve. Bitangent points are employed in the construction of a feature vector to be used in visual control. Experimental results obtained on a 7 DOF Mitsubishi PA10 robot, verifies the proposed method

Uncalibrated image-based visual servoing

This paper develops a new method for uncalibrated image-based visual servoing. In contrast to traditional image-based visual servo, the proposed solution does not require a known value of camera focal length for the computation of the image Jacobian. Instead, it is estimated at run time from the observation of the tracked target. The technique is shown to outperform classical visual servoing schemes in situations with noisy calibration parameters and for unexpected changes in the camera zoom. The method’s performance is demonstrated both in simulation experiments and in a ROS implementation of a quadrotor servoing task. The developed solution is tightly integrated with ROS and is made available as part of the IRI ROS stack.Peer ReviewedPostprint (author draft version

Model-based vs. model-free visual servoing: A Performance evaluation in microsystems

In this paper, model-based and model-free image based visual servoing (VS) approaches are implemented on a microassembly workstation, and their regulation and tracking performances are evaluated. A precise image based VS relies on computation of the image jacobian. In the model-based visual servoing, the image Jacobian is computed via calibrating the optical system. Precisely calibrated model based VS promises better positioning and tracking performance than the model-free approach. However, in the model-free approach, optical system calibration is not required due to the dynamic Jacobian estimation, thus it has the advantage of adapting to the different operating modes

Uncalibrated image-based visual servoing

This paper develops a new method for uncalibrated image-based visual servoing. In contrast to traditional image-based visual servo, the proposed solution does not require a known value of camera focal length for the computation of the image Jacobian. Instead, it is estimated at run time from the observation of the tracked target. The technique is shown to outperform classical visual servoing schemes in situations with noisy calibration parameters and for unexpected changes in the camera zoom. The method’s performance is demonstrated both in simulation experiments and in a ROS implementation of a quadrotor servoing task. The developed solution is tightly integrated with ROS and is made available as part of the IRI ROS stack.Peer ReviewedPostprint (author draft version

Uncalibrated visual servo for unmanned aerial manipulation

© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper addresses the problem of autonomous servoing an unmanned redundant aerial manipulator using computer vision. The overactuation of the system is exploited by means of a hierarchical control law, which allows to prioritize several tasks during flight. We propose a safety-related primary task to avoid possible collisions. As a secondary task, we present an uncalibrated image-based visual servo strategy to drive the arm end-effector to a desired position and orientation by using a camera attached to it. In contrast to the previous visual servo approaches, a known value of camera focal length is not strictly required. To further improve flight behavior, we hierarchically add one task to reduce dynamic effects by vertically aligning the arm center of gravity to the multirotor gravitational vector, and another one that keeps the arm close to a desired configuration of high manipulability and avoiding arm joint limits. The performance of the hierarchical control law, with and without activation of each of the tasks, is shown in simulations and in real experiments confirming the viability of such prioritized control scheme for aerial manipulation.Peer ReviewedPostprint (author's final draft

A comparative study of conventional visual servoing schemes in microsystem applications

This paper presents an experimental comparison of conventional (calibrated and uncalibrated) image based visual servoing methods in various microsystem applications. Both visual servoing techniques were tested on a microassembly workstation, and their regulation and tracking performances are evaluated. Calibrated visual servoing demands the optical system calibration for the image Jacobian estimation and if a precise optical system calibration is done, it ensures a better accuracy, precision and settling time compared with the uncalibrated approach. On the other hand, in the uncalibrated approach, optical system calibration is not required and since the Jacobian is estimated dynamically, it is more flexible

Visual guidance of unmanned aerial manipulators

The ability to fly has greatly expanded the possibilities for robots to perform surveillance, inspection or map generation tasks. Yet it was only in recent years that research in aerial robotics was mature enough to allow active interactions with the environment. The robots responsible for these interactions are called aerial manipulators and usually combine a multirotor platform and one or more robotic arms. The main objective of this thesis is to formalize the concept of aerial manipulator and present guidance methods, using visual information, to provide them with autonomous functionalities. A key competence to control an aerial manipulator is the ability to localize it in the environment. Traditionally, this localization has required external infrastructure of sensors (e.g., GPS or IR cameras), restricting the real applications. Furthermore, localization methods with on-board sensors, exported from other robotics fields such as simultaneous localization and mapping (SLAM), require large computational units becoming a handicap in vehicles where size, load, and power consumption are important restrictions. In this regard, this thesis proposes a method to estimate the state of the vehicle (i.e., position, orientation, velocity and acceleration) by means of on-board, low-cost, light-weight and high-rate sensors. With the physical complexity of these robots, it is required to use advanced control techniques during navigation. Thanks to their redundancy on degrees-of-freedom, they offer the possibility to accomplish not only with mobility requirements but with other tasks simultaneously and hierarchically, prioritizing them depending on their impact to the overall mission success. In this work we present such control laws and define a number of these tasks to drive the vehicle using visual information, guarantee the robot integrity during flight, and improve the platform stability or increase arm operability. The main contributions of this research work are threefold: (1) Present a localization technique to allow autonomous navigation, this method is specifically designed for aerial platforms with size, load and computational burden restrictions. (2) Obtain control commands to drive the vehicle using visual information (visual servo). (3) Integrate the visual servo commands into a hierarchical control law by exploiting the redundancy of the robot to accomplish secondary tasks during flight. These tasks are specific for aerial manipulators and they are also provided. All the techniques presented in this document have been validated throughout extensive experimentation with real robotic platforms.La capacitat de volar ha incrementat molt les possibilitats dels robots per a realitzar tasques de vigilància, inspecció o generació de mapes. Tot i això, no és fins fa pocs anys que la recerca en robòtica aèria ha estat prou madura com per començar a permetre interaccions amb l’entorn d’una manera activa. Els robots per a fer-ho s’anomenen manipuladors aeris i habitualment combinen una plataforma multirotor i un braç robòtic. L’objectiu d’aquesta tesi és formalitzar el concepte de manipulador aeri i presentar mètodes de guiatge, utilitzant informació visual, per dotar d’autonomia aquest tipus de vehicles. Una competència clau per controlar un manipulador aeri és la capacitat de localitzar-se en l’entorn. Tradicionalment aquesta localització ha requerit d’infraestructura sensorial externa (GPS, càmeres IR, etc.), limitant així les aplicacions reals. Pel contrari, sistemes de localització exportats d’altres camps de la robòtica basats en sensors a bord, com per exemple mètodes de localització i mapejat simultànis (SLAM), requereixen de gran capacitat de còmput, característica que penalitza molt en vehicles on la mida, pes i consum elèctric son grans restriccions. En aquest sentit, aquesta tesi proposa un mètode d’estimació d’estat del robot (posició, velocitat, orientació i acceleració) a partir de sensors instal·lats a bord, de baix cost, baix consum computacional i que proporcionen mesures a alta freqüència. Degut a la complexitat física d’aquests robots, és necessari l’ús de tècniques de control avançades. Gràcies a la seva redundància de graus de llibertat, aquests robots ens ofereixen la possibilitat de complir amb els requeriments de mobilitat i, simultàniament, realitzar tasques de manera jeràrquica, ordenant-les segons l’impacte en l’acompliment de la missió. En aquest treball es presenten aquestes lleis de control, juntament amb la descripció de tasques per tal de guiar visualment el vehicle, garantir la integritat del robot durant el vol, millorar de l’estabilitat del vehicle o augmentar la manipulabilitat del braç. Aquesta tesi es centra en tres aspectes fonamentals: (1) Presentar una tècnica de localització per dotar d’autonomia el robot. Aquest mètode està especialment dissenyat per a plataformes amb restriccions de capacitat computacional, mida i pes. (2) Obtenir les comandes de control necessàries per guiar el vehicle a partir d’informació visual. (3) Integrar aquestes accions dins una estructura de control jeràrquica utilitzant la redundància del robot per complir altres tasques durant el vol. Aquestes tasques son específiques per a manipuladors aeris i també es defineixen en aquest document. Totes les tècniques presentades en aquesta tesi han estat avaluades de manera experimental amb plataformes robòtiques real

Model free visual servoing in macro and micro domain robotic applications

This thesis explores model free visual servoing algorithms by experimentally evaluating their performances for various tasks performed both in macro and micro domains. Model free or so called uncalibrated visual servoing does not need the system (vision system + robotic system) calibration and the model of the observed scene, since it provides an online estimation of the composite (image + robot) Jacobian. It is robust to parameter changes and disturbances. A model free visual servoing scheme is tested on a 7 DOF Mitsubishi PA10 robotic arm and on a microassembly workstation which is developed in our lab. In macro domain, a new approach for planar shape alignment is presented. The alignment task is performed based on bitangent points which are acquired using convex-hull of a curve. Both calibrated and uncalibrated visual servoing schemes are employed and compared. Furthermore, model free visual servoing is used for various trajectory following tasks such as square, circle, sine etc. and these reference trajectories are generated by a linear interpolator which produces midway targets along them. Model free visual servoing can provide more exibility in microsystems, since the calibration of the optical system is a tedious and error prone process, and recalibration is required at each focusing level of the optical system. Therefore, micropositioning and three di erent trajectory following tasks are also performed in micro world. Experimental results validate the utility of model free visual servoing algorithms in both domains