2,513 research outputs found

    Visual Servoing from straight lines

    Get PDF
    In this paper we consider the problem of controlling a robotic system by using the projection of 3D straight lines in the image plane of central catadioptric systems. Most of the effort in visual servoing are devoted to points, only few works have investigated the use of lines in visual servoing with traditional cameras and none has explored the case of omnidirectional cameras. First a generic central catadioptric interaction matrix for the projection of 3D straight lines is derived from the projection model of an entire class of camera. Then an image-based control law is designed and validated through simulation results and real experiments with a mobile robot.Dans cet article, nous présentons une stratégie de commande de systèmes robotiques en utilisant comme entrées d'une boucle d'asservissement visuel des primitives relatives à la projection de droites dans le plan image d'une caméra panoramique à point central unique. Afin de réaliser la commande d'un système robotique par asservissement visuel, il est nécessaire d'estimer la matrice d'interaction liant les mouvements de la caméra aux mouvements des primitives visuelles dans l'image. Dans cet article, nous dérivons la forme analytique de la matrice d'interaction générique relative à la projection de droites à partir d'un modèle de projection englobant la classe entière des caméras à point central unique. Elle est ensuite utilisée dans un schéma d'asservissement visuel. Des simulations ainsi que des résultats expérimentaux sur un robot mobile valident l'approche proposée

    Image based visual servoing using algebraic curves applied to shape alignment

    Get PDF
    Visual servoing schemes generally employ various image features (points, lines, moments etc.) in their control formulation. This paper presents a novel method for using boundary information in visual servoing. Object boundaries are modeled by algebraic equations and decomposed as a unique sum of product of lines. We propose that these lines can be used to extract useful features for visual servoing purposes. In this paper, intersection of these lines are used as point features in visual servoing. Simulations are performed with a 6 DOF Puma 560 robot using Matlab Robotics Toolbox for the alignment of a free-form object. Also, experiments are realized with a 2 DOF SCARA direct drive robot. Both simulation and experimental results are quite promising and show potential of our new method

    Image based visual servoing using bitangent points applied to planar shape alignment

    Get PDF
    We present visual servoing strategies based on bitangents for aligning planar shapes. In order to acquire bitangents we use convex-hull of a curve. Bitangent points are employed in the construction of a feature vector to be used in visual control. Experimental results obtained on a 7 DOF Mitsubishi PA10 robot, verifies the proposed method

    Visual Servoing from Deep Neural Networks

    Get PDF
    We present a deep neural network-based method to perform high-precision, robust and real-time 6 DOF visual servoing. The paper describes how to create a dataset simulating various perturbations (occlusions and lighting conditions) from a single real-world image of the scene. A convolutional neural network is fine-tuned using this dataset to estimate the relative pose between two images of the same scene. The output of the network is then employed in a visual servoing control scheme. The method converges robustly even in difficult real-world settings with strong lighting variations and occlusions.A positioning error of less than one millimeter is obtained in experiments with a 6 DOF robot.Comment: fixed authors lis

    Exploring Convolutional Networks for End-to-End Visual Servoing

    Full text link
    Present image based visual servoing approaches rely on extracting hand crafted visual features from an image. Choosing the right set of features is important as it directly affects the performance of any approach. Motivated by recent breakthroughs in performance of data driven methods on recognition and localization tasks, we aim to learn visual feature representations suitable for servoing tasks in unstructured and unknown environments. In this paper, we present an end-to-end learning based approach for visual servoing in diverse scenes where the knowledge of camera parameters and scene geometry is not available a priori. This is achieved by training a convolutional neural network over color images with synchronised camera poses. Through experiments performed in simulation and on a quadrotor, we demonstrate the efficacy and robustness of our approach for a wide range of camera poses in both indoor as well as outdoor environments.Comment: IEEE ICRA 201

    A Multi-Robot Cooperation Framework for Sewing Personalized Stent Grafts

    Get PDF
    This paper presents a multi-robot system for manufacturing personalized medical stent grafts. The proposed system adopts a modular design, which includes: a (personalized) mandrel module, a bimanual sewing module, and a vision module. The mandrel module incorporates the personalized geometry of patients, while the bimanual sewing module adopts a learning-by-demonstration approach to transfer human hand-sewing skills to the robots. The human demonstrations were firstly observed by the vision module and then encoded using a statistical model to generate the reference motion trajectories. During autonomous robot sewing, the vision module plays the role of coordinating multi-robot collaboration. Experiment results show that the robots can adapt to generalized stent designs. The proposed system can also be used for other manipulation tasks, especially for flexible production of customized products and where bimanual or multi-robot cooperation is required.Comment: 10 pages, 12 figures, accepted by IEEE Transactions on Industrial Informatics, Key words: modularity, medical device customization, multi-robot system, robot learning, visual servoing, robot sewin

    Visual servoing of an autonomous helicopter in urban areas using feature tracking

    Get PDF
    We present the design and implementation of a vision-based feature tracking system for an autonomous helicopter. Visual sensing is used for estimating the position and velocity of features in the image plane (urban features like windows) in order to generate velocity references for the flight control. These visual-based references are then combined with GPS-positioning references to navigate towards these features and then track them. We present results from experimental flight trials, performed in two UAV systems and under different conditions that show the feasibility and robustness of our approach

    Sim2Real View Invariant Visual Servoing by Recurrent Control

    Full text link
    Humans are remarkably proficient at controlling their limbs and tools from a wide range of viewpoints and angles, even in the presence of optical distortions. In robotics, this ability is referred to as visual servoing: moving a tool or end-point to a desired location using primarily visual feedback. In this paper, we study how viewpoint-invariant visual servoing skills can be learned automatically in a robotic manipulation scenario. To this end, we train a deep recurrent controller that can automatically determine which actions move the end-point of a robotic arm to a desired object. The problem that must be solved by this controller is fundamentally ambiguous: under severe variation in viewpoint, it may be impossible to determine the actions in a single feedforward operation. Instead, our visual servoing system must use its memory of past movements to understand how the actions affect the robot motion from the current viewpoint, correcting mistakes and gradually moving closer to the target. This ability is in stark contrast to most visual servoing methods, which either assume known dynamics or require a calibration phase. We show how we can learn this recurrent controller using simulated data and a reinforcement learning objective. We then describe how the resulting model can be transferred to a real-world robot by disentangling perception from control and only adapting the visual layers. The adapted model can servo to previously unseen objects from novel viewpoints on a real-world Kuka IIWA robotic arm. For supplementary videos, see: https://fsadeghi.github.io/Sim2RealViewInvariantServoComment: Supplementary video: https://fsadeghi.github.io/Sim2RealViewInvariantServ
    corecore