Visual servoing sequencing able to avoid obstacles.

International audienceClassical visual servoing approaches tend to constrain all degrees of freedom (DOF) of the robot during the execution of a task. In this article a new approach is proposed. The key idea is to control the robot with a very under-constrained task when it is far from the desired position, and to incrementally constrain the global task by adding further tasks as the robot moves closer to the goal. As long as they are sufficient, the remaining DOF are used to avoid undesirable configurations, such as joint limits. Closer from the goal, when not enough DOF remain available for avoidance, an execution controller selects a task to be temporary removed from the applied tasks. The released DOF can then be used for the joint limits avoidance. A complete solution to implement this general idea is proposed. Experiments that prove the validity of the approach are also provided

Visual servoing with safe interaction using image moments

The problem of image based visual servoing for robots working in a cluttered dynamic environment is addressed in this paper. It is assumed that the environment is observed by depth sensors which allow to measure the distance between any moving obstacle and the robot. Also an eye-in-hand camera is used to extract image features. The main idea is to control suitable image moments and to relax a certain number of robot's degrees of freedom during the interaction phase. If an obstacle approaches the robot, the main visual servoing task is relaxed partially or completely, while the image features are kept in the camera field of view by controlling the image moments. Fuzzy rules are used to set the desired values of the image moments. Beside that, the relaxed redundancy of the robot is exploited to avoid collisions. After removing the risk of collision, the main visual servoing task is resumed. The effectiveness of the algorithm is shown by several case studies on a KUKA LWR 4 robot arm

Visual Servoing for Nonholonomically Constrained Three Degree of Freedom Kinematic Systems

This paper addresses problems of robot navigation with nonholonomic motion constraints and perceptual cues arising from onboard visual servoing in partially engineered environments. We propose a general hybrid procedure that adapts to the constrained motion setting the standard feedback controller arising from a navigation function in the fully actuated case. This is accomplished by switching back and forth between moving down and across the associated gradient field toward the stable manifold it induces in the constrained dynamics. Guaranteed to avoid obstacles in all cases, we provide conditions under which the new procedure brings initial configurations to within an arbitrarily small neighborhood of the goal. We summarize simulation results on a sample of visual servoing problems with a few different perceptual models. We document the empirical effectiveness of the proposed algorithm by reporting the results of its application to outdoor autonomous visual registration experiments with the robot RHex guided by engineered beacons

Autonomous Mechanical Assembly on the Space Shuttle: An Overview

The space shuttle will be equipped with a pair of 50 ft. manipulators used to handle payloads and to perform mechanical assembly operations. Although current plans call for these manipulators to be operated by a human teleoperator. The possibility of using results from robotics and machine intelligence to automate this shuttle assembly system was investigated. The major components of an autonomous mechanical assembly system are examined, along with the technology base upon which they depend. The state of the art in advanced automation is also assessed

Analysis and Observations from the First Amazon Picking Challenge

This paper presents a overview of the inaugural Amazon Picking Challenge along with a summary of a survey conducted among the 26 participating teams. The challenge goal was to design an autonomous robot to pick items from a warehouse shelf. This task is currently performed by human workers, and there is hope that robots can someday help increase efficiency and throughput while lowering cost. We report on a 28-question survey posed to the teams to learn about each team's background, mechanism design, perception apparatus, planning and control approach. We identify trends in this data, correlate it with each team's success in the competition, and discuss observations and lessons learned based on survey results and the authors' personal experiences during the challenge

Visual servo control on a humanoid robot

Includes bibliographical referencesThis thesis deals with the control of a humanoid robot based on visual servoing. It seeks to confer a degree of autonomy to the robot in the achievement of tasks such as reaching a desired position, tracking or/and grasping an object. The autonomy of humanoid robots is considered as crucial for the success of the numerous services that this kind of robots can render with their ability to associate dexterity and mobility in structured, unstructured or even hazardous environments. To achieve this objective, a humanoid robot is fully modeled and the control of its locomotion, conditioned by postural balance and gait stability, is studied. The presented approach is formulated to account for all the joints of the biped robot. As a way to conform the reference commands from visual servoing to the discrete locomotion mode of the robot, this study exploits a reactive omnidirectional walking pattern generator and a visual task Jacobian redefined with respect to a floating base on the humanoid robot, instead of the stance foot. The redundancy problem stemming from the high number of degrees of freedom coupled with the omnidirectional mobility of the robot is handled within the task priority framework, allowing thus to achieve con- figuration dependent sub-objectives such as improving the reachability, the manipulability and avoiding joint limits. Beyond a kinematic formulation of visual servoing, this thesis explores a dynamic visual approach and proposes two new visual servoing laws. Lyapunov theory is used first to prove the stability and convergence of the visual closed loop, then to derive a robust adaptive controller for the combined robot-vision dynamics, yielding thus an ultimate uniform bounded solution. Finally, all proposed schemes are validated in simulation and experimentally on the humanoid robot NAO

Robot Visual Servoing Using Discontinuous Control

This work presents different proposals to deal with common problems in robot visual servoing based on the application of discontinuous control methods. The feasibility and effectiveness of the proposed approaches are substantiated by simulation results and real experiments using a 6R industrial manipulator. The main contributions are: - Geometric invariance using sliding mode control (Chapter 3): the defined higher-order invariance is used by the proposed approaches to tackle problems in visual servoing. Proofs of invariance condition are presented. - Fulfillment of constraints in visual servoing (Chapter 4): the proposal uses sliding mode methods to satisfy mechanical and visual constraints in visual servoing, while a secondary task is considered to properly track the target object. The main advantages of the proposed approach are: low computational cost, robustness and fully utilization of the allowed space for the constraints. - Robust auto tool change for industrial robots using visual servoing (Chapter 4): visual servoing and the proposed method for constraints fulfillment are applied to an automated solution for tool changing in industrial robots. The robustness of the proposed method is due to the control law of the visual servoing, which uses the information acquired by the vision system to close a feedback control loop. Furthermore, sliding mode control is simultaneously used in a prioritized level to satisfy the aforementioned constraints. Thus, the global control accurately places the tool in the warehouse, but satisfying the robot constraints. - Sliding mode controller for reference tracking (Chapter 5): an approach based on sliding mode control is proposed for reference tracking in robot visual servoing using industrial robot manipulators. The novelty of the proposal is the introduction of a sliding mode controller that uses a high-order discontinuous control signal, i.e., joint accelerations or joint jerks, in order to obtain a smoother behavior and ensure the robot system stability, which is demonstrated with a theoretical proof. - PWM and PFM for visual servoing in fully decoupled approaches (Chapter 6): discontinuous control based on pulse width and pulse frequency modulation is proposed for fully decoupled position based visual servoing approaches, in order to get the same convergence time for camera translation and rotation. Moreover, other results obtained in visual servoing applications are also described.Este trabajo presenta diferentes propuestas para tratar problemas habituales en el control de robots por realimentación visual, basadas en la aplicación de métodos de control discontinuos. La viabilidad y eficacia de las propuestas se fundamenta con resultados en simulación y con experimentos reales utilizando un robot manipulador industrial 6R. Las principales contribuciones son: - Invariancia geométrica utilizando control en modo deslizante (Capítulo 3): la invariancia de alto orden definida aquí es utilizada después por los métodos propuestos, para tratar problemas en control por realimentación visual. Se apuertan pruebas teóricas de la condición de invariancia. - Cumplimiento de restricciones en control por realimentación visual (Capítulo 4): esta propuesta utiliza métodos de control en modo deslizante para satisfacer restricciones mecánicas y visuales en control por realimentación visual, mientras una tarea secundaria se encarga del seguimiento del objeto. Las principales ventajas de la propuesta son: bajo coste computacional, robustez y plena utilización del espacio disponible para las restricciones. - Cambio de herramienta robusto para un robot industrial mediante control por realimentación visual (Capítulo 4): el control por realimentación visual y el método propuesto para el cumplimiento de las restricciones se aplican a una solución automatizada para el cambio de herramienta en robots industriales. La robustez de la propuesta radica en el uso del control por realimentación visual, que utiliza información del sistema de visión para cerrar el lazo de control. Además, el control en modo deslizante se utiliza simultáneamente en un nivel de prioridad superior para satisfacer las restricciones. Así pues, el control es capaz de dejar la herramienta en el intercambiador de herramientas de forma precisa, a la par que satisface las restricciones del robot. - Controlador en modo deslizante para seguimiento de referencia (Capítulo 5): se propone un enfoque basado en el control en modo deslizante para seguimiento de referencia en robots manipuladores industriales controlados por realimentación visual. La novedad de la propuesta radica en la introducción de un controlador en modo deslizante que utiliza la señal de control discontinua de alto orden, i.e. aceleraciones o jerks de las articulaciones, para obtener un comportamiento más suave y asegurar la estabilidad del sistema robótico, lo que se demuestra con una prueba teórica. - Control por realimentación visual mediante PWM y PFM en métodos completamente desacoplados (Capítulo 6): se propone un control discontinuo basado en modulación del ancho y frecuencia del pulso para métodos completamente desacoplados de control por realimentación visual basados en posición, con el objetivo de conseguir el mismo tiempo de convergencia para los movimientos de rotación y traslación de la cámara . Además, se presentan también otros resultados obtenidos en aplicaciones de control por realimentación visual.Aquest treball presenta diferents propostes per a tractar problemes habituals en el control de robots per realimentació visual, basades en l'aplicació de mètodes de control discontinus. La viabilitat i eficàcia de les propostes es fonamenta amb resultats en simulació i amb experiments reals utilitzant un robot manipulador industrial 6R. Les principals contribucions són: - Invariància geomètrica utilitzant control en mode lliscant (Capítol 3): la invariància d'alt ordre definida ací és utilitzada després pels mètodes proposats, per a tractar problemes en control per realimentació visual. S'aporten proves teòriques de la condició d'invariància. - Compliment de restriccions en control per realimentació visual (Capítol 4): aquesta proposta utilitza mètodes de control en mode lliscant per a satisfer restriccions mecàniques i visuals en control per realimentació visual, mentre una tasca secundària s'encarrega del seguiment de l'objecte. Els principals avantatges de la proposta són: baix cost computacional, robustesa i plena utilització de l'espai disponible per a les restriccions. - Canvi de ferramenta robust per a un robot industrial mitjançant control per realimentació visual (Capítol 4): el control per realimentació visual i el mètode proposat per al compliment de les restriccions s'apliquen a una solució automatitzada per al canvi de ferramenta en robots industrials. La robustesa de la proposta radica en l'ús del control per realimentació visual, que utilitza informació del sistema de visió per a tancar el llaç de control. A més, el control en mode lliscant s'utilitza simultàniament en un nivell de prioritat superior per a satisfer les restriccions. Així doncs, el control és capaç de deixar la ferramenta en l'intercanviador de ferramentes de forma precisa, a la vegada que satisfà les restriccions del robot. - Controlador en mode lliscant per a seguiment de referència (Capítol 5): es proposa un enfocament basat en el control en mode lliscant per a seguiment de referència en robots manipuladors industrials controlats per realimentació visual. La novetat de la proposta radica en la introducció d'un controlador en mode lliscant que utilitza senyal de control discontínua d'alt ordre, i.e. acceleracions o jerks de les articulacions, per a obtindre un comportament més suau i assegurar l'estabilitat del sistema robòtic, la qual cosa es demostra amb una prova teòrica. - Control per realimentació visual mitjançant PWM i PFM en mètodes completament desacoblats (Capítol 6): es proposa un control discontinu basat en modulació de l'ample i la freqüència del pols per a mètodes completament desacoblats de control per realimentació visual basats en posició, amb l'objectiu d'aconseguir el mateix temps de convergència per als moviments de rotació i translació de la càmera. A més, es presenten també altres resultats obtinguts en aplicacions de control per realimentació visual.Muñoz Benavent, P. (2017). Robot Visual Servoing Using Discontinuous Control [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/90430TESI

Model-based autonomous system for performing dexterous, human-level manipulation tasks

This article presents a model based approach to autonomous dexterous manipulation, developed as part of the DARPA Autonomous Robotic Manipulation Software (ARM-S) program. Performing human-level manipulation tasks is achieved through a novel combination of perception in uncertain environments, precise tool use, forceful dual-arm planning and control, persistent environmental tracking, and task level verification. Deliberate interaction with the environment is incorporated into planning and control strategies, which, when coupled with world estimation, allows for refinement of models and precise manipulation. The system takes advantage of sensory feedback immediately with little open-loop execution, attempting true autonomous reasoning and multi-step sequencing that adapts in the face of changing and uncertain environments. A tire change scenario utilizing human tools, discussed throughout the article, is used to described the system approach. A second scenario of cutting a wire is also presented, and is used to illustrate system component reuse and generality.United States. Defense Advanced Research Projects Agency. Autonomous Robotic Manipulation Progra

Hybrid Vision and Force Control in Robotic Manufacturing Systems

The ability to provide a physical interaction between an industrial robot and a workpiece in the environment is essential for a successful manipulation task. In this context, a wide range of operations such as deburring, pushing, and polishing are considered. The key factor to successfully accomplish such operations by a robot is to simultaneously control the position of the tool-tip of the end-effector and interaction force between the tool and the workpiece, which is a challenging task. This thesis aims to develop new reliable control strategies combining vision and force feedbacks to track a path on the workpiece while controlling the contacting force. In order to fulfill this task, the novel robust hybrid vision and force control approaches are presented for industrial robots subject to uncertainties and interacting with unknown workpieces. The main contributions of this thesis lie in several parts. In the first part of the thesis, a robust cascade vision and force approach is suggested to control industrial robots interacting with unknown workpieces considering model uncertainties. This cascade structure, consisting of an inner vision loop and an outer force loop, avoids the conflict between the force and vision control in traditional hybrid methods without decoupling force and vision systems. In the second part of the thesis, a novel image-based task-sequence/path planning scheme coupled with a robust vision and force control method for solving the multi-task operation problem of an eye-in-hand (EIH) industrial robot interacting with a workpiece is suggested. Each task is defined as tracking a predefined path or positioning to a single point on the workpiece’s surface with a desired interacting force signal, i.e., interaction with the workpiece. The proposed method suggests an optimal task sequence planning scheme to carry out all the tasks and an optimal path planning method to generate a collision-free path between the tasks, i.e., when the robot performs free-motion (pure vision control). In the third part of the project, a novel multi-stage method for robust hybrid vision and force control of industrial robots, subject to model uncertainties is proposed. It aims to improve the performance of the three phases of the control process: a) free-motion using the image-based visual servoing (IBVS) before the interaction with the workpiece; b) the moment that the end-effector touches the workpiece; and c) hybrid vision and force control during the interaction with the workpiece. In the fourth part of the thesis, a novel approach for hybrid vision and force control of eye-in-hand industrial robots is presented which addresses the problem of camera’s field-of-view (FOV) limitation. The merit of the proposed method is that it is capable of expanding the workpiece for eye-in-hand industrial robots to cope with the FOV limitation of the interaction tasks on the workpiece. All the developed algorithms in the thesis are validated via tests on a 6-DOF Denso robot in an eye-in-hand configuration