Autonomous Visual Servo Robotic Capture of Non-cooperative Target

This doctoral research develops and validates experimentally a vision-based control scheme for the autonomous capture of a non-cooperative target by robotic manipulators for active space debris removal and on-orbit servicing. It is focused on the final capture stage by robotic manipulators after the orbital rendezvous and proximity maneuver being completed. Two challenges have been identified and investigated in this stage: the dynamic estimation of the non-cooperative target and the autonomous visual servo robotic control. First, an integrated algorithm of photogrammetry and extended Kalman filter is proposed for the dynamic estimation of the non-cooperative target because it is unknown in advance. To improve the stability and precision of the algorithm, the extended Kalman filter is enhanced by dynamically correcting the distribution of the process noise of the filter. Second, the concept of incremental kinematic control is proposed to avoid the multiple solutions in solving the inverse kinematics of robotic manipulators. The proposed target motion estimation and visual servo control algorithms are validated experimentally by a custom built visual servo manipulator-target system. Electronic hardware for the robotic manipulator and computer software for the visual servo are custom designed and developed. The experimental results demonstrate the effectiveness and advantages of the proposed vision-based robotic control for the autonomous capture of a non-cooperative target. Furthermore, a preliminary study is conducted for future extension of the robotic control with consideration of flexible joints

Design of an Anthropomorphic, Compliant, and Lightweight Dual Arm for Aerial Manipulation

This paper presents an anthropomorphic, compliant and lightweight dual arm manipulator designed and developed for aerial manipulation applications with multi-rotor platforms. Each arm provides four degrees of freedom in a human-like kinematic configuration for end effector positioning: shoulder pitch, roll and yaw, and elbow pitch. The dual arm, weighting 1.3 kg in total, employs smart servo actuators and a customized and carefully designed aluminum frame structure manufactured by laser cut. The proposed design reduces the manufacturing cost as no computer numerical control machined part is used. Mechanical joint compliance is provided in all the joints, introducing a compact spring-lever transmission mechanism between the servo shaft and the links, integrating a potentiometer for measuring the deflection of the joints. The servo actuators are partially or fully isolated against impacts and overloads thanks to the ange bearings attached to the frame structure that support the rotation of the links and the deflection of the joints. This simple mechanism increases the robustness of the arms and safety in the physical interactions between the aerial robot and the environment. The developed manipulator has been validated through different experiments in fixed base test-bench and in outdoor flight tests.Unión Europea H2020-ICT-2014- 644271Ministerio de Economía y Competitividad DPI2015-71524-RMinisterio de Economía y Competitividad DPI2017-89790-

NASA Automated Rendezvous and Capture Review. Executive summary

In support of the Cargo Transfer Vehicle (CTV) Definition Studies in FY-92, the Advanced Program Development division of the Office of Space Flight at NASA Headquarters conducted an evaluation and review of the United States capabilities and state-of-the-art in Automated Rendezvous and Capture (AR&C). This review was held in Williamsburg, Virginia on 19-21 Nov. 1991 and included over 120 attendees from U.S. government organizations, industries, and universities. One hundred abstracts were submitted to the organizing committee for consideration. Forty-two were selected for presentation. The review was structured to include five technical sessions. Forty-two papers addressed topics in the five categories below: (1) hardware systems and components; (2) software systems; (3) integrated systems; (4) operations; and (5) supporting infrastructure

Visual and Kinematic Coordinated Control of Mobile Manipulating Unmanned Aerial Vehicles

Manipulating objects using arms mounted to unmanned aerial vehicles (UAVs) is attractive because UAVs may access many locations that are otherwise inaccessible to traditional mobile manipulation platforms such as ground vehicles. Historically, UAVs have been employed in ways that avoid interaction with the environment at all costs. The recent trend of increasing small UAV lift capacity and the reduction of the weight of manipulator components make the realization of mobile manipulating UAVs imminent. Despite recent work, several major challenges remain to be overcome before it will be common practice to manipulate objects from UAVs. Among these challenges, the constantly moving UAV platform and compliance of manipulator arms make it difficult to position the UAV and end-effector relative to an object of interest precisely enough for reliable manipulation. Solving this challenge will bring UAVs one step closer to being able to perform meaningful tasks such as infrastructure repair, disaster response, law enforcement, and personal assistance. Toward a solution to this challenge, this thesis describes a way forward that uses the UAV as a means to crudely position a manipulator within reach of the end-effector's goal position in the world. The manipulator then performs the fine positioning of the end-effector, rejecting position perturbations caused by UAV motions. An algorithm to coordinate the redundant degrees of freedom of an aerial manipulation system is described that allows the motions of the manipulator to serve as inputs to the UAV's position controller. To demonstrate this algorithm, the manipulator's six degrees of freedom are servoed using visual sensing to drive an eye-in-hand camera to a specified pose relative to a target while treating motions of the host platform as perturbations. Simultaneously, the host platform's degrees of freedom are regulated using kinematic information from the manipulator. This ultimately drives the UAV to a position that allows the manipulator to assume a pose relative to the UAV that maximizes reachability, thus facilitating the arm's ability to compensate for undesired UAV motions. Maintaining this loose kinematic coupling between the redundant degrees of freedom of the host UAV and manipulator allows this type of controller to be applied to a wide variety of platforms, including manned aircraft, rather than a single instance of a purpose-built system. As a result of this loose coupling, careful consideration must be given to the manipulator design so that it can achieve useful poses while minimally influencing the stability of the host UAV. Accordingly, the novel application of a parallel manipulator mechanism is described.Ph.D., Mechanical Engineering -- Drexel University, 201

The AEROARMS Project: Aerial Robots with Advanced Manipulation Capabilities for Inspection and Maintenance

This article summarizes new aerial robotic manipulation technologies and methods—aerial robotic manipulators with dual arms and multidirectional thrusters—developed in the AEROARMS project for outdoor industrial inspection and maintenance (I&M). Our report deals with the control systems, including the control of the interaction forces and the compliance the teleoperation, which uses passivity to tackle the tradeoff between stability and performance the perception methods for localization, mapping, and inspection the planning methods, including a new control-aware approach for aerial manipulation. Finally, we describe a novel industrial platform with multidirectional thrusters and a new arm design to increase the robustness in industrial contact inspections. In addition, the lessons learned in applying the platform to outdoor aerial manipulation for I&M are pointed out

Visual guidance of unmanned aerial manipulators

The ability to fly has greatly expanded the possibilities for robots to perform surveillance, inspection or map generation tasks. Yet it was only in recent years that research in aerial robotics was mature enough to allow active interactions with the environment. The robots responsible for these interactions are called aerial manipulators and usually combine a multirotor platform and one or more robotic arms. The main objective of this thesis is to formalize the concept of aerial manipulator and present guidance methods, using visual information, to provide them with autonomous functionalities. A key competence to control an aerial manipulator is the ability to localize it in the environment. Traditionally, this localization has required external infrastructure of sensors (e.g., GPS or IR cameras), restricting the real applications. Furthermore, localization methods with on-board sensors, exported from other robotics fields such as simultaneous localization and mapping (SLAM), require large computational units becoming a handicap in vehicles where size, load, and power consumption are important restrictions. In this regard, this thesis proposes a method to estimate the state of the vehicle (i.e., position, orientation, velocity and acceleration) by means of on-board, low-cost, light-weight and high-rate sensors. With the physical complexity of these robots, it is required to use advanced control techniques during navigation. Thanks to their redundancy on degrees-of-freedom, they offer the possibility to accomplish not only with mobility requirements but with other tasks simultaneously and hierarchically, prioritizing them depending on their impact to the overall mission success. In this work we present such control laws and define a number of these tasks to drive the vehicle using visual information, guarantee the robot integrity during flight, and improve the platform stability or increase arm operability. The main contributions of this research work are threefold: (1) Present a localization technique to allow autonomous navigation, this method is specifically designed for aerial platforms with size, load and computational burden restrictions. (2) Obtain control commands to drive the vehicle using visual information (visual servo). (3) Integrate the visual servo commands into a hierarchical control law by exploiting the redundancy of the robot to accomplish secondary tasks during flight. These tasks are specific for aerial manipulators and they are also provided. All the techniques presented in this document have been validated throughout extensive experimentation with real robotic platforms.La capacitat de volar ha incrementat molt les possibilitats dels robots per a realitzar tasques de vigilància, inspecció o generació de mapes. Tot i això, no és fins fa pocs anys que la recerca en robòtica aèria ha estat prou madura com per començar a permetre interaccions amb l’entorn d’una manera activa. Els robots per a fer-ho s’anomenen manipuladors aeris i habitualment combinen una plataforma multirotor i un braç robòtic. L’objectiu d’aquesta tesi és formalitzar el concepte de manipulador aeri i presentar mètodes de guiatge, utilitzant informació visual, per dotar d’autonomia aquest tipus de vehicles. Una competència clau per controlar un manipulador aeri és la capacitat de localitzar-se en l’entorn. Tradicionalment aquesta localització ha requerit d’infraestructura sensorial externa (GPS, càmeres IR, etc.), limitant així les aplicacions reals. Pel contrari, sistemes de localització exportats d’altres camps de la robòtica basats en sensors a bord, com per exemple mètodes de localització i mapejat simultànis (SLAM), requereixen de gran capacitat de còmput, característica que penalitza molt en vehicles on la mida, pes i consum elèctric son grans restriccions. En aquest sentit, aquesta tesi proposa un mètode d’estimació d’estat del robot (posició, velocitat, orientació i acceleració) a partir de sensors instal·lats a bord, de baix cost, baix consum computacional i que proporcionen mesures a alta freqüència. Degut a la complexitat física d’aquests robots, és necessari l’ús de tècniques de control avançades. Gràcies a la seva redundància de graus de llibertat, aquests robots ens ofereixen la possibilitat de complir amb els requeriments de mobilitat i, simultàniament, realitzar tasques de manera jeràrquica, ordenant-les segons l’impacte en l’acompliment de la missió. En aquest treball es presenten aquestes lleis de control, juntament amb la descripció de tasques per tal de guiar visualment el vehicle, garantir la integritat del robot durant el vol, millorar de l’estabilitat del vehicle o augmentar la manipulabilitat del braç. Aquesta tesi es centra en tres aspectes fonamentals: (1) Presentar una tècnica de localització per dotar d’autonomia el robot. Aquest mètode està especialment dissenyat per a plataformes amb restriccions de capacitat computacional, mida i pes. (2) Obtenir les comandes de control necessàries per guiar el vehicle a partir d’informació visual. (3) Integrar aquestes accions dins una estructura de control jeràrquica utilitzant la redundància del robot per complir altres tasques durant el vol. Aquestes tasques son específiques per a manipuladors aeris i també es defineixen en aquest document. Totes les tècniques presentades en aquesta tesi han estat avaluades de manera experimental amb plataformes robòtiques real

Carnegie Mellon Team Tartan: Mission-level Robustness with Rapidly Deployed Autonomous Aerial Vehicles in the MBZIRC 2020

For robotics systems to be used in high risk, real-world situations, they have to be quickly deployable and robust to environmental changes, under-performing hardware, and mission subtask failures. Robots are often designed to consider a single sequence of mission events, with complex algorithms lowering individual subtask failure rates under some critical constraints. Our approach is to leverage common techniques in vision and control and encode robustness into mission structure through outcome monitoring and recovery strategies, aided by a system infrastructure that allows for quick mission deployments under tight time constraints and no central communication. We also detail lessons in rapid field robotics development and testing. Systems were developed and evaluated through real-robot experiments at an outdoor test site in Pittsburgh, Pennsylvania, USA, as well as in the 2020 Mohamed Bin Zayed International Robotics Challenge. All competition trials were completed in fully autonomous mode without RTK-GPS. Our system led to 4th place in Challenge 2 and 7th place in the Grand Challenge, and achievements like popping five balloons (Challenge 1), successfully picking and placing a block (Challenge 2), and dispensing the most water autonomously with a UAV of all teams onto an outdoor, real fire (Challenge 3).Comment: 28 pages, 26 figures. To appear in Field Robotics, Special Issues on MBZIRC 202

Visual Servo Based Space Robotic Docking for Active Space Debris Removal

This thesis developed a 6DOF pose detection algorithm using machine learning capable of providing the orientation and location of an object in various lighting conditions and at different angles, for the purposes of space robotic rendezvous and docking control. The computer vision algorithm was paired with a virtual robotic simulation to test the feasibility of using the proposed algorithm for visual servo. This thesis also developed a method for generating virtual training images and corresponding ground truth data including both location and orientation information. Traditional computer vision techniques struggle to determine the 6DOF pose of an object when certain colors or edges are not found, therefore training a network is an optimal choice. The 6DOF pose detection algorithm was implemented on MATLAB and Python. The robotic simulation was implemented on Simulink and ROS Gazebo. Finally, the generation of training data was done with Python and Blender

Design and Operational Elements of the Robotic Subsystem for the e.deorbit Debris Removal Mission

This paper presents a robotic capture concept that was developed as part of the e.deorbit study by ESA. The defective and tumbling satellite ENVISAT was chosen as a potential target to be captured, stabilized, and subsequently de-orbited in a controlled manner. A robotic capture concept was developed that is based on a chaser satellite equipped with a seven degrees-of-freedom dexterous robotic manipulator, holding a dedicated linear two-bracket gripper. The satellite is also equipped with a clamping mechanism for achieving a stiff fixation with the grasped target, following their combined satellite-stack de-tumbling and prior to the execution of the de-orbit maneuver. Driving elements of the robotic design, operations and control are described and analyzed. These include pre and post-capture operations, the task-specific kinematics of the manipulator, the intrinsic mechanical arm flexibility and its effect on the arm's positioning accuracy, visual tracking, as well as the interaction between the manipulator controller and that of the chaser satellite. The kinematics analysis yielded robust reachability of the grasp point. The effects of intrinsic arm flexibility turned out to be noticeable but also effectively scalable through robot joint speed adaption throughout the maneuvers. During most of the critical robot arm operations, the internal robot joint torques are shown to be within the design limits. These limits are only reached for a limiting scenario of tumbling motion of ENVISAT, consisting of an initial pure spin of 5 deg/s about its unstable intermediate axis of inertia. The computer vision performance was found to be satisfactory with respect to positioning accuracy requirements. Further developments are necessary and are being pursued to meet the stringent mission-related robustness requirements. Overall, the analyses conducted in this study showed that the capture and de-orbiting of ENVISAT using the proposed robotic concept is feasible with respect to relevant mission requirements and for most of the operational scenarios considered. Future work aims at developing a combined chaser-robot system controller. This will include a visual servo to minimize the positioning errors during the contact phases of the mission (grasping and clamping). Further validation of the visual tracking in orbital lighting conditions will be pursued