Visual Servoing of a UGV from a UAV using Differential Flatness

In this paper the problem of controlling the motion of a nonholonomic vehicle along a desired trajectory using observations from an overhead camera is considered. The control problem is formulated in the image plane. We show that the system in the image plane is differentially flat and use this property to generate effective control strategies using only visual feedback. Simulation results illustrate the methodology and show robustness to errors in the camera calibration parameters

Teleoperating a mobile manipulator and a free-flying camera from a single haptic device

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksThe paper presents a novel teleoperation system that allows the simultaneous and continuous command of a ground mobile manipulator and a free flying camera, implemented using an UAV, from which the operator can monitor the task execution in real-time. The proposed decoupled position and orientation workspace mapping allows the teleoperation from a single haptic device with bounded workspace of a complex robot with unbounded workspace. When the operator is reaching the position and orientation boundaries of the haptic workspace, linear and angular velocity components are respectively added to the inputs of the mobile manipulator and the flying camera. A user study on a virtual environment has been conducted to evaluate the performance and the workload on the user before and after proper training. Analysis on the data shows that the system complexity is not an obstacle for an efficient performance. This is a first step towards the implementation of a teleoperation system with a real mobile manipulator and a low-cost quadrotor as the free-flying camera.Accepted versio

Homography-based pose estimation to guide a miniature helicopter during 3D-trajectory tracking

This work proposes a pose-based visual servoing control, through using planar homography, to estimate the position and orientation of a miniature helicopter relative to a known pattern. Once having the current flight information, the nonlinear underactuated controller presented in one of our previous works, which attends all flight phases, is used to guide the rotorcraft during a 3Dtrajectory tracking task. In the sequel, the simulation framework and the results obtained using it are presented and discussed, validating the proposed controller when a visual system is used to determine the helicopter pose information.Fil: Brandão, Alexandre . Universidade Federal Do Espirito Santo. Centro Tecnologico. Departamento de Ingenieria Electrica; BrasilFil: Sarapura, Jorge Antonio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico San Juan. Instituto de Automática; Argentina. Universidad Nacional de San Juan; ArgentinaFil: Sarcinelli Filho, Mario . Universidade Federal Do Espirito Santo. Centro Tecnologico. Departamento de Ingenieria Electrica; BrasilFil: Carelli Albarracin, Ricardo Oscar. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico San Juan. Instituto de Automática; Argentina. Universidad Nacional de San Juan; Argentin

Risk-aware Path and Motion Planning for a Tethered Aerial Visual Assistant in Unstructured or Confined Environments

This research aims at developing path and motion planning algorithms for a tethered Unmanned Aerial Vehicle (UAV) to visually assist a teleoperated primary robot in unstructured or confined environments. The emerging state of the practice for nuclear operations, bomb squad, disaster robots, and other domains with novel tasks or highly occluded environments is to use two robots, a primary and a secondary that acts as a visual assistant to overcome the perceptual limitations of the sensors by providing an external viewpoint. However, the benefits of using an assistant have been limited for at least three reasons: (1) users tend to choose suboptimal viewpoints, (2) only ground robot assistants are considered, ignoring the rapid evolution of small unmanned aerial systems for indoor flying, (3) introducing a whole crew for the second teleoperated robot is not cost effective, may introduce further teamwork demands, and therefore could lead to miscommunication. This dissertation proposes to use an autonomous tethered aerial visual assistant to replace the secondary robot and its operating crew. Along with a pre-established theory of viewpoint quality based on affordances, this dissertation aims at defining and representing robot motion risk in unstructured or confined environments. Based on those theories, a novel high level path planning algorithm is developed to enable risk-aware planning, which balances the tradeoff between viewpoint quality and motion risk in order to provide safe and trustworthy visual assistance flight. The planned flight trajectory is then realized on a tethered UAV platform. The perception and actuation are tailored to fit the tethered agent in the form of a low level motion suite, including a novel tether-based localization model with negligible computational overhead, motion primitives for the tethered airframe based on position and velocity control, and two differentComment: Ph.D Dissertatio

Risk-aware Path and Motion Planning for a Tethered Aerial Visual Assistant in Unstructured or Confined Environments

This research aims at developing path and motion planning algorithms for a tethered Unmanned Aerial Vehicle (UAV) to visually assist a teleoperated primary robot in unstructured or confined environments. The emerging state of the practice for nuclear operations, bomb squad, disaster robots, and other domains with novel tasks or highly occluded environments is to use two robots, a primary and a secondary that acts as a visual assistant to overcome the perceptual limitations of the sensors by providing an external viewpoint. However, the benefits of using an assistant have been limited for at least three reasons: (1) users tend to choose suboptimal viewpoints, (2) only ground robot assistants are considered, ignoring the rapid evolution of small unmanned aerial systems for indoor flying, (3) introducing a whole crew for the second teleoperated robot is not cost effective, may introduce further teamwork demands, and therefore could lead to miscommunication. This dissertation proposes to use an autonomous tethered aerial visual assistant to replace the secondary robot and its operating crew. Along with a pre-established theory of viewpoint quality based on affordances, this dissertation aims at defining and representing robot motion risk in unstructured or confined environments. Based on those theories, a novel high level path planning algorithm is developed to enable risk-aware planning, which balances the tradeoff between viewpoint quality and motion risk in order to provide safe and trustworthy visual assistance flight. The planned flight trajectory is then realized on a tethered UAV platform. The perception and actuation are tailored to fit the tethered agent in the form of a low level motion suite, including a novel tether-based localization model with negligible computational overhead, motion primitives for the tethered airframe based on position and velocity control, and two different approaches to negotiate tether with complex obstacle-occupied environments. The proposed research provides a formal reasoning of motion risk in unstructured or confined spaces, contributes to the field of risk-aware planning with a versatile planner, and opens up a new regime of indoor UAV navigation: tethered indoor flight to ensure battery duration and failsafe in case of vehicle malfunction. It is expected to increase teleoperation productivity and reduce costly errors in scenarios such as safe decommissioning and nuclear operations in the Fukushima Daiichi facility

Dynamic Landing of an Autonomous Quadrotor on a Moving Platform in Turbulent Wind Conditions

Autonomous landing on a moving platform presents unique challenges for multirotor vehicles, including the need to accurately localize the platform, fast trajectory planning, and precise/robust control. Previous works studied this problem but most lack explicit consideration of the wind disturbance, which typically leads to slow descents onto the platform. This work presents a fully autonomous vision-based system that addresses these limitations by tightly coupling the localization, planning, and control, thereby enabling fast and accurate landing on a moving platform. The platform's position, orientation, and velocity are estimated by an extended Kalman filter using simulated GPS measurements when the quadrotor-platform distance is large, and by a visual fiducial system when the platform is nearby. The landing trajectory is computed online using receding horizon control and is followed by a boundary layer sliding controller that provides tracking performance guarantees in the presence of unknown, but bounded, disturbances. To improve the performance, the characteristics of the turbulent conditions are accounted for in the controller. The landing trajectory is fast, direct, and does not require hovering over the platform, as is typical of most state-of-the-art approaches. Simulations and hardware experiments are presented to validate the robustness of the approach.Comment: 7 pages, 8 figures, ICRA2020 accepted pape

Uncrewed Aerial Vehicle Fruit Picking with Perceptual Imitation Learning Trajectory Generation

This thesis studies the problem of Uncrewed Aerial Vehicle (UAV) path planning and manipulation in unmapped environments. This thesis the specific task of orange picking with a quadrotor UAV. Robotic fruit harvesting is a fitting example problem to tackle in this research, as there is a worldwide need for improved agricultural technologies. This task is difficult because it requires comprehending and navigating a complex, unknown environment. To accomplish this task, we present a novel visual servoing controller which fuses information from onboard camera images with odometry data. This was used to calculate the relative position of an orange and a safe approach angle. By following a series of reference trajectories to the computed goal location, the system was able to grasp an orange autonomously and remove it from the tree. This visual servoing method has several inherent limitations. It cannot search for an occluded orange or handle any paths that remove the orange from its view. To improve upon this approach, and correct these shortcomings, we develop a novel neural network architecture to perform the same task using a learned implicit visual encoding. In the next section, we present the design of a simulation of this same orange picking task, and a Model Predictive Control (MPC) method for computing optimal trajectories within it. We trained the neural network to imitate the MPC expert, validating the network structure and cost function. In the subsequent chapter, we trained the same architecture on a dataset derived from the visual servoing controller. These experiments led to useful innovations in the neural network architecture, but even with these efforts, no network was able to vastly improve on the baseline data. In the final chapter, we discuss the relative strengths and weaknesses of these algorithms. Each has areas where it exceeds the others, and we propose new avenues of research to improve them all