Optimal scheduling for refueling multiple autonomous aerial vehicles

The scheduling, for autonomous refueling, of multiple unmanned aerial vehicles (UAVs) is posed as a combinatorial optimization problem. An efficient dynamic programming (DP) algorithm is introduced for finding the optimal initial refueling sequence. The optimal sequence needs to be recalculated when conditions change, such as when UAVs join or leave the queue unexpectedly. We develop a systematic shuffle scheme to reconfigure the UAV sequence using the least amount of shuffle steps. A similarity metric over UAV sequences is introduced to quantify the reconfiguration effort which is treated as an additional cost and is integrated into the DP algorithm. Feasibility and limitations of this novel approach are also discussed

An Image Based Visual Servo Method for Probe-and-Drogue Autonomous Aerial Refueling

With the high focus on autonomous aerial refueling recently, it becomes increasingly urgent to design efficient methods or algorithms to solve AAR problems in complicated aerial environments. Apart from the complex aerodynamic disturbance, another problem is the pose estimation error caused by the camera calibration error, installation error, or 3D object modeling error, which may not satisfy the highly accurate docking. The main objective of the effort described in this paper is the implementation of an image-based visual servo control method, which contains the establishment of an image-based visual servo model involving the receiver's dynamics and the design of the corresponding controller. Simulation results indicate that the proposed method can make the system dock successfully under complicated conditions and improve the robustness against pose estimation error

Evaluation of machine vision techniques for use within flight control systems

In this thesis, two of the main technical limitations for a massive deployment of Unmanned Aerial Vehicle (UAV) have been considered.;The Aerial Refueling problem is analyzed in the first section. A solution based on the integration of \u27conventional\u27 GPS/INS and Machine Vision sensor is proposed with the purpose of measuring the relative distance between a refueling tanker and UAV. In this effort, comparisons between Point Matching (PM) algorithms and Pose Estimation (PE) algorithms have been developed in order to improve the performance of the Machine Vision sensor. A method of integration based on Extended Kalman Filter (EKF) between GPS/INS and Machine Vision system is also developed with the goal of reducing the tracking error in the \u27pre-contact\u27 to contact and refueling phases.;In the second section of the thesis the issue of Collision Identification (CI) is addressed. A proposed solution consists on the use of Optical Flow (OF) algorithms for the detection of possible collisions in the range of vision of a single camera. The effort includes a study of the performance of different Optical Flow algorithms in different scenarios as well as a method to compute the ideal optical flow with the aim of evaluating the algorithms. An analysis on the suitability for a future real time implementation is also performed for all the analyzed algorithms.;Results of the tests show that the Machine Vision technology can be used to improve the performance in the Aerial Refueling problem. In the Collision Identification problem, the Machine Vision has to be integrated with standard sensors in order to be used inside the Flight Control System

Addressing corner detection issues for machine vision based UAV aerial refueling

The need for developing autonomous aerial refueling capabilities for an Unmanned Aerial Vehicle (UAV) has risen out of the growing importance of UAVs in military and non-military applications. The AAR capabilities would improve the range and the loiter time capabilities of UAVs. A number of AAR techniques have been proposed, based on GPS based measurements and Machine Vision based measurements. The GPS based measurements suffer from distorted data in the wake of the tanker. The MV based techniques proposed the use of optical markers which---when detected---were used to determine relative orientation and position of the tanker and the UAV. The drawback of the MV based techniques is the assumption that all the optical markers are always visible and functional. This research effort proposes an alternative approach where the pose estimation does not depend on optical markers but on Feature Extraction methods. The thesis describes the results of the analysis of specific \u27corner detection\u27 algorithms within a Machine Vision---based approach for the problem of Aerial Refueling for Unmanned Aerial Vehicles. Specifically, the performances of the SUSAN and the Harris corner detection algorithms have been compared. Special emphasis was placed on evaluating their accuracy, the required computational effort, and the robustness of both methods to different sources of noise. Closed loop simulations were performed using a detailed SimulinkRTM -based simulation environment to reproduce docking maneuvers, using the US Air Force refueling boom

Docking control for probe-drogue refueling: An additive-state-decomposition-based output feedback iterative learning control method

Designing a controller for the docking maneuver in Probe-Drogue Refueling (PDR) is an important but challenging task, due to the complex system model and the high precision requirement. In order to overcome the disadvantage of only feedback control, a feedforward control scheme known as Iterative Learning Control (ILC) is adopted in this paper. First, Additive State Decomposition (ASD) is used to address the tight coupling of input saturation, nonlinearity and the property of NonMinimum Phase (NMP) by separating these features into two subsystems (a primary system and a secondary system). After system decomposition, an adjoint-type ILC is applied to the Linear Time-Invariant (LTI) primary system with NMP to achieve entire output trajectory tracking, whereas state feedback is used to stabilize the secondary system with input saturation. The two controllers designed for the two subsystems can be combined to achieve the original control goal of the PDR system. Furthermore, to compensate for the receiver-independent uncertainties, a correction action is proposed by using the terminal docking error, which can lead to a smaller docking error at the docking moment. Simulation tests have been carried out to demonstrate the performance of the proposed control method, which has some advantages over the traditional derivative-type ILC and adjoint-type ILC in the docking control of PDR

Survey of computer vision algorithms and applications for unmanned aerial vehicles

This paper presents a complete review of computer vision algorithms and vision-based intelligent applications, that are developed in the field of the Unmanned Aerial Vehicles (UAVs) in the latest decade. During this time, the evolution of relevant technologies for UAVs; such as component miniaturization, the increase of computational capabilities, and the evolution of computer vision techniques have allowed an important advance in the development of UAVs technologies and applications. Particularly, computer vision technologies integrated in UAVs allow to develop cutting-edge technologies to cope with aerial perception difficulties; such as visual navigation algorithms, obstacle detection and avoidance and aerial decision-making. All these expert technologies have developed a wide spectrum of application for UAVs, beyond the classic military and defense purposes. Unmanned Aerial Vehicles and Computer Vision are common topics in expert systems, so thanks to the recent advances in perception technologies, modern intelligent applications are developed to enhance autonomous UAV positioning, or automatic algorithms to avoid aerial collisions, among others. Then, the presented survey is based on artificial perception applications that represent important advances in the latest years in the expert system field related to the Unmanned Aerial Vehicles. In this paper, the most significant advances in this field are presented, able to solve fundamental technical limitations; such as visual odometry, obstacle detection, mapping and localization, et cetera. Besides, they have been analyzed based on their capabilities and potential utility. Moreover, the applications and UAVs are divided and categorized according to different criteria.This research is supported by the Spanish Government through the CICYT projects (TRA2015-63708-R and TRA2013-48314-C3-1-R)

• Addressing pose estimation issues for application of machine vision to UAV Autonomous Aerial Refueling

ABSTRACT The purpose of this thesis is to describe the results of an effort on the analysis of the performance of specific algorithms for the ‘pose estimation’ problem within the context of applying Machine Vision-based control laws for the problem of Autonomous Aerial Refueling (AAR) for UAVs. It is assumed that the MV-based AAR approach features several optical markers installed on specific points of the refueling tanker. However, the approach can be applied without any loss of generality to the more general case of the use feature extraction methods to detect specific points and corners of the tanker in lieu of the optical markers. The document proposes a robust ‘detection and labeling algorithm’ for the correct identification of the optical markers, which is then provided to the ‘pose estimation’ algorithm. Furthermore, a detailed study of the performance of two specific ‘pose estimation’ algorithms (the GLSDC and the LHM algorithms) is performed with special emphasis on required computational effort, robustness, error propagation. Extensive simulation studies demonstrate the potential of the LHM algorithm and also highlight the importance of the robustness of the ‘detection and labeling’ algorithm. The simulation effort is performed with a detailed modeling of the AAR maneuver using the USAF refueling method. SOMMARIO Lo scopo di questa tesi è descrivere i risultati di uno studio riguardante l’analisi delle prestazioni di specifici algoritmi per il problema della stima della posizione in un contesto applicato ad una legge di controllo basata su Machine Vision (MV) per il problema del rifornimento aereo in modo autonomo (AAR) per veicoli aerei non pilotati (UAVs). Si assume che l’avvicinamento durante il rifornimento avvenga grazie a diversi markers ottici installati in punti specifici dell’aeromobile che fornisce il carburante (Tanker). Il metodo può comunque essere usato senza perdita di generalità in un caso dove si usa il metodo della feature extraction per trovare dei punti specifici sul contorno del Tanker al posto dei markers ottici. Il documento propone un algoritmo robusto per la corretta identificazione dei markers ottici, i quali vengono poi forniti agli algoritmi di stima della posizione. Inoltre si propone uno studio dettagliato di due specifici algoritmi per la stima della posizione (il GLSDC e LHM) dove si da particolare importanza al peso computazione, la robustezza e la propagazione dell’errore. Numerose simulazioni dimostrano il potenziale dell’algoritmo LHM evidenziando l’importanza della robustezza e dell’algoritmo di identificazione dei markers ottici. Le simulazioni sono state eseguite con un modello dettagliato della manovra di AAR usando il metodo proposto da USAF

Avionics and control system development for mid-air rendezvous of two unmanned aerial vehicles

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2004.Includes bibliographical references (p. 177-181).This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.A flight control system was developed to achieve mid-air rendezvous of two unmanned aerial vehicles (UAVs) as a part of the Parent Child Unmanned Aerial Vehicle (PCUAV) project at MIT and the Draper Laboratory. A lateral guidance logic was developed for tightly tracking a desired flight path. The guidance logic is derived from geometric and kinematic properties, and has been demonstrated to work better than the conventional aircraft guidance method in waypoint navigation. A simple, low-order attitude estimation was developed that combines aircraft kinematics, GPS and low-quality rate gyros. It is demonstrated in simulation that the performance of the proposed method is as good as other advanced complex methods when the aircraft bank angle is relative small(<40 degrees). The end-game control strategy for the final phase of the rendezvous was also developed, using proportional navigation guidance in conjunction with an optical sensor. The associated miss distance was analyzed with regard to the wind effect and initial conditions. A series of flight tests was performed using two UAVs which were built as a part of the project. It was demonstrated that each individual aircraft can follow a desired flight path within a position accuracy of 2 meters (based on sensor data) while also tracking the air speed command to within 1 m/s. At the time of this thesis writing, it has been demonstrated that the developed control system can bring the two UAVs from any arbitrary initial positions into a configuration of a tight formation flight, where one vehicle trails the other with a commanded separation of 12 meters while maintaining the relative position error within 2 meters in both horizontal and vertical directions for 85% of the flight time.by Sanghyuk Park.Ph.D