Motion planning with dynamics awareness for long reach manipulation in aerial robotic systems with two arms

Human activities in maintenance of industrial plants pose elevated risks as well as significant costs due to the required shutdowns of the facility. An aerial robotic system with two arms for long reach manipulation in cluttered environments is presented to alleviate these constraints. The system consists of a multirotor with a long bar extension that incorporates a lightweight dual arm in the tip. This configuration allows aerial manipulation tasks even in hard-to-reach places. The objective of this work is the development of planning strategies to move the aerial robotic system with two arms for long reach manipulation in a safe and efficient way for both navigation and manipulation tasks. The motion planning problem is addressed considering jointly the aerial platform and the dual arm in order to achieve wider operating conditions. Since there exists a strong dynamical coupling between the multirotor and the dual arm, safety in obstacle avoidance will be assured by introducing dynamics awareness in the operation of the planner. On the other hand, the limited maneuverability of the system emphasizes the importance of energy and time efficiency in the generated trajectories. Accordingly, an adapted version of the optimal Rapidly-exploring Random Tree algorithm has been employed to guarantee their optimality. The resulting motion planning strategy has been evaluated through simulation in two realistic industrial scenarios, a riveting application and a chimney repairing task. To this end, the dynamics of the aerial robotic system with two arms for long reach manipulation has been properly modeled, and a distributed control scheme has been derived to complete the test bed. The satisfactory results of the simulations are presented as a first validation of the proposed approach.Unión Europea H2020-644271Ministerio de Ciencia, Innovación y Universidades DPI2014-59383-C2-1-

Continuous Autonomous UAV Inspection for FPSO vessels

This Master's thesis represents the preliminary design study and proposes the unmanned aerial vehicle (UAV) -based inspection framework, comprising several multirotors with automatic charging and deployment for 24/7 integrity inspection tasks. This project has three main topics. First one describes the operational environment and existing regulations that cover use of UAVs. It forms the basis for proposal of the relevant use-case scenarios. Third part comprises two chapters, where design of concept and framework is being based on the previous factors. It shows that before implementation of fully autonomous inspection system, there is a need to cover both regulatory and technical gaps. It can be explained by the fact that there does not exist any autonomous inspection system today. Thus, this project can be seen as a base for future development of the UAV-based inspection system, as it focuses on creation of a general framework

Assisted Control for Semi-Autonomous Power Infrastructure Inspection using Aerial Vehicles

This paper presents the design and implementation of an assisted control technology for a small multirotor platform for aerial inspection of fixed energy infrastructure. Sensor placement is supported by a theoretical analysis of expected sensor performance and constrained platform behaviour to speed up implementation. The optical sensors provide relative position information between the platform and the asset, which enables human operator inputs to be autonomously adjusted to ensure safe separation. The assisted control approach is designed to reduced operator workload during close proximity inspection tasks, with collision avoidance and safe separation managed autonomously. The energy infrastructure includes single vertical wooden poles and crossarm with attached overhead wires. Simulated and real experimental results are provided.Comment: to appear in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018

Optimal Multi-UAV Trajectory Planning for Filming Applications

Teams of multiple Unmanned Aerial Vehicles (UAVs) can be used to record large-scale outdoor scenarios and complementary views of several action points as a promising system for cinematic video recording. Generating the trajectories of the UAVs plays a key role, as it should be ensured that they comply with requirements for system dynamics, smoothness, and safety. The rise of numerical methods for nonlinear optimization is finding a ourishing field in optimization-based approaches to multi- UAV trajectory planning. In particular, these methods are rather promising for video recording applications, as they enable multiple constraints and objectives to be formulated, such as trajectory smoothness, compliance with UAV and camera dynamics, avoidance of obstacles and inter-UAV con icts, and mutual UAV visibility. The main objective of this thesis is to plan online trajectories for multi-UAV teams in video applications, formulating novel optimization problems and solving them in real time. The thesis begins by presenting a framework for carrying out autonomous cinematography missions with a team of UAVs. This framework enables media directors to design missions involving different types of shots with one or multiple cameras, running sequentially or concurrently. Second, the thesis proposes a novel non-linear formulation for the challenging problem of computing optimal multi-UAV trajectories for cinematography, integrating UAV dynamics and collision avoidance constraints, together with cinematographic aspects such as smoothness, gimbal mechanical limits, and mutual camera visibility. Lastly, the thesis describes a method for autonomous aerial recording with distributed lighting by a team of UAVs. The multi-UAV trajectory optimization problem is decoupled into two steps in order to tackle non-linear cinematographic aspects and obstacle avoidance at separate stages. This allows the trajectory planner to perform in real time and to react online to changes in dynamic environments. It is important to note that all the methods in the thesis have been validated by means of extensive simulations and field experiments. Moreover, all the software components have been developed as open source.Los equipos de vehículos aéreos no tripulados (UAV) son sistemas prometedores para grabar eventos cinematográficos, en escenarios exteriores de grandes dimensiones difíciles de cubrir o para tomar vistas complementarias de diferentes puntos de acción. La generación de trayectorias para este tipo de vehículos desempeña un papel fundamental, ya que debe garantizarse que se cumplan requisitos dinámicos, de suavidad y de seguridad. Los enfoques basados en la optimización para la planificación de trayectorias de múltiples UAVs se pueden ver beneficiados por el auge de los métodos numéricos para la resolución de problemas de optimización no lineales. En particular, estos métodos son bastante prometedores para las aplicaciones de grabación de vídeo, ya que permiten formular múltiples restricciones y objetivos, como la suavidad de la trayectoria, el cumplimiento de la dinámica del UAV y de la cámara, la evitación de obstáculos y de conflictos entre UAVs, y la visibilidad mutua. El objetivo principal de esta tesis es planificar trayectorias para equipos multi-UAV en aplicaciones de vídeo, formulando novedosos problemas de optimización y resolviéndolos en tiempo real. La tesis comienza presentando un marco de trabajo para la realización de misiones cinematográficas autónomas con un equipo de UAVs. Este marco permite a los directores de medios de comunicación diseñar misiones que incluyan diferentes tipos de tomas con una o varias cámaras, ejecutadas de forma secuencial o concurrente. En segundo lugar, la tesis propone una novedosa formulación no lineal para el difícil problema de calcular las trayectorias óptimas de los vehículos aéreos no tripulados en cinematografía, integrando en el problema la dinámica de los UAVs y las restricciones para evitar colisiones, junto con aspectos cinematográficos como la suavidad, los límites mecánicos del cardán y la visibilidad mutua de las cámaras. Por último, la tesis describe un método de grabación aérea autónoma con iluminación distribuida por un equipo de UAVs. El problema de optimización de trayectorias se desacopla en dos pasos para abordar los aspectos cinematográficos no lineales y la evitación de obstáculos en etapas separadas. Esto permite al planificador de trayectorias actuar en tiempo real y reaccionar en línea a los cambios en los entornos dinámicos. Es importante señalar que todos los métodos de la tesis han sido validados mediante extensas simulaciones y experimentos de campo. Además, todos los componentes del software se han desarrollado como código abierto

Safe local aerial manipulation for the installation of devices on power lines: Aerial-core first year results and designs

Article number 6220The power grid is an essential infrastructure in any country, comprising thousands of kilometers of power lines that require periodic inspection and maintenance, carried out nowadays by human operators in risky conditions. To increase safety and reduce time and cost with respect to conventional solutions involving manned helicopters and heavy vehicles, the AERIAL-CORE project proposes the development of aerial robots capable of performing aerial manipulation operations to assist human operators in power lines inspection and maintenance, allowing the installation of devices, such as bird flight diverters or electrical spacers, and the fast delivery and retrieval of tools. This manuscript describes the goals and functionalities to be developed for safe local aerial manipulation, presenting the preliminary designs and experimental results obtained in the first year of the project.European Union (UE). H2020 871479Ministerio de Ciencia, Innovación y Universidades de España FPI 201

Impact of UAV Hardware Options on Bridge Inspection Mission Capabilities

Uncrewed Aerial Vehicles (UAV) constitute a rapidly evolving technology field that is becoming more accessible and capable of supplementing, expanding, and even replacing some traditionally manual bridge inspections. Given the classification of the bridge inspection types as initial, routine, in-depth, damage, special, and fracture critical members, specific UAV mission requirements can be developed, and their suitability for UAV application examined. Results of a review of 23 applications of UAVs in bridge inspections indicate that mission sensor and payload needs dictate the UAV configuration and size, resulting in quadcopter configurations being most suitable for visual camera inspections (43% of visual inspections use quadcopters), and hexa- and octocopter configurations being more suitable for higher payload hyperspectral, multispectral, and Light Detection and Ranging (LiDAR) inspections (13%). In addition, the number of motors and size of the aircraft are the primary drivers in the cost of the vehicle. 75% of vehicles rely on GPS for navigation, and none of them are capable of contact inspections. Factors that limit the use of UAVs in bridge inspections include the UAV endurance, the capability of navigation in GPS deprived environments, the stability in confined spaces in close proximity to structural elements, and the cost. Current research trends in UAV technologies address some of these limitations, such as obstacle detection and avoidance methods, autonomous flight path planning and optimization, and UAV hardware optimization for specific mission requirements

경첩문을 여는 비행 매니퓰레이터에 대한 모델 예측 제어

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 기계항공공학부, 2020. 8. 김현진.From aerial pick-and-place to aerial transportation, aerial manipulation has been extensively studied in recent years thanks to its mobility and dexterity, each of which is a merit of an aerial vehicle and a robotic arm. However, to fully harness the concept of aerial manipulation, more complex tasks including interaction with movable structures should also be considered. Among various types of movable structures, this paper presents a multirotor-based aerial manipulator opening a daily-life moving structure, a hinged door. Two additional issues that would arise in interaction with a movable structure are addressed: 1) a constrained motion of the structure, and 2) collision avoidance with a moving structure. To handle these issues, we formulate a model predictive control (MPC) problem with a system dynamics constraint and state constraints for collision avoidance. A coupled system dynamics of a multirotor-based aerial manipulator and a hinged door is derived and later simplified for faster computation in MPC. State constraints for collision avoidance with itself, a door, and a doorframe are generated. By implementing a constrained version of differential dynamic programming (DDP), we can generate reference trajectories through MPC in real-time. The proposed method is validated through simulation results and experiments with a real-like hinged door in which a disturbance observer (DOB) based robust motion controller is employed.비행 매니퓰레이터는 3차원 공간 속에 빠르게 위치할 수 있는 비행체의 장점과 외부와의 상호작용이 가능한 로봇팔의 장점이 결합된 비행체로, 최근 물건 집고 옮기기부터 물품 운송까지 다양한 임무를 수행하기 위해 활발하게 연구되어 왔다. 그러나, 온전히 비행 매니퓰레이터의 가능성을 활용하기 위해서는 움직일 수 있는 외부 구조와의 상호작용과 같이 더욱 복잡한 임무 또한 수행할 수 있어야 할 것이다. 여러 종류의 움직일 수 있는 구조물 중 본 논문에서는 일상 속에서 쉽게 마주칠 수 있는 경첩문을 여는 멀티로터 기반의 비행 매니퓰레이터에 대해 제시한다. 정적인 구조물과의 상호작용과는 달리 동적인 구조물과의 상호작용에 있어서 발생할 수 있는 1) 구조물의 제약된 움직임, 그리고 2) 움직이는 구조물과의 충돌 회피의 2가지 추가적인 문제에 대해 다루었다. 이러한 문제를 다루기 위해 모델 예측 제어 (MPC)를 적용하였으며, 시스템 동역학에 대한 제약조건 및 충돌 회피에 대한 제약 조건을 부여하였다. 멀티로터 기반의 비행 매니퓰레이터와 경첩문의 결합 시스템에 대한 동역학을 유도하였으며, 이후 모델 예측 제어에서의 빠른 계산 속도를 위해 단순화되었다. 충돌 회피에 대한 제약 조건은 모두 상태 변수로 표현되었으며, 비행 매니퓰레이터의 멀티로터 프레임과 로봇팔 사이의 충돌 (자기 충돌), 문과의 충돌, 그리고 문틀과의 충돌을 고려하였다. 미분 기반의 동적 프로그래밍 기법 (differential dynamic programming)에 제약조건이 고려된 알고리즘을 구현함으로써 모델 기반 예측 제어를 통해 실시간으로 경로를 계획할 수 있다. 제안된 방법은 시뮬레이션과 실제 크기의 문을 활용한 실험을 통해 검증되었으며, 외란 관측기 기반의 강건 제어 기법이 실험에 활용되었다.1 Introduction 1 1.1 Literature review 2 1.2 Thesis contribution 3 1.3 Thesis outline 3 2 Equations of motion 4 2.1 Assumption 4 2.2 Kinematics 5 2.3 Dynamics 6 2.4 Simpli ed dynamics 8 3 Model predictive control 10 3.1 Problem formulation 10 3.1.1 Objective function 11 3.1.2 Hard constraints 11 3.2 Collision avoidance constraints 11 3.2.1 Self collision avoidance 13 3.2.2 Door collision avoidance 13 3.2.3 Doorframe collision avoidance 14 3.3 Optimal control solver 14 3.3.1 Differential dynamic programming 14 3.3.2 DDP with augmented Lagrangian method 15 4 Experimental setup 17 4.1 Door state estimation 17 4.2 Multirotor robust controller 18 4.3 Hardware setup 19 5 Results 20 5.1 Simulation results 20 5.2 Experimental results 25 6 Conclusion 29Maste

Aerial Robotics for Inspection and Maintenance

Aerial robots with perception, navigation, and manipulation capabilities are extending the range of applications of drones, allowing the integration of different sensor devices and robotic manipulators to perform inspection and maintenance operations on infrastructures such as power lines, bridges, viaducts, or walls, involving typically physical interactions on flight. New research and technological challenges arise from applications demanding the benefits of aerial robots, particularly in outdoor environments. This book collects eleven papers from different research groups from Spain, Croatia, Italy, Japan, the USA, the Netherlands, and Denmark, focused on the design, development, and experimental validation of methods and technologies for inspection and maintenance using aerial robots