81 research outputs found
Optimal Multi-UAV Trajectory Planning for Filming Applications
Teams of multiple Unmanned Aerial Vehicles (UAVs) can be used to record large-scale
outdoor scenarios and complementary views of several action points as a promising
system for cinematic video recording. Generating the trajectories of the UAVs plays
a key role, as it should be ensured that they comply with requirements for system
dynamics, smoothness, and safety. The rise of numerical methods for nonlinear
optimization is finding a
ourishing field in optimization-based approaches to multi-
UAV trajectory planning. In particular, these methods are rather promising for
video recording applications, as they enable multiple constraints and objectives to
be formulated, such as trajectory smoothness, compliance with UAV and camera
dynamics, avoidance of obstacles and inter-UAV con
icts, and mutual UAV visibility.
The main objective of this thesis is to plan online trajectories for multi-UAV teams in
video applications, formulating novel optimization problems and solving them in real
time.
The thesis begins by presenting a framework for carrying out autonomous cinematography
missions with a team of UAVs. This framework enables media directors
to design missions involving different types of shots with one or multiple cameras,
running sequentially or concurrently. Second, the thesis proposes a novel non-linear
formulation for the challenging problem of computing optimal multi-UAV trajectories
for cinematography, integrating UAV dynamics and collision avoidance constraints,
together with cinematographic aspects such as smoothness, gimbal mechanical limits,
and mutual camera visibility. Lastly, the thesis describes a method for autonomous
aerial recording with distributed lighting by a team of UAVs. The multi-UAV trajectory
optimization problem is decoupled into two steps in order to tackle non-linear cinematographic aspects and obstacle avoidance at separate stages. This allows the
trajectory planner to perform in real time and to react online to changes in dynamic
environments.
It is important to note that all the methods in the thesis have been validated
by means of extensive simulations and field experiments. Moreover, all the software
components have been developed as open source.Los equipos de vehículos aéreos no tripulados (UAV) son sistemas prometedores para grabar
eventos cinematográficos, en escenarios exteriores de grandes dimensiones difíciles de cubrir
o para tomar vistas complementarias de diferentes puntos de acción. La generación de
trayectorias para este tipo de vehículos desempeña un papel fundamental, ya que debe
garantizarse que se cumplan requisitos dinámicos, de suavidad y de seguridad.
Los enfoques basados en la optimización para la planificación de trayectorias de múltiples
UAVs se pueden ver beneficiados por el auge de los métodos numéricos para la resolución de
problemas de optimización no lineales. En particular, estos métodos son bastante
prometedores para las aplicaciones de grabación de vídeo, ya que permiten formular múltiples
restricciones y objetivos, como la suavidad de la trayectoria, el cumplimiento de la dinámica
del UAV y de la cámara, la evitación de obstáculos y de conflictos entre UAVs, y la visibilidad
mutua.
El objetivo principal de esta tesis es planificar trayectorias para equipos multi-UAV en
aplicaciones de vídeo, formulando novedosos problemas de optimización y resolviéndolos en
tiempo real.
La tesis comienza presentando un marco de trabajo para la realización de misiones
cinematográficas autónomas con un equipo de UAVs. Este marco permite a los directores de
medios de comunicación diseñar misiones que incluyan diferentes tipos de tomas con una o
varias cámaras, ejecutadas de forma secuencial o concurrente. En segundo lugar, la tesis
propone una novedosa formulación no lineal para el difícil problema de calcular las
trayectorias óptimas de los vehículos aéreos no tripulados en cinematografía, integrando en el
problema la dinámica de los UAVs y las restricciones para evitar colisiones, junto con aspectos
cinematográficos como la suavidad, los límites mecánicos del cardán y la visibilidad mutua de
las cámaras. Por último, la tesis describe un método de grabación aérea autónoma con
iluminación distribuida por un equipo de UAVs. El problema de optimización de trayectorias se
desacopla en dos pasos para abordar los aspectos cinematográficos no lineales y la evitación
de obstáculos en etapas separadas. Esto permite al planificador de trayectorias actuar en
tiempo real y reaccionar en línea a los cambios en los entornos dinámicos.
Es importante señalar que todos los métodos de la tesis han sido validados mediante extensas
simulaciones y experimentos de campo. Además, todos los componentes del software se han
desarrollado como código abierto
Optimal Trajectory Planning for Cinematography with Multiple Unmanned Aerial Vehicles
This paper presents a method for planning optimal trajectories with a team of
Unmanned Aerial Vehicles (UAVs) performing autonomous cinematography. The
method is able to plan trajectories online and in a distributed manner,
providing coordination between the UAVs. We propose a novel non-linear
formulation for this challenging problem of computing multi-UAV optimal
trajectories for cinematography; integrating UAVs dynamics and collision
avoidance constraints, together with cinematographic aspects like smoothness,
gimbal mechanical limits and mutual camera visibility. We integrate our method
within a hardware and software architecture for UAV cinematography that was
previously developed within the framework of the MultiDrone project; and
demonstrate its use with different types of shots filming a moving target
outdoors. We provide extensive experimental results both in simulation and
field experiments. We analyze the performance of the method and prove that it
is able to compute online smooth trajectories, reducing jerky movements and
complying with cinematography constraints.Comment: This paper has been published as: Optimal trajectory planning for
cinematography with multiple Unmanned Aerial Vehicles. Alfonso Alcantara and
Jesus Capitan and Rita Cunha and Anibal Ollero. Robotics and Autonomous
Systems. 103778 (2021) 10.1016/j.robot.2021.10377
Deep Reinforcement Learning with semi-expert distillation for autonomous UAV cinematography
Unmanned Aerial Vehicles (UAVs, or drones) have revolutionized modern media production. Being rapidly deployable “flying cameras”, they can easily capture aesthetically pleasing aerial footage of static or moving filming targets/subjects. Current approaches rely either on manual UAV/gimbal control by human experts or on a combination of complex computer vision algorithms and hardware configurations for automating the flight+flying process. This paper explores an efficient Deep Reinforcement Learning (DRL) alternative, which implicitly merges the target detection and path planning steps into a single algorithm. To achieve this, a baseline DRL approach is augmented with a novel policy distillation component, which transfers knowledge from a suitable, semi-expert Model Predictive Control (MPC) controller into the DRL agent. Thus, the latter is able to autonomously execute a specific UAV cinematography task with purely visual input. Unlike the MPC controller, the proposed DRL agent does not need to know the 3D world position of the filming target during inference. Experiments conducted in a photorealistic simulator showcase superior performance and training speed compared to the baseline agent while surpassing the MPC controller in terms of visual occlusion avoidance
Autonomous Execution of Cinematographic Shots with Multiple Drones
This paper presents a system for the execution of autonomous cinematography
missions with a team of drones. The system allows media directors to design
missions involving different types of shots with one or multiple cameras,
running sequentially or concurrently. We introduce the complete architecture,
which includes components for mission design, planning and execution. Then, we
focus on the components related to autonomous mission execution. First, we
propose a novel parametric description for shots, considering different types
of camera motion and tracked targets; and we use it to implement a set of
canonical shots. Second, for multi-drone shot execution, we propose distributed
schedulers that activate different shot controllers on board the drones.
Moreover, an event-based mechanism is used to synchronize shot execution among
the drones and to account for inaccuracies during shot planning. Finally, we
showcase the system with field experiments filming sport activities, including
a real regatta event. We report on system integration and lessons learnt during
our experimental campaigns
Aerial multi-camera robotic jib crane
A formulation based on a team of unmanned aerial vehicles operating as a fully articulated multi-camera jib crane is proposed for the application of aerial cinematography. An optimization-based controller commands the formation to follow an artistic trajectory defined by the director of photography, while actively avoiding collisions and cameras' mutual visibility. The proposed scheme, based on the cluster-space formulation, presents an intuitive way of maneuvering the virtual camera fixture while automatically adjusting the motions by imposing artistic and safety constraints, facilitating the operator task.Fil: Moreno, Patricio. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Saavedra 15. Instituto Argentino de Matemática Alberto Calderón; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería. Departamento de Electronica; ArgentinaFil: Presenza, Juan Francisco. Universidad de Buenos Aires. Facultad de Ingeniería. Departamento de Electronica; ArgentinaFil: Mas, Ignacio Agustin. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Instituto Tecnológico de Buenos Aires; ArgentinaFil: Giribet, Juan Ignacio. Universidad de Buenos Aires. Facultad de Ingeniería. Departamento de Electronica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Saavedra 15. Instituto Argentino de Matemática Alberto Calderón; Argentin
A Multiple-UAV Software Architecture for Autonomous Media Production
The use of UAVs in media production has taken off during the past few years, with increasingly more functions becoming automated. However, current solutions leave a lot to be desired with regard to autonomy and drone fleet support. This paper presents a novel, complete software architecture suited to an intelligent, multiple-UAV platform for media production/cinematography applications, covering outdoor events (e.g., sports) typically distributed over large expanses. Increased multiple drone decisional autonomy, so as to minimize production crew load, and improved multiple drone robustness/safety mechanisms (e.g., regarding communications, flight regulation compliance, crowd avoidance and emergency landing mechanisms) are supported.publishersversionpublishe
- …