Real-time model-based video stabilization for microaerial vehicles

The emerging branch of micro aerial vehicles (MAVs) has attracted a great interest for their indoor navigation capabilities, but they require a high quality video for tele-operated or autonomous tasks. A common problem of on-board video quality is the effect of undesired movements, so different approaches solve it with both mechanical stabilizers or video stabilizer software. Very few video stabilizer algorithms in the literature can be applied in real-time but they do not discriminate at all between intentional movements of the tele-operator and undesired ones. In this paper, a novel technique is introduced for real-time video stabilization with low computational cost, without generating false movements or decreasing the performance of the stabilized video sequence. Our proposal uses a combination of geometric transformations and outliers rejection to obtain a robust inter-frame motion estimation, and a Kalman filter based on an ANN learned model of the MAV that includes the control action for motion intention estimation.Peer ReviewedPostprint (author's final draft

Online Digital Image Stabilization for an Unmanned Aerial Vehicle (UAV)

The Unmanned Aerial Vehicle (UAV) video system uses a portable camera mounted on the robot to monitor scene activities. In general, UAVs have very little stabilization equipment, so getting good and stable images of UAVs in real-time is still a challenge. This paper presents a novel framework for digital image stabilization for online applications using a UAV. This idea aims to solve the problem of unwanted vibration and motion when recording video using a UAV. The proposed method is based on dense optical flow to select features representing the displacement of two consecutive frames. K-means clustering is used to find the cluster of the motion vector field that has the largest members. The centroid of the largest cluster was chosen to estimate the rigid transform motion that handles rotation and translation. Then, the trajectory is compensated using the Kalman filter. The experimental results show that the proposed method is suitable for online video stabilization and achieves an average computation time performance of 47.5 frames per second (fps)

System for deployment of groups of unmanned micro aerial vehicles in GPS-denied environments using onboard visual relative localization

A complex system for control of swarms of micro aerial vehicles (MAV), in literature also called as unmanned aerial vehicles (UAV) or unmanned aerial systems (UAS), stabilized via an onboard visual relative localization is described in this paper. The main purpose of this work is to verify the possibility of self-stabilization of multi-MAV groups without an external global positioning system. This approach enables the deployment of MAV swarms outside laboratory conditions, and it may be considered an enabling technique for utilizing fleets of MAVs in real-world scenarios. The proposed visual-based stabilization approach has been designed for numerous different multi-UAV robotic applications (leader-follower UAV formation stabilization, UAV swarm stabilization and deployment in surveillance scenarios, cooperative UAV sensory measurement) in this paper. Deployment of the system in real-world scenarios truthfully verifies its operational constraints, given by limited onboard sensing suites and processing capabilities. The performance of the presented approach (MAV control, motion planning, MAV stabilization, and trajectory planning) in multi-MAV applications has been validated by experimental results in indoor as well as in challenging outdoor environments (e.g., in windy conditions and in a former pit mine)

Robot salamandra anfibio con locomoción bioinspirada

anfibio con una dinámica de movimiento bioinspirada en la locomoción de la salamandra (Cryptobranchidae). El robot es teleoperado mediante una aplicación para dispositivos móviles (Smartphones, tablets, etc.). Se propone una estructura que permita al robot llevar a cabo dos acciones: caminar y nadar. Los movimientos de una salamandra real se han estimado basándose en una cámara cenital y se ha diseñado un algoritmo de control de locomoción que replique esos movimientos. El desempeño del robot se ha evaluado utilizando como métrica el error cuadrático medio entre el movimiento del robot y de la salamandra obteniendo errores menores al 5 % en los ángulos de movimiento de la espina dorsal. // This paper presents the development of an amphibious robot with a motion dynamics bioinspired on the locomotion of the salamander (Cryptobranchidae). The robot is teleoperated by an application for handled devices. We propose a structure to perform two different motions: walk and swim. We extract the movements from a real salamander by a zenith camera, and a locomotion control algorithm is designed to reply this movements. We evaluate the performance of the robot in comparison with the real animal movements using the RMSE (Root Mean Square Error) as metric of evaluation. We obtain errors less than 5 % in the angles of backbone movement

Motion intention optimization for multirotor robust video stabilization

© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksIn this paper we present an optimization algorithm for simultaneously detecting video freeze and obtaining the minimum number of the frame required in motion intention estimation for real time robust video stabilization on multirotor unmanned aerial vehicles. A combination of a filter and a threshold is used to the video freeze detection, and for optimizing the algorithm, we find the minimum number of frames for motion intention estimation without decrease the performance.Peer ReviewedPostprint (author's final draft

3D environment mapping using the Kinect V2 and path planning based on RRT algorithms

This paper describes a 3D path planning system that is able to provide a solution trajectory for the automatic control of a robot. The proposed system uses a point cloud obtained from the robot workspace, with a Kinect V2 sensor to identify the interest regions and the obstacles of the environment. Our proposal includes a collision-free path planner based on the Rapidly-exploring Random Trees variant (RRT*), for a safe and optimal navigation of robots in 3D spaces. Results on RGB-D segmentation and recognition, point cloud processing, and comparisons between different RRT* algorithms, are presented.Peer ReviewedPostprint (published version

Obstacle avoidance based-visual navigation for micro aerial vehicles

This paper describes an obstacle avoidance system for low-cost Unmanned Aerial Vehicles (UAVs) using vision as the principal source of information through the monocular onboard camera. For detecting obstacles, the proposed system compares the image obtained in real time from the UAV with a database of obstacles that must be avoided. In our proposal, we include the feature point detector Speeded Up Robust Features (SURF) for fast obstacle detection and a control law to avoid them. Furthermore, our research includes a path recovery algorithm. Our method is attractive for compact MAVs in which other sensors will not be implemented. The system was tested in real time on a Micro Aerial Vehicle (MAV), to detect and avoid obstacles in an unknown controlled environment; we compared our approach with related works.Peer ReviewedPostprint (published version

Robot salamandra anfibio con locomoción bioinspirada

This paper presents the development of an amphibious robot with a motion dynamics bioinspired on the locomotion of the salamander (Cryptobranchidae). The robot is teleoperated by an application for handled devices. We propose a structure to perform two different motions: walk and swim. We extract the movements from a real salamander by a zenith camera, and a locomotion control algorithm is designed to reply this movements. We evaluate the performance of the robot in comparison with the real animal movements using the RMSE (Root Mean Square Error) as metric of evaluation. We obtain errors less than 5 % in the angles of backbone movement.En este artículo se presenta el desarrollo de un robot anfibio con una dinámica de movimiento bioinspirada en la locomoción de la salamandra (Cryptobranchidae). El robot es teleoperado mediante una aplicación para dispositivos móviles (Smartphones, tablets, etc.). Se propone una estructura que permita al robot llevar a cabo dos acciones: caminar y nadar. Los movimientos de una salamandra real se han estimado basándose en una cámara cenital y se ha diseñado un algoritmo de control de locomoción que replique esos movimientos. El desempeño del robot se ha evaluado utilizando como métrica el error cuadrático medio entre el movimiento del robot y de la salamandra obteniendo errores menores al 5 % en los ángulos de movimiento de la espina dorsal

Localization of Unmanned Aerial Vehicles Using an Optical Flow in Camera Images

Existuje mnoho technik navigace bezpilotních letadel v prostředích bez dostupnosti GPS. Kalkulace optického toku na palubě pomocí jedné kamery poskytuje uživateli rychle nasaditelné a spolehlivé řešení. Cílem této práce bylo vytvořit náhradu pro populární senzor PX4FLOW Smart Camera, který je zatížen mnoha nevýhodami, a integrovat vzniklé řešení na UAV platformu. Použili jsme fázovou korelaci pro odhad optického toku, pro následné zpracování byla použita metoda inspirována algoritmem RANSAC. Řešení bylo otestováno na datech z reálného světa a porovnáno se senzorem PX4FLOW. Byli jsme schopni poskytnout výrazně větší přesnost a spolehlivost měření horizontální rychlosti v rámci našich testů. Dále byla také vytvořena a otestována metoda pro určení vertikální rychlosti a rychlosti rotace, která používá odhadnutý optický tok z více částí obrazu. Testy na datech z reálného světa ukázaly, že přesnost měření rotace je dostatečná pro praktické použití. To umožňuje, aby metoda byla nasazena i v prostředí, kde není možné používat kompas např. v železobetonových budovách.Navigation of Unmanned Aerial Vehicles in GPS-denied environments can be done with multiple techniques. On-board optical flow calculation using single camera gives the user fast-deployable and reliable solution. The goal of this work was to create a replacement for popular PX4FLOW Smart Camera, which is burdened by many drawbacks, and to integrate the solution onto a UAV platform. We used Phase correlation for optical flow estimation and a RANSAC-inspired post-processing method. The solution was tested on real-world datasets and compared with PX4FLOW sensor. We were able to provide significantly higher accuracy and reliability of horizontal speed measurement in our tests. Moreover, a method for yaw rate and vertical velocity measurement using optical flow in different parts of the image was designed and tested. Tests on real-world datasets showed that the accuracy of the yaw rate estimation method was good enough for practical applications. This makes the method open for usage in magnetometer-denied environments such as reinforced concrete buildings

System Identification of a Micro Aerial Vehicle

The purpose of this thesis was to implement an Model Predictive Control based system identification method on a micro-aerial vehicle (DJI Matrice 100) as outlined in a study performed by ETH Zurich. Through limited test flights, data was obtained that allowed for the generation of first and second order system models. The first order models were robust, but the second order model fell short due to the fact that the data used for the model was not sufficient