Object Detection and Tracking Based on Optical Flow in Unmanned Aerial Vehicles

Abstract

In recent years the Unmanned Aerial Vehicle (UAV) community has discovered the enormous amount of information that can be extracted from a video camera. It can be used for collision avoidance, navigation, velocity estimation, terrain mapping, object detection and object tracking in addition to many other applications. This thesis looks into several different ways a video camera in the payload of a fixed-wing UAV can be utilized. The first part investigates a way to calculate the body-fixed velocity of the UAV with the images captured by the camera. It can be achieved with optical flow and measurements of the roll angle, pitch angle and the altitude of the UAV. The calculated body-fixed velocity can be used as a measurement in the navigation system. This thesis looks into a navigation system based on a nonlinear observer that estimates attitude, position, velocity and gyro bias. In order to estimate these states the nonlinear observer utilizes measurements from an Inertial Measurement Unit (IMU), a Global Navigation Satellite System (GNSS) receiver and a video camera. The second part looks into moving object detection and tracking. An algorithm for detection of moving objects has been developed. It utilizes the navigation states of the UAV and optical flow in order to extract the moving objects from the images. Furthermore a tracking system based on the moving object detection algorithm has been proposed. The tracking system consists of the moving object detection algorithm, a classifier that describes each object and a discrete Kalman filter that estimates the motion of the object. The objects are tracked in the image plane and the estimates are transformed to the North-East-Down (NED) coordinate system. A fixed-wing UAV experiment has been carried out to gather images captured from an airborne UAV and collect data from an IMU and a GPS receiver. The navigation system has been evaluated offline by computer simulations based on the data collected at the experiment. The tracking system and the moving object detection algorithm have also been evaluated in computer simulations. Moving objects have been inserted into the images because moving objects did not exist in the experiment. Promising and accurate results were shown for both the navigation system and the proposed tracking system