Automatic generation of dense non-rigid optical flow

There hardly exists any large-scale datasets with dense optical flow of non-rigid motion from real-world imagery as of today. The reason lies mainly in the difficulty of human annotation to generate optical flow ground-truth. To circumvent the need for human annotation, we propose a framework to automatically generate optical flow from real-world videos. The method extracts and matches objects from video frames to compute initial constraints, and applies a deformation over the objects of interest to obtain dense optical flow fields. We propose several ways to augment the optical flow variations. Extensive experimental results show that training on our automatically generated optical flow outperforms methods that are trained on rigid synthetic data using FlowNet-S, PWC-Net, and LiteFlowNet. Datasets and algorithms of our optical flow generation framework is available at https://github.com/lhoangan/arap_flow.Comment: The paper is under consideration at Computer Vision and Image Understandin

Outdoor image understanding from multiple vision modalities

The thesis investigates various computer vision modalities, which, taken from the broad definitions, include both sensory data as well as subsequent interpretation such as RGB, depth, intrinsic images, semantic maps, surface normals, optical flow, and point clouds. Specifically, the thesis focuses on the research question how various computer vision modalities can be exploited and combined. The question is tackled from multiple perspectives, starting with decomposing a primary modality, followed by the study of modality complement and combination. The subsequent chapters explore multimodality from a generative perspective, how a modality benefits generation of the others, and concludes with the construction of a multimodal synthetic dataset

DETECTION OF DEGRADED ACACIA TREE SPECIES USING DEEP NEURAL NETWORKS ON UAV DRONE IMAGERY

Abstract. Deep-learning-based image classification and object detection has been applied successfully to tree monitoring. However, studies of tree crowns and fallen trees, especially on flood inundated areas, remain largely unexplored. Detection of degraded tree trunks on natural environments such as water, mudflats, and natural vegetated areas is challenging due to the mixed colour image backgrounds. In this paper, Unmanned Aerial Vehicles (UAVs), or drones, with embedded RGB cameras were used to capture the fallen Acacia Xanthophloea trees from six designated plots around Lake Nakuru, Kenya. Motivated by the need to detect fallen trees around the lake, two well-established deep neural networks, i.e. Faster Region-based Convolution Neural Network (Faster R-CNN) and Retina-Net were used for fallen tree detection. A total of 7,590 annotations of three classes on 256×256 image patches were used for this study. Experimental results show the relevance of deep learning in this context, with Retina-Net model achieving 38.9% precision and 57.9% recall