The Right (Angled) Perspective: Improving the Understanding of Road Scenes Using Boosted Inverse Perspective Mapping

Many tasks performed by autonomous vehicles such as road marking detection, object tracking, and path planning are simpler in bird's-eye view. Hence, Inverse Perspective Mapping (IPM) is often applied to remove the perspective effect from a vehicle's front-facing camera and to remap its images into a 2D domain, resulting in a top-down view. Unfortunately, however, this leads to unnatural blurring and stretching of objects at further distance, due to the resolution of the camera, limiting applicability. In this paper, we present an adversarial learning approach for generating a significantly improved IPM from a single camera image in real time. The generated bird's-eye-view images contain sharper features (e.g. road markings) and a more homogeneous illumination, while (dynamic) objects are automatically removed from the scene, thus revealing the underlying road layout in an improved fashion. We demonstrate our framework using real-world data from the Oxford RobotCar Dataset and show that scene understanding tasks directly benefit from our boosted IPM approach.Comment: equal contribution of first two authors, 8 full pages, 6 figures, accepted at IV 201

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Deep Probabilistic Models for Camera Geo-Calibration

The ultimate goal of image understanding is to transfer visual images into numerical or symbolic descriptions of the scene that are helpful for decision making. Knowing when, where, and in which direction a picture was taken, the task of geo-calibration makes it possible to use imagery to understand the world and how it changes in time. Current models for geo-calibration are mostly deterministic, which in many cases fails to model the inherent uncertainties when the image content is ambiguous. Furthermore, without a proper modeling of the uncertainty, subsequent processing can yield overly confident predictions. To address these limitations, we propose a probabilistic model for camera geo-calibration using deep neural networks. While our primary contribution is geo-calibration, we also show that learning to geo-calibrate a camera allows us to implicitly learn to understand the content of the scene

Classification of Human and Vehicles with The Deep Learning Based on Transfer Learning Method

There has been a significant increase in the use of deep learning algorithms in recent years. Convolutional neural network (CNN), one of the deep learning models, is frequently used in applications to distinguish important objects such as humans and vehicles from other objects, especially in image processing. After the ImageNet Large Scale Visual Recognition Competition (ILSVRC) in 2012, the use of ESA in applications is becoming quite common. With the development of image processing hardware, the image processing process is significantly reduced. Thanks to these developments, the performance of studies on deep learning is increasing. In this study, a system based on deep learning has been developed to detect and classify objects (human, car and motorcycle / bicycle) from images captured by drones. Two datasets, the image set of Stanford University and the drone image set created at Afyon Kocatepe University (AKÜ), are used to train and test the deep neural network with the transfer learning method. Training and testing processes are carried out using a total of 3841 images, 2591 from the Stanford dataset and 1250 from the AKÜ dataset. The precision, recall and f1 score values are evaluated according to the process of determining and classifying human, car and motorcycle / bicycle classes using GoogleNet, VggNet and ResNet50 deep learning algorithms. According to this evaluation result, high performance results are obtained with 0.916 precision, 0.895 recall and 0.906 f1 score value in the ResNet50 model