A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Road Feature Extraction from High Resolution Aerial Images Upon Rural Regions Based on Multi-Resolution Image Analysis and Gabor Filters

Accurate, detailed and up-to-date road information is of special importance in geo-spatial databases as it is used in a variety of applications such as vehicle navigation, traffic management and advanced driver assistance systems (ADAS). The commercial road maps utilized for road navigation or the geographical information system (GIS) today are based on linear road centrelines represented in vector format with poly-lines (i.e., series of nodes and shape points, connected by segments), which present a serious lack of accuracy, contents, and completeness for their applicability at the sub-road level. For instance, the accuracy level of the present standard maps is around 5 to 20 meters. The roads/streets in the digital maps are represented as line segments rendered using different colours and widths. However, the widths of line segments do not necessarily represent the actual road widths accurately. Another problem with the existing road maps is that few precise sub-road details, such as lane markings and stop lines, are included, whereas such sub-road information is crucial for applications such as lane departure warning or lane-based vehicle navigation. Furthermore, the vast majority of roadmaps aremodelled in 2D space, whichmeans that some complex road scenes, such as overpasses and multi-level road systems, cannot be effectively represented. In addition, the lack of elevation information makes it infeasible to carry out applications such as driving simulation and 3D vehicle navigation

Applications of ISES for vegetation and land use

Remote sensing relative to applications involving vegetation cover and land use is reviewed to consider the potential benefits to the Earth Observing System (Eos) of a proposed Information Sciences Experiment System (ISES). The ISES concept has been proposed as an onboard experiment and computational resource to support advanced experiments and demonstrations in the information and earth sciences. Embedded in the concept is potential for relieving the data glut problem, enhancing capabilities to meet real-time needs of data users and in-situ researchers, and introducing emerging technology to Eos as the technology matures. These potential benefits are examined in the context of state-of-the-art research activities in image/data processing and management

Automation of road feature extraction from high resolution images

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial TechnologiesThe detection of road features from remotely sensed images has become a critical factor in maintaining a reliable and updated road network in a country to provide a base reference for transportation, emergency planning, and navigation. With the recent advances of convolutional neural networks in image processing, several publications are devoted to the development of a method for automatically extract roads from satellite images. However, a reliable feature extraction method has not yet been developed with the desired accuracy and precision, and always seems to be a proportionality between the accuracy and the complexity of these developed methods. The aim of this study was therefore to develop an accurate road extraction method without compromising computational efficiency. In this paper, a semantic segmentation neural network that combines the strengths of transfer learning and U-net architecture is proposed with a minimal network complexity. Further, post-processing based on morphological operations and regional properties of the extracted segments were used to remove the noises from the final output. The results have been compared with different automatic classification and segmentation methods and the results of the proposed method produced an F1 score of 0.83 and high accuracy of 95.57%, more accurate and precise than all the other models for the freely available Massachusetts dataset. Finally, the developed method stood superior to the preexisting methods in terms of performance measure and network complexity

Combining Multiple Algorithms for Road Network Tracking from Multiple Source Remotely Sensed Imagery: a Practical System and Performance Evaluation

In light of the increasing availability of commercial high-resolution imaging sensors, automatic interpretation tools are needed to extract road features. Currently, many approaches for road extraction are available, but it is acknowledged that there is no single method that would be successful in extracting all types of roads from any remotely sensed imagery. In this paper, a novel classification of roads is proposed, based on both the roads' geometrical, radiometric properties and the characteristics of the sensors. Subsequently, a general road tracking framework is proposed, and one or more suitable road trackers are designed or combined for each type of roads. Extensive experiments are performed to extract roads from aerial/satellite imagery, and the results show that a combination strategy can automatically extract more than 60% of the total roads from very high resolution imagery such as QuickBird and DMC images, with a time-saving of approximately 20%, and acceptable spatial accuracy. It is proven that a combination of multiple algorithms is more reliable, more efficient and more robust for extracting road networks from multiple-source remotely sensed imagery than the individual algorithms

A new asymmetrical corner detector(ACD) for a semi-automatic image co-registration scheme

Co-registration of multi-sensor and multi-temporal images is essential for remote sensing applications. In the image co-registration process, automatic Ground Control Points (GCPs) selection is a key technical issue and the accuracy of GCPs localization largely accounts for the final image co-registration accuracy. In this thesis, a novel Asymmetrical Corner Detector (ACD) algorithm based on auto-correlation is presented and a semi-automatic image co-registration scheme is proposed. The ACD is designed with the consideration of the fact that asymmetrical corner points are the most common reality in remotely sensed imagery data. The ACD selects points more favourable to asymmetrical points rather than symmetrical points to avoid incorrect selection of flat points which are often highly symmetrical. The experimental results using images taken by different sensors indicate that the ACD has obtained excellent performance in terms of point localization and computation efficiency. It is more capable of selecting high quality GCPs than some well established corner detectors favourable to symmetrical corner points such as the Harris Corner Detector (Harris and Stephens, 1988). A semi-automatic image co-registration scheme is then proposed, which employs the ACD algorithm to extract evenly distributed GCPs across the overlapped area in the reference image. The scheme uses three manually selected pairs of GCPs to determine the initial transformation model and the overlapped area. Grid-control and nonmaximum suppression methods are used to secure the high quality and spread distribution of GCPs selected. It also involves the FNCC (fast normalised crosscorrelation) algorithm (Lewis, 1995) to refine the corresponding point locations in the input image and thus the GCPs are semi-automatically selected to proceed to the polynomial fitting image rectification. The performance of the proposed coregistration scheme has been demonstrated by registering multi-temporal, multi-sensor and multi-resolution images taken by Landsat TM, ETM+ and SPOT sensors. Experimental results show that consistent high registration accuracy of less than 0.7 pixels RMSE has been achieved. Keywords: Asymmetrical corner points, image co-registration, AC

Automatic and semi-automatic extraction of curvilinear features from SAR images

Extraction of curvilinear features from synthetic aperture radar (SAR) images is important for automatic recognition of various targets, such as fences, surrounding the buildings. The bright pixels which constitute curvilinear features in SAR images are usually disrupted and also degraded by high amount of speckle noise which makes extraction of such curvilinear features very difficult. In this paper an approach for the extraction of curvilinear features from SAR images is presented. The proposed approach is based on searching the curvilinear features as an optimum unidirectional path crossing over the vertices of the features determined after a despeckling operation. The proposed method can be used in a semi-automatic mode if the user supplies the starting vertex or in an automatic mode otherwise. In the semi-automatic mode, the proposed method produces reasonably accurate real-time solutions for SAR images

Remote Sensing

This dual conception of remote sensing brought us to the idea of preparing two different books; in addition to the first book which displays recent advances in remote sensing applications, this book is devoted to new techniques for data processing, sensors and platforms. We do not intend this book to cover all aspects of remote sensing techniques and platforms, since it would be an impossible task for a single volume. Instead, we have collected a number of high-quality, original and representative contributions in those areas

Investigation related to multispectral imaging systems

A summary of technical progress made during a five year research program directed toward the development of operational information systems based on multispectral sensing and the use of these systems in earth-resource survey applications is presented. Efforts were undertaken during this program to: (1) improve the basic understanding of the many facets of multispectral remote sensing, (2) develop methods for improving the accuracy of information generated by remote sensing systems, (3) improve the efficiency of data processing and information extraction techniques to enhance the cost-effectiveness of remote sensing systems, (4) investigate additional problems having potential remote sensing solutions, and (5) apply the existing and developing technology for specific users and document and transfer that technology to the remote sensing community