A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

A Review of Landcover Classification with Very-High Resolution Remotely Sensed Optical Images—Analysis Unit, Model Scalability and Transferability

As an important application in remote sensing, landcover classification remains one of the most challenging tasks in very-high-resolution (VHR) image analysis. As the rapidly increasing number of Deep Learning (DL) based landcover methods and training strategies are claimed to be the state-of-the-art, the already fragmented technical landscape of landcover mapping methods has been further complicated. Although there exists a plethora of literature review work attempting to guide researchers in making an informed choice of landcover mapping methods, the articles either focus on the review of applications in a specific area or revolve around general deep learning models, which lack a systematic view of the ever advancing landcover mapping methods. In addition, issues related to training samples and model transferability have become more critical than ever in an era dominated by data-driven approaches, but these issues were addressed to a lesser extent in previous review articles regarding remote sensing classification. Therefore, in this paper, we present a systematic overview of existing methods by starting from learning methods and varying basic analysis units for landcover mapping tasks, to challenges and solutions on three aspects of scalability and transferability with a remote sensing classification focus including (1) sparsity and imbalance of data; (2) domain gaps across different geographical regions; and (3) multi-source and multi-view fusion. We discuss in detail each of these categorical methods and draw concluding remarks in these developments and recommend potential directions for the continued endeavor

A gabor filter-based protocol for automated image-based building detection

Detecting buildings from high-resolution satellite imagery is beneficial in mapping, environmental preparation, disaster management, military planning, urban planning and research purposes. Differentiating buildings from the images is possible however, it may be a time-consuming or complicated process. Therefore, the high-resolution imagery from satellites needs to be automated to detect the buildings. Additionally, buildings exhibit several different characteristics, and their appearance in these images is unplanned. Moreover, buildings in the metropolitan environment are typically crowded and complicated. Therefore, it is challenging to identify the building and hard tolocate them. To resolve this situation, a novel probabilistic method has been suggested using local features and probabilistic approaches. A local feature extraction technique was implemented, which was used to calculate the probability density function. The locations in the image were represented as joint probability distributions and were used to estimate their probability distribution function (pdf). The density of building locations in the image was extracted. Kernel density distribution was also used to find the density flow for different metropolitan cities such as Sydney (Australia), Tokyo (Japan), and Mumbai (India), which is useful for distribution intensity and pattern of facility point f interest (POI). The purpose system can detect buildings/rooftops and to test our system, we choose some crops with panchromatic high-resolution satellite images from Australia and our results looks promising with high efficiency and minimal computational time for feature extraction. We were able to detect buildings with shadows and building without shadows in 0.4468 (seconds) and 0.5126 (seconds) respectively

Whales from space: Assessing the feasibility of using satellite imagery to monitor whales

By the mid-twentieth century, the majority of great whale species were threatened with extinction, following centuries of commercial whaling. Since the implementation of a moratorium on commercial whaling in 1985 by the International Whaling Commission, the recovery of whale population is being regularly assessed. Various methods are used to survey whale populations, though most are spatially limited and prevent remote areas from being studied. Satellites orbiting Earth can access most regions of the planet, offering a potential solution to surveying remote locations. With recent improvements in the spatial resolution of satellite imagery, it is now possible to detect wildlife from space, including whales. In this thesis, I aimed to further investigate the feasibility of very high resolution (VHR) satellite imagery as a tool to reliably monitor whales. The first objective was to describe, both visually and spectrally, how four morphologically distinct species appear in VHR satellite imagery. The second objective was to explore different ways to automatically detect whales in such imagery, as the current alternative is manual detection, which is time-consuming and impractical when monitoring large areas. With the third objective, I attempted to give some insights on how to estimate the maximum depth at which a whale can be detected in VHR satellite imagery, as this will be crucial to estimate whale abundance from space. This thesis shows that the four species targeted could be detected with varying degrees of accuracy, some contrasting better with their surroundings. Compared to manual detection, the automated systems trialled here took longer, were not as accurate, and were not transferable to other images, suggesting to focus future automation research on machine learning and the creation of a well-labelled database required to train and validate. The maximum depth of detection could be assessed only approximately using nautical charts. Other methods such as the installation of panels at various depths should be trialled, although it requires prior knowledge of the spectral reflectance of whales above the surface, which I tested on post-mortem samples of whale integument and proved unreliable. Such reflectance should be measured on free-swimming whale using unmanned aerial vehicles or small aircraft. Overall, this thesis shows that currently VHR satellite imagery can be a useful tool to assess the presence or absence of whales, encouraging further developments to make VHR satellite imagery a reliable method to monitor whale numbers.The MAVA Foundation (16035

Very High Resolution (VHR) Satellite Imagery: Processing and Applications

Recently, growing interest in the use of remote sensing imagery has appeared to provide synoptic maps of water quality parameters in coastal and inner water ecosystems;, monitoring of complex land ecosystems for biodiversity conservation; precision agriculture for the management of soils, crops, and pests; urban planning; disaster monitoring, etc. However, for these maps to achieve their full potential, it is important to engage in periodic monitoring and analysis of multi-temporal changes. In this context, very high resolution (VHR) satellite-based optical, infrared, and radar imaging instruments provide reliable information to implement spatially-based conservation actions. Moreover, they enable observations of parameters of our environment at greater broader spatial and finer temporal scales than those allowed through field observation alone. In this sense, recent very high resolution satellite technologies and image processing algorithms present the opportunity to develop quantitative techniques that have the potential to improve upon traditional techniques in terms of cost, mapping fidelity, and objectivity. Typical applications include multi-temporal classification, recognition and tracking of specific patterns, multisensor data fusion, analysis of land/marine ecosystem processes and environment monitoring, etc. This book aims to collect new developments, methodologies, and applications of very high resolution satellite data for remote sensing. The works selected provide to the research community the most recent advances on all aspects of VHR satellite remote sensing

Efficacy of machine learning, earth observation and geomorphometry for mapping salt-affected soils in irrigation fields

Thesis (MSc)--Stellenbosch University, 2018.ENGLISH ABSTRACT: There is a need to monitor salt accumulation throughout agricultural irrigation schemes as it can have a major negative impact on crop yields and subsequently result in a lower food production. Salt accumulation can result from natural processes, human interference or prolonged waterlogging. Most irrigation schemes are large and therefore difficult to monitor via conventional methods (e.g. regular field visits). More cost-effective, less time-consuming approaches in identifying salt-affected and salt-prone areas in large irrigation schemes are therefore needed. Remote sensing has been proposed as an alternative approach due to its ability to cover a large region on a timely basis. The approach is also more cost-effective because less field visits are required. A literature review on salt accumulation and remote sensing identified several direct and indirect methods for identifying salt-affected or salt-prone areas. Direct methods focus on the delineation of salt crusts visible on the bare soil in multispectral satellite imagery, whereas indirect methods, which include vegetation stress monitoring and geomorphometry (terrain analysis), attempt to take subsurface conditions into account. A disadvantage of the direct approach is that it does not take subsurface conditions into account, while vegetation stress monitoring (an indirect method) can produce inaccurate results because the vegetation stress can be a result of other factors (e.g. poor farming practices). Geomorphometry offers an alternative (modelling) approach that can either replace or augment direct and other indirect methods. Two experiments were carried out in this study, both of which focussed on machine learning (ML) algorithms (namely k-nearest neighbour (kNN), support vector machine (SVM), decision tree (DT) and random forest (RF)) and statistical analyses (regression or geostatistics) to identify salt-affected soils. The first experiment made use of very high resolution WorldView-2 (WV2) imagery. A number of texture measures and salinity indices were derived from the WV2 bands and considered as predictor variables. In addition to the ML and statistical analyses, a classification and regression tree (CART) model and Jeffries-Matusita (JM) distance thresholds were also produced from the predictors. The CART model was the most accurate in differentiating salt-affected and unaffected soils, but the accuracy of kNN and RF classifications were only marginally lower. The normalized difference salinity index showed the most promise among the predictors as it featured in the best JM, regression and CART models. The second experiment applied geomorphometry approaches to two South African irrigation schemes. Elevation sources include the Shuttle Radar Topographic Mission (SRTM) digital elevation model (DEM) and a digital surface model (DSM) produced from stereoscopic aerial photography. A number of morphological (e.g. slope gradient) and hydrological (e.g. flow direction) terrain parameters were derived from the SRTM DEM and the DSM and used as predictors. In addition to the algorithms used for the first experiment, the geostatistical method Kriging with external drift (KED) was also evaluated in this experiment. The source of elevation had an insignificant impact on the accuracies, although the DSM did show promise when combined with ML. KED outperformed regression modelling and ML in most cases, but ML produced similar results for one of the study areas. The experiments showed that direct and geomorphometry approaches hold much potential for mapping salt-affected soil. ML also proved to be a viable option for identifying salt-affected or salt-prone soil. It is recommended that a combination of direct and indirect (e.g. vegetation stress monitoring) approaches are considered in future research. Making use of alternative data sources such as hyperspectral imagery or higher spatial resolution DEMs may also prove useful. Clearly, more research is needed before such approaches can be operationalized for detecting, monitoring and mapping salt accumulation in irrigated areas.AFRIKAANSE OPSOMMING: Daar is 'n behoefte om soutophoping deur middel van landboubesproeiingskemas te monitor aangesien dit 'n beduidende negatiewe uitwerking op oesopbrengste kan hê en gevolglik tot laer voedselproduksie kan lei. Soutophoping kan voortspruit uit natuurlike prosesse, menslike inmenging of langdurige deurdrenking. Die meeste besproeiingskemas is groot en daarom moeilik om te monitor via konvensionele metodes (bv. gereelde veldbesoeke). Meer koste-effektiewe, minder tydrowende benaderings is dus nodig om soutgeaffekteerde areas en areas wat geneig is tot soutophoping in groot besproeiingskemas te identifiseer. Afstandswaarneming is voorgestel as 'n alternatiewe benadering weens sy vermoë om 'n groot streek op 'n tydige basis te dek. Die benadering is ook meer koste-effektief omdat minder veldbesoeke vereis word. 'n Literatuuroorsig oor soutophoping en afstandswaarneming het verskeie direkte en indirekte metodes geïdentifiseer om soutgeaffekteerde areas of areas geneig tot soutophoping te identifiseer. Direkte metodes fokus op die afbakening van soutkorste wat in multispektrale satellietbeelde op die kaal grond sigbaar is. Indirekte metodes, insluitende plantstresmonitering en geomorfometrie (terreinanalise), aan die ander kant, poog om die ondergrondse toestande in ag te neem. 'n Nadeel van die direkte benadering is dat dit nie ondergrondse toestande in ag neem nie, terwyl plantstresmonitering ('n indirekte metode) onakkurate resultate kan veroorsaak, aangesien die plantstres die gevolg kan wees van ander faktore (bv. swak boerderypraktyke). Geomorfometrie bied 'n alternatiewe (modellering) benadering wat direkte of ander indirekte metodes kan vervang of uitbrei. In hierdie studie is twee eksperimente uitgevoer. Albei het gefokus op masjienleer (ML) algoritmes, naamlik k-nearest neighour (kNN), ondersteunende vektormasjien, besluitboom en ewekansige woud (EW), en statistiese ontledings (regressie of geostatistiek) om soutgeaffekteerde gronde te identifiseer. Die eerste eksperiment het gebruik gemaak van baie hoë resolusie WorldView-2 (WV2) beelde. 'n Aantal tekstuurmaatreëls en soutindekse is afgelei van die WV2-bande en is beskou as voorspeller-veranderlikes. Benewens die ML en statistiese ontledings, is 'n klassifikasie- en regressieboom (KARB) model en Jeffries-Matusita (JM) afstandsdrempels ook van die voorspellers vervaardig. Die KARB-model het die mees akkuraatste differensiasie tussen sout-geaffekteerde en ongeaffekteerde grond gemaak, maar die akkuraatheid van kNN- en EW-klassifikasies was slegs marginaal laer. Van al die voorspellers het die genormaliseerde-verskil-saliniteit-indeks die meeste belofte getoon aangesien dit in die beste JM-, regressie- en KARB-modelle presteer het. Stellenbosch University https://scholar.sun.ac.za vi Die tweede eksperiment het geomorfometriese benaderings toegepas op twee Suid-Afrikaanse besproeiingskemas. Elevasiebronne sluit in die Shuttle Radar Topographic Mission (SRTM) digitale elevasie-model (DEM) en 'n digitale oppervlakmodel (DOM) wat uit stereoskopiese lugfotografie vervaardig word. 'n Aantal morfologiese (bv. hellingsgradiënt) en hidrologiese (bv. vloeirigting) terreinparameters is afgelei van die SRTM DEM en die DOM en is gebruik as voorspellers. Benewens die algoritmes wat vir die eerste eksperiment gebruik is, is die geostatistiese metode Kriging met eksterne dryf (KED) ook in hierdie eksperiment geëvalueer. Die bron van elevasie het 'n onbeduidende impak op die akkuraatheid gehad, hoewel die DOM belofte getoon het wanneer dit met ML gekombineer is. KED het in meeste gevalle beter presteer as regressie modellering en ML, maar ML het soortgelyke resultate vir een van die studiegebiede opgelewer. Die eksperimente het getoon dat direkte en geomorfometriese benaderings baie potensiaal het vir die kartering van soutgeaffekteerde grond. ML het ook bewys dat dit 'n lewensvatbare opsie is om soutgeaffekteerde grond of grond wat geneig is tot soutophoping, te identifiseer. Daar word aanbeveel dat 'n kombinasie van direkte en indirekte (bv. plantegroei-stresmonitering) benaderings in toekomstige navorsing oorweeg word. Die gebruik van alternatiewe databronne soos hiperspektrale beelde of hoër ruimtelike resolusie-DOM's kan ook nuttig wees. Dit is duidelik dat meer navorsing nodig is voordat sulke benaderings geoperasionaliseer kan word vir die opsporing, monitering en kartering van soutophoping in besproeide gebiede