Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

PERSIANN-CNN: Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks-Convolutional Neural Networks

Abstract Accurate and timely precipitation estimates are critical for monitoring and forecasting natural disasters such as floods. Despite having high-resolution satellite information, precipitation estimation from remotely sensed data still suffers from methodological limitations. State-of-the-art deep learning algorithms, renowned for their skill in learning accurate patterns within large and complex datasets, appear well suited to the task of precipitation estimation, given the ample amount of high-resolution satellite data. In this study, the effectiveness of applying convolutional neural networks (CNNs) together with the infrared (IR) and water vapor (WV) channels from geostationary satellites for estimating precipitation rate is explored. The proposed model performances are evaluated during summer 2012 and 2013 over central CONUS at the spatial resolution of 0.08° and at an hourly time scale. Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN)–Cloud Classification System (CCS), which is an operational satellite-based product, and PERSIANN–Stacked Denoising Autoencoder (PERSIANN-SDAE) are employed as baseline models. Results demonstrate that the proposed model (PERSIANN-CNN) provides more accurate rainfall estimates compared to the baseline models at various temporal and spatial scales. Specifically, PERSIANN-CNN outperforms PERSIANN-CCS (and PERSIANN-SDAE) by 54% (and 23%) in the critical success index (CSI), demonstrating the detection skills of the model. Furthermore, the root-mean-square error (RMSE) of the rainfall estimates with respect to the National Centers for Environmental Prediction (NCEP) Stage IV gauge–radar data, for PERSIANN-CNN was lower than that of PERSIANN-CCS (PERSIANN-SDAE) by 37% (14%), showing the estimation accuracy of the proposed model

A multi-sensor approach for volcanic ash cloud retrieval and eruption characterization: the 23 November 2013 Etna lava fountain

Volcanic activity is observed worldwide with a variety of ground and space-based remote sensing instruments, each with advantages and drawbacks. No single system can give a comprehensive description of eruptive activity, and so, a multi-sensor approach is required. This work integrates infrared and microwave volcanic ash retrievals obtained from the geostationary Meteosat Second Generation (MSG)-Spinning Enhanced Visible and Infrared Imager (SEVIRI), the polar-orbiting Aqua-MODIS and ground-based weather radar. The expected outcomes are improvements in satellite volcanic ash cloud retrieval (altitude, mass, aerosol optical depth and effective radius), the generation of new satellite products (ash concentration and particle number density in the thermal infrared) and better characterization of volcanic eruptions (plume altitude, total ash mass erupted and particle number density from thermal infrared to microwave). This approach is the core of the multi-platform volcanic ash cloud estimation procedure being developed within the European FP7-APhoRISM project. The Mt. Etna (Sicily, Italy) volcano lava fountaining event of 23 November 2013 was considered as a test case. The results of the integration show the presence of two volcanic cloud layers at different altitudes. The improvement of the volcanic ash cloud altitude leads to a mean difference between the SEVIRI ash mass estimations, before and after the integration, of about the 30%. Moreover, the percentage of the airborne “fine” ash retrieved from the satellite is estimated to be about 1%–2% of the total ash emitted during the eruption. Finally, all of the estimated parameters (volcanic ash cloud altitude, thickness and total mass) were also validated with ground-based visible camera measurements, HYSPLIT forward trajectories, Infrared Atmospheric Sounding Interferometer (IASI) satellite data and tephra deposits

Estimating snow cover from publicly available images

In this paper we study the problem of estimating snow cover in mountainous regions, that is, the spatial extent of the earth surface covered by snow. We argue that publicly available visual content, in the form of user generated photographs and image feeds from outdoor webcams, can both be leveraged as additional measurement sources, complementing existing ground, satellite and airborne sensor data. To this end, we describe two content acquisition and processing pipelines that are tailored to such sources, addressing the specific challenges posed by each of them, e.g., identifying the mountain peaks, filtering out images taken in bad weather conditions, handling varying illumination conditions. The final outcome is summarized in a snow cover index, which indicates for a specific mountain and day of the year, the fraction of visible area covered by snow, possibly at different elevations. We created a manually labelled dataset to assess the accuracy of the image snow covered area estimation, achieving 90.0% precision at 91.1% recall. In addition, we show that seasonal trends related to air temperature are captured by the snow cover index.Comment: submitted to IEEE Transactions on Multimedi

PERSIANN-MSA: A precipitation estimation method from satellite-based multispectral analysis

Visible and infrared data obtained from instruments onboard geostationary satellites have been extensively used for monitoring clouds and their evolution. The Advanced Baseline Imager (ABI) that will be launched onboard the Geostationary Operational Environmental Satellite-R (GOES-R) series in the near future will offer a larger range of spectral bands; hence, it will provide observations of cloud and rain systems at even finer spatial, temporal, and spectral resolutions than are possible with the current GOES. In this paper, a new method called Precipitation Estimation from Remotely Sensed information using Artificial Neural Networks-Multispectral Analysis (PERSIANN-MSA) is proposed to evaluate the effect of using multispectral imagery on precipitation estimation. The proposed approach uses a self-organizing feature map (SOFM) to classify multidimensional input information, extracted from each grid box and corresponding textural features of multispectral bands. In addition, principal component analysis (PCA) is used to reduce the dimensionality to a few independent input features while preserving most of the variations of all input information. The above method is applied to estimate rainfall using multiple channels of the Spinning Enhanced Visible and Infrared Imager (SEVIRI) onboard the Meteosat Second Generation (MSG) satellite. In comparison to the use of a single thermal infrared channel, the analysis shows that using multispectral data has the potential to improve rain detection and estimation skills with an average of more than 50% gain in equitable threat score for rain/no-rain detection, and more than 20% gain in correlation coefficient associated with rain-rate estimation. © 2009 American Meteorological Society

MISR stereoscopic image matchers: techniques and results

The Multi-angle Imaging SpectroRadiometer (MISR) instrument, launched in December 1999 on the NASA EOS Terra satellite, produces images in the red band at 275-m resolution, over a swath width of 360 km, for the nine camera angles 70.5/spl deg/, 60/spl deg/, 45.6/spl deg/, and 26.1/spl deg/ forward, nadir, and 26.1/spl deg/, 45.6/spl deg/, 60/spl deg/, and 70.5/spl deg/ aft. A set of accurate and fast algorithms was developed for automated stereo matching of cloud features to obtain cloud-top height and motion over the nominal six-year lifetime of the mission. Accuracy and speed requirements necessitated the use of a combination of area-based and feature-based stereo-matchers with only pixel-level acuity. Feature-based techniques are used for cloud motion retrieval with the off-nadir MISR camera views, and the motion is then used to provide a correction to the disparities used to measure cloud-top heights which are derived from the innermost three cameras. Intercomparison with a previously developed "superstereo" matcher shows that the results are very comparable in accuracy with much greater coverage and at ten times the speed. Intercomparison of feature-based and area-based techniques shows that the feature-based techniques are comparable in accuracy at a factor of eight times the speed. An assessment of the accuracy of the area-based matcher for cloud-free scenes demonstrates the accuracy and completeness of the stereo-matcher. This trade-off has resulted in the loss of a reliable quality metric to predict accuracy and a slightly high blunder rate. Examples are shown of the application of the MISR stereo-matchers on several difficult scenes which demonstrate the efficacy of the matching approach