Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Methods to Retrieve the Cloud-Top Height in the Frame of the JEM-EUSO Mission

The Japanese Experiment Module-Extreme Universe Space Observatory (JEM-EUSO) telescope will measure ultrahigh-energy cosmic ray properties by detecting the UV fluorescence light generated in the interaction between cosmic rays and the atmosphere. Therefore, information on the state of clouds in the atmosphere is crucial for a proper interpretation of the data. For a real-time observation of the clouds in the telescope field of view, the JEM-EUSO will use an atmospheric monitoring system composed of a light detection and ranging and an infrared (IR) camera. In this paper, the focus is on the IR camera data. To retrieve the cloud-top height (CTH) from IR images, three different methods are considered here. The first one is based on bispectral stereo vision algorithms and requires two different views of the same scene in different spectral bands. For the second one, brightness temperatures provided by the IR camera are converted to effective cloud-top temperatures, from which the CTH is estimated using the vertical temperature profiles. A third method that uses the primary numerical weather prediction model output parameters, such as the cloud fraction, has also been considered to retrieve the CTH. This paper presents a first analysis, in which the heights retrieved by these three methodologies are compared with the heights given by the Moderate Resolution Imaging Spectroradiometer sensor installed on the polar satellite Terra. Since all these methods are suitable for the JEM-EUSO mission, they could be used in the future in a complementary way to improve the accuracy of the CTH retrieval

Comparing different methods to retrieve cloud top height from Meteosat satellite data

Cloud parameters such as the Cloud Top Height (CTH), Cloud Top Temperature (CTT), emissivity, particle size and optical depth have always been matter of interest for the atmospheric community. Particularly the CTH provides information leading to better understand the cloud radiative effects. Although there are many meteorological satellites providing the CTH, there are other sensors, not devoted to this purpose, that give some information from which this crucial parameter can be estimated. In this contribution we will describe three different methodologies to retrieve the CTH. The first technique is based on stereo-vision algorithms and requires two different views of the same scene and does not need of extra atmospheric information. In the second one, brightness temperatures in two IR spectral bands are converted to real cloud temperature by means of the proposed algorithms. From the CTT, the CTH is estimated using temperature vertical profiles (measured or modeled). The third technique retrieves the CTH from the output parameters of post event simulations performed by a Numerical Weather Prediction (NWP) model that in this work will be the mesoscale model WRF (Weather Research Forecast). This article presents a preliminary work, in which the heights retrieved by the three methodologies applied to the geostationary satellite Meteosat 10 are compared with the heights given by MODIS sensor installed on the polar satellite AQUA. This promising results show that valuable information about CTH can be retrieved from Meteosat which provide high frequency and large scale data useful for weather and climate research

Multi-Scale 3D Scene Flow from Binocular Stereo Sequences

Scene flow methods estimate the three-dimensional motion field for points in the world, using multi-camera video data. Such methods combine multi-view reconstruction with motion estimation. This paper describes an alternative formulation for dense scene flow estimation that provides reliable results using only two cameras by fusing stereo and optical flow estimation into a single coherent framework. Internally, the proposed algorithm generates probability distributions for optical flow and disparity. Taking into account the uncertainty in the intermediate stages allows for more reliable estimation of the 3D scene flow than previous methods allow. To handle the aperture problems inherent in the estimation of optical flow and disparity, a multi-scale method along with a novel region-based technique is used within a regularized solution. This combined approach both preserves discontinuities and prevents over-regularization – two problems commonly associated with the basic multi-scale approaches. Experiments with synthetic and real test data demonstrate the strength of the proposed approach.National Science Foundation (CNS-0202067, IIS-0208876); Office of Naval Research (N00014-03-1-0108

A Data-Driven Regularization Model for Stereo and Flow

Data-driven techniques can reliably build semantic correspondence among images. In this paper, we present a new regularization model for stereo or flow through transferring the shape information of the disparity or flow from semantically matched patches in the training database. Compared to previous regularization models based on image appearance alone, we can better resolve local ambiguity of the disparity or flow by considering the semantic information without explicit object modeling. We incorporate this data-driven regularization model into a standard Markov Random Field (MRF) model, inferred with a gradient descent algorithm and learned with a discriminative learning approach. Compared to prior state-of-the-art methods, our full model achieves comparable or better results on the KITTI stereo and flow datasets, and improves results on the Sintel Flow dataset under an online estimation setting.National Science Foundation (U.S.) (CGV 1212849)United States. Office of Naval Research. Multidisciplinary University Research Initiative (Award N00014-09-1-1051

Proposal Flow: Semantic Correspondences from Object Proposals

Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that the corresponding sparse proposal flow can effectively be transformed into a conventional dense flow field. We introduce two new challenging datasets that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use these benchmarks to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.Comment: arXiv admin note: text overlap with arXiv:1511.0506