Demonstration of Large Area Land Cover Classification with a One Dimensional Convolutional Neural Network Applied to Single Pixel Temporal Metric Percentiles

Over large areas, land cover classification has conventionally been undertaken using satellite time series. Typically temporal metric percentiles derived from single pixel location time series have been used to take advantage of spectral differences among land cover classes over time and to minimize the impact of missing observations. Deep convolutional neural networks (CNNs) have demonstrated potential for land cover classification of single date images. However, over large areas and using time series their application is complicated because they are sensitive to missing observations and they may misclassify small and spatially fragmented surface features due to their spatial patch-based implementation. This study demonstrates, for the first time, a one-dimensional (1D) CNN single pixel time series land classification approach that uses temporal percentile metrics and that does not have these issues. This is demonstrated for all the Conterminous United States (CONUS) considering two different 1D CNN structures with 5 and 8 layers, respectively. CONUS 30 m land cover classifications were derived using all the available Landsat-5 and -7 imagery over a seven-month growing season in 2011 with 3.3 million 30 m land cover class labelled samples extracted from the contemporaneous CONUS National Land Cover Database (NLCD) 16 class land cover product. The 1D CNNs and, a conventional random forest model, were trained using 10%, 50% and 90% samples, and the classification accuracies were evaluated with an independent 10% proportion. Temporal metrics were classified using 5, 7 and 9 percentiles for each of five Landsat reflective wavelength bands and their eight band ratios. The CONUS and detailed 150 × 150 km classification results demonstrate that the approach is effective at scale and locally. The 1D CNN classification land cover class boundaries were preserved for small axis dimension features, such as roads and rivers, with no stripes or anomalous spatial patterns. The 8-layer 1D CNN provided the highest overall classification accuracies and both the 5-layer and 8-layer 1D CNN architectures provided higher accuracies than the random forest by 1.9% - 2.8% which as all the accuracies were \u3e 83% is a meaningful increase. The CONUS overall classification accuracies increased marginally with the number of percentiles (86.21%, 86.40%, and 86.43% for 5, 7 and 9 percentiles, respectively) using the 8-layer 1D-CNN. Class specific producer and user accuracies were quantified, with lower accuracies for the developed land, crop and pasture/hay classes, but no systematic pattern among classes with respect to the number of temporal percentiles used. Application of the trained model to a different year of CONUS Landsat ARD showed moderately decreased accuracy (80.79% for 7 percentiles) that we illustrate is likely due to different intra-annual surface variations between years. These encouraging results are discussed with recommended research for deep learning using temporal metric percentiles

Utility of Daily 3 m Planet Fusion Surface Reflectance Data for Tillage Practice Mapping with Deep Learning

Tillage practices alter soil surface structure that can be potentially captured by satellite images with both high spatial and temporal resolution. This study explored tillage practice mapping using the daily Planet Fusion surface reflectance (PF-SR) gap-free 3 m data generated by fusing PlanetScope with Landsat-8, Sentinel-2 and MODIS surface reflectance data. The study area is a 220 × 220 km2 agricultural area in South Dakota, USA, and the study used 3285 PF-SR images from September 1, 2020 to August 31, 2021. The PF-SR images for the surveyed 433 fields were sliced into 10,747 training (70%) and evaluation (30%) non-overlapping time series patches. The training and evaluation patches were from different fields for evaluation data independence. The performance of four deep learning models (i.e., 2D convolutional neural networks (CNN), 3D CNN, CNN-Long short-term memory (LSTM), and attention CNN-LSTM) in tillage practice mapping, as well as their sensitivity to different spatial (i.e., 3 m, 24 m, and 96 m) and temporal resolutions (16-day, 8-day, 4-day, 2-day and 1-day) were examined. Classification accuracy continuously increased with increases in both temporal and spatial resolutions. The optimal models (3D CNN and attention CNN-LSTM) achieved ~77% accuracy using 2-day or daily 3 m resolution data as opposed to ~72% accuracy using 16-day 3 m resolution data or daily 24 m resolution data. This study also analyzed the feature importance of different acquisition dates for the two optimal models. The 3D CNN model feature importances were found to agree well with the tillage practice time. High feature importance was associated with observations during the fall and spring tillage period (i.e., fresh tillage signals) whereas the crop peak growing period (i.e., tillage signals weathered and confounded by dense canopy) was characterized by a relatively low feature importance. The work provides valuable insights into the utility of deep learning for tillage mapping and change event time identification based on high resolution imagery

Landsat 15-m Panchromatic-Assisted Downscaling (LPAD) of the 30-m Reflective Wavelength Bands to Sentinel-2 20-m Resolution

The Landsat 15-m Panchromatic-Assisted Downscaling (LPAD) method to downscale Landsat-8 Operational Land Imager (OLI) 30-m data to Sentinel-2 multi-spectral instrument (MSI) 20-m resolution is presented. The method first downscales the Landsat-8 30-m OLI bands to 15-m using the spatial detail provided by the Landsat-8 15-m panchromatic band and then reprojects and resamples the downscaled 15-m data into registration with Sentinel-2A 20-m data. The LPAD method is demonstrated using pairs of contemporaneous Landsat-8 OLI and Sentinel-2A MSI images sensed less than 19 min apart over diverse geographic environments. The LPAD method is shown to introduce less spectral and spatial distortion and to provide visually more coherent data than conventional bilinear and cubic convolution resampled 20-m Landsat OLI data. In addition, results for a pair of Landsat-8 and Sentinel-2A images sensed one day apart suggest that image fusion should be undertaken with caution when the images are acquired under different atmospheric conditions. The LPAD source code is available at GitHub for public use

Deep Convolutional Neural Network for Mapping Smallholder Agriculture Using High Spatial Resolution Satellite Image

In classification of satellite images acquired over smallholder agricultural landscape with complex spectral profiles of various crop types, exploring image spatial information is important. The deep convolutional neural network (CNN), originally designed for natural image recognition in the computer vision field, can automatically explore high level spatial information and thus is promising for such tasks. This study tried to evaluate different CNN structures for classification of four smallholder agricultural landscapes in Heilongjiang, China using pan-sharpened 2 m GaoFen-1 (meaning high resolution in Chinese) satellite images. CNN with three pooling strategies: without pooling, with max pooling and with average pooling, were evaluated and compared with random forest. Two different numbers (~70,000 and ~290,000) of CNN learnable parameters were examined for each pooling strategy. The training and testing samples were systematically sampled from reference land cover maps to ensure sample distribution proportional to the reference land cover occurrence and included 60,000–400,000 pixels to ensure effective training. Testing sample classification results in the four study areas showed that the best pooling strategy was the average pooling CNN and that the CNN significantly outperformed random forest (2.4–3.3% higher overall accuracy and 0.05–0.24 higher kappa coefficient). Visual examination of CNN classification maps showed that CNN can discriminate better the spectrally similar crop types by effectively exploring spatial information. CNN was still significantly outperformed random forest using training samples that were evenly distributed among classes. Furthermore, future research to improve CNN performance was discussed

Classifying raw irregular time series (CRIT) for large area land cover mapping by adapting transformer model

For Landsat land cover classification, the time series observations are typically irregular in the number of observations in a period (e.g., a year) and acquisition dates due to cloud cover variations over large areas and acquisition plan variations over long periods. Compositing or temporal percentile calculation are usually used to transform the irregular time series to regular temporal variables so that the machine and deep learning classifiers can be applied. Recognizing that the composite and percentile calculations have information loss, this study presents a method directly Classifying the Raw Irregular Time series (CRIT) (‘raw’ means irregular good-quality surface reflectance time series without any composite or temporal percentile derivation) by adapting Transformer. CRIT uses the acquisition day of year as classification input to align time series and also takes the Landsat satellite platform (Landsat 5, 7 and 8) as input to address the inter-sensor reflectance differences.The CRIT was demonstrated by classifying Landsat analysis ready data (ARD) surface reflectance time series acquired across one year for three years (1985, 2006 and 2018) over the Conterminous United States (CONUS) with both spatial and temporal variations in Landsat availability. 20,047 training and 4949 evaluation 30-m pixel were used where each pixel was annotated as one of seven land cover classes for each year. The CRIT was compared with classifying 16-day composite time series and temporal percentiles and compared with a 1D convolution neural network (CNN) method. Results showed that the CRIT trained with three years of samples had 1.4–1.5% higher overall accuracies with less computation time than classifying 16-day composites and 2.3–2.4% higher than classifying temporal percentiles. The CRIT advantages over 16-day composites were pronounced for developed (0.05 F1-score) and cropland (0.02 F1-score) classes and for mixed or boundary pixels. This was reasonable as the 16-day composites had only on average 7.02, 16.49 and 15.78 good quality observations for the three years, respectively, in contrast to 7.89, 27.72, and 26.60 for the raw irregular time series. The CNN was not as good as CRIT in classifying the raw irregular time series as CNN simply filling temporal positions with no observations as zeros while the CRIT used a masking mechanism to rule out their contribution. The CRIT can also take the pixel coordinates and DEM variables as input which further increased the overall accuracies by 1.1–2.6% and achieved 84.33%, 87.54% and 87.01% overall accuracies for the 1985, 2006 and 2018 classifications, respectively. The CRIT land cover maps were shown consistent with the USGS Land Change Monitoring, Assessment, and Projection (LCMAP) maps. The developed codes, training data and maps were made publicly available

10 m crop type mapping using Sentinel-2 reflectance and 30 m cropland data layer product

The 30 m resolution U.S. Department of Agriculture (USDA) crop data layer (CDL) is a widely used crop type map for agricultural management and assessment, environmental impact assessment, and food security. A finer resolution crop type map can potentially reduce errors related to crop area estimation, field size characterization, and precision agriculture activities that requires crop growth information at scales finer than crop field. This study is to develop a method for crop type mapping using Sentinel-2 10 m bands (i.e., red, green, blue, and near-infrared) and to examine the benefit of the derived 10 m crop type map. The crop type mapping was conducted for two study areas with significantly different field sizes and crop types in South Dakota and California, respectively. The Sentinel-2 10 m surface reflectance and the derived normalized difference vegetation index (NDVI) acquired in the 2019 growing season were used to generate monthly median composites as classification input. The training and evaluation samples were derived from CDL by (i) finding good quality 30 m CDL pixels and (ii) identifying a single representative Sentinel-2 10 m pixel time series for each 30 m good quality CDL pixel. The random forest algorithm was trained using 80% of the samples and evaluated using the 20% remaining samples, and the results showed high overall accuracies of 94% and 83% for South Dakota and California study areas, respectively. The major crops in both study areas obtained high user’s and producer’s accuracies (>87%). There is a good agreement between the class proportions in the 10 m crop type map and 30 m CDL for both study areas with R2 ≥ 0.94 and root mean square error (RMSE) ≤ 3%. More importantly, compared to the 30 m CDL, the 10 m crop type map has much less salt-pepper and crop boundary-aliasing effects and defines better the small surface features (e.g., small fields, roads, and rivers). The potential of the method for large area 10 m crop type mapping is discussed

MODIS Evapotranspiration Downscaling Using a Deep Neural Network Trained Using Landsat 8 Reflectance and Temperature Data

The MODIS 8-day composite evapotranspiration (ET) product (MOD16A2) is widely used to study large-scale hydrological cycle and energy budgets. However, the MOD16A2 spatial resolution (500 m) is too coarse for local and regional water resource management in agricultural applications. In this study, we propose a Deep Neural Network (DNN)-based MOD16A2 downscaling approach to generate 30 m ET using Landsat 8 surface reflectance and temperature and AgERA5 meteorological variables. The model was trained at a 500 m resolution using the MOD16A2 ET as reference and applied to the Landsat 8 30 m resolution. The approach was tested on 15 Landsat 8 images over three agricultural study sites in the United States and compared with the classical random forest regression model that has been often used for ET downscaling. All evaluation sample sets applied to the DNN regression model had higher R2 and lower root-mean-square deviations (RMSD) and relative RMSD (rRMSD) (the average values: 0.67, 2.63 mm/8d and 14.25%, respectively) than the random forest model (0.64, 2.76 mm/8d and 14.92%, respectively). Spatial improvement was visually evident both in the DNN and the random forest downscaled 30 m ET maps compared with the 500 m MOD16A2, while the DNN-downscaled ET appeared more consistent with land surface cover variations. Comparison with the in situ ET measurements (AmeriFlux) showed that the DNN-downscaled ET had better accuracy, with R2 of 0.73, RMSD of 5.99 mm/8d and rRMSD of 48.65%, than the MOD16A2 ET (0.65, 7.18 and 50.42%, respectively)

A Deep-Neural-Network-Based Aerosol Optical Depth (AOD) Retrieval from Landsat-8 Top of Atmosphere Data

The 30 m resolution Landsat data have been used for high resolution aerosol optical depth (AOD) retrieval based on radiative transfer models. In this paper, a Landsat-8 AOD retrieval algorithm is proposed based on the deep neural network (DNN). A total of 6390 samples were obtained for model training and validation by collocating 8 years of Landsat-8 top of atmosphere (TOA) data and aerosol robotic network (AERONET) AOD data acquired from 329 AERONET stations over 30°W–160°E and 60°N–60°S. The Google Earth Engine (GEE) cloud-computing platform is used for the collocation to avoid a large download volume of Landsat data. Seventeen predictor variables were used to estimate AOD at 500 nm, including the seven bands TOA reflectance, two bands TOA brightness (BT), solar and viewing zenith and azimuth angles, scattering angle, digital elevation model (DEM), and the meteorological reanalysis total columnar water vapor and ozone concentration. The leave-one-station-out cross-validation showed that the estimated AOD agreed well with AERONET AOD with a correlation coefficient of 0.83, root-mean-square error of 0.15, and approximately 61% AOD retrievals within 0.05 + 20% of the AERONET AOD. Theoretical comparisons with the physical-based methods and the adaptation of the developed DNN method to Sentinel-2 TOA data with a different spectral band configuration are discussed