Google Earth Engine cloud computing platform for remote sensing big data applications: a comprehensive review

Remote sensing (RS) systems have been collecting massive volumes of datasets for decades, managing and analyzing of which are not practical using common software packages and desktop computing resources. In this regard, Google has developed a cloud computing platform, called Google Earth Engine (GEE), to effectively address the challenges of big data analysis. In particular, this platformfacilitates processing big geo data over large areas and monitoring the environment for long periods of time. Although this platformwas launched in 2010 and has proved its high potential for different applications, it has not been fully investigated and utilized for RS applications until recent years. Therefore, this study aims to comprehensively explore different aspects of the GEE platform, including its datasets, functions, advantages/limitations, and various applications. For this purpose, 450 journal articles published in 150 journals between January 2010 andMay 2020 were studied. It was observed that Landsat and Sentinel datasets were extensively utilized by GEE users. Moreover, supervised machine learning algorithms, such as Random Forest, were more widely applied to image classification tasks. GEE has also been employed in a broad range of applications, such as Land Cover/land Use classification, hydrology, urban planning, natural disaster, climate analyses, and image processing. It was generally observed that the number of GEE publications have significantly increased during the past few years, and it is expected that GEE will be utilized by more users from different fields to resolve their big data processing challenges.Peer ReviewedPostprint (published version

A 30-m landsat-derived cropland extent product of Australia and China using random forest machine learning algorithm on Google Earth Engine cloud computing platform

© 2018 The Author(s) Mapping high resolution (30-m or better) cropland extent over very large areas such as continents or large countries or regions accurately, precisely, repeatedly, and rapidly is of great importance for addressing the global food and water security challenges. Such cropland extent products capture individual farm fields, small or large, and are crucial for developing accurate higher-level cropland products such as cropping intensities, crop types, crop watering methods (irrigated or rainfed), crop productivity, and crop water productivity. It also brings many challenges that include handling massively large data volumes, computing power, and collecting resource intensive reference training and validation data over complex geographic and political boundaries. Thereby, this study developed a precise and accurate Landsat 30-m derived cropland extent product for two very important, distinct, diverse, and large countries: Australia and China. The study used of eight bands (blue, green, red, NIR, SWIR1, SWIR2, TIR1, and NDVI) of Landsat-8 every 16-day Operational Land Imager (OLI) data for the years 2013–2015. The classification was performed by using a pixel-based supervised random forest (RF) machine learning algorithm (MLA) executed on the Google Earth Engine (GEE) cloud computing platform. Each band was time-composited over 4–6 time-periods over a year using median value for various agro-ecological zones (AEZs) of Australia and China. This resulted in a 32–48-layer mega-file data-cube (MFDC) for each of the AEZs. Reference training and validation data were gathered from: (a) field visits, (b) sub-meter to 5-m very high spatial resolution imagery (VHRI) data, and (c) ancillary sources such as from the National agriculture bureaus. Croplands versus non-croplands knowledge base for training the RF algorithm were derived from MFDC using 958 reference-training samples for Australia and 2130 reference-training samples for China. The resulting 30-m cropland extent product was assessed for accuracies using independent validation samples: 900 for Australia and 1972 for China. The 30-m cropland extent product of Australia showed an overall accuracy of 97.6% with a producer's accuracy of 98.8% (errors of omissions = 1.2%), and user's accuracy of 79% (errors of commissions = 21%) for the cropland class. For China, overall accuracies were 94% with a producer's accuracy of 80% (errors of omissions = 20%), and user's accuracy of 84.2% (errors of commissions = 15.8%) for cropland class. Total cropland areas of Australia were estimated as 35.1 million hectares and 165.2 million hectares for China. These estimates were higher by 8.6% for Australia and 3.9% for China when compared with the traditionally derived national statistics. The cropland extent product further demonstrated the ability to estimate sub-national cropland areas accurately by providing an R2 value of 0.85 when compared with province-wise cropland areas of China. The study provides a paradigm-shift on how cropland maps are produced using multi-date remote sensing. These products can be browsed at www.croplands.org and made available for download at NASA's Land Processes Distributed Active Archive Center (LP DAAC) https://www.lpdaac.usgs.gov/node/1282

Recent Advances in Forest Observation with Visual Interpretation of Very High-Resolution Imagery

The land area covered by freely available very high-resolution (VHR) imagery has grown dramatically over recent years, which has considerable relevance for forest observation and monitoring. For example, it is possible to recognize and extract a number of features related to forest type, forest management, degradation and disturbance using VHR imagery. Moreover, time series of medium-to-high-resolution imagery such as MODIS, Landsat or Sentinel has allowed for monitoring of parameters related to forest cover change. Although automatic classification is used regularly to monitor forests using medium-resolution imagery, VHR imagery and changes in web-based technology have opened up new possibilities for the role of visual interpretation in forest observation. Visual interpretation of VHR is typically employed to provide training and/or validation data for other remote sensing-based techniques or to derive statistics directly on forest cover/forest cover change over large regions. Hence, this paper reviews the state of the art in tools designed for visual interpretation of VHR, including Geo-Wiki, LACO-Wiki and Collect Earth as well as issues related to interpretation of VHR imagery and approaches to quality assurance. We have also listed a number of success stories where visual interpretation plays a crucial role, including a global forest mask harmonized with FAO FRA country statistics; estimation of dryland forest area; quantification of deforestation; national reporting to the UNFCCC; and drivers of forest change

Use and Improvement of Remote Sensing and Geospatial Technologies in Support of Crop Area and Yield Estimations in the West African Sahel

In arid and semi-arid West Africa, agricultural production and regional food security depend largely on small-scale subsistence farming and rainfed crops, both of which are vulnerable to climate variability and drought. Efforts made to improve crop monitoring and our ability to estimate crop production (areas planted and yield estimations by crop type) in the major agricultural zones of the region are critical paths for minimizing climate risks and to support food security planning. The main objective of this dissertation research was to contribute to these efforts using remote sensing technologies. In this regard, the first analysis documented the low reliability of existing land cover products for cropland area estimation (Chapter 2). Then two satellite remote sensing-based datasets were developed that 1) accurately map cropland areas in the five countries of Sahelian West Africa (Senegal, Mauritania, Mali, Burkina Faso and Niger; Chapter 3), and 2) focus on the country of Mali to identify the location and prevalence of the major subsistence crops (millet, sorghum, maize and non-irrigated rice; Chapter 4). The regional cropland area product is distributed as the West African Sahel Cropland area at 30 m (WASC30). The development of the new dataset involved high density training data (380,000 samples) developed by USGS in collaboration with CILSS for training about 200 locally optimized random forest (RF) classifiers using Landsat 8 surface reflectances and vegetation indices and the Google Earth Engine platform. WASC30 greatly improves earlier estimates through inclusion of cropland information for both rainfed and irrigated areas mapped with a class-specific accuracy of 79% across the West Africa Sahel. Used as a mask in crop monitoring systems, the new cropland area data could bring critical insights by reducing uncertainties in xv identification of croplands as crop growth condition metrics are extracted. WASC30 allowed us to derive detailed statistics on cultivated areas in the Sahel, at country and agroclimatic scales. Intensive agricultural zones were highlighted as well. The second dataset, mapping crop types for the country of Mali, is meant to separate signals of different crop types for improved crop yield estimation. The crop type map was used to derive detailed agricultural statistics (e.g. acreage by crop types, spatial distribution) at finer administrative scales than has previously been possible. The crop fraction information by crop type extracted from the map, gives additional details on farmers preferences by regions, and the natural adaptability of different crop types. The final analysis of this dissertation explores the use of ensemble machine learning techniques to predict maize yield in Mali (Chapter 5). Climate data (precipitation and temperature), and vegetation indices (Normalized Difference Vegetation Index, NDVI, the Enhanced Vegetation Index, EVI, and the Normalized Difference Water Index, NDWI) are used as predictors, while actual yields collected in 2017 by the Malian Ministry of Agriculture are the reference data. Random forest presented better predictive performance as compared to boosted regression trees (BRT). Results showed that climate variables have more predictive power for maize yield compared to vegetation indices. Among vegetation indices, the NDWI appeared to be the most influential predictor, maybe because of water requirement of maize and the sensitivity of this index to water in semi-arid regions. Tested with two different independent datasets, one constituted by 20% of the reference information, and another including observed yields for year 2018 (a one-year-left analysis), maize yield predictions were promising for year 2017 (RMSE = 362 kg/ha), but showed higher error for 2018 (RMSE = 707 kg/ha). That is, the fitted model may not capture accurately year to year variabilities in predicted maize yield. In this analysis, predictions were limited to field samples (~600 fields) across the country of Mali. It would be valuable in the future to predict maize yield for each pixel of the new developed crop type map. That will lead to a detailed spatial analysis of maize yield, allowing identification of low yielding regions for targeted interventions which could improve food security. Keywords: Agricultural identification of croplands as crop growth condition metrics are extracted. WASC30 allowed us to derive detailed statistics on cultivated areas in the Sahel, at country and agroclimatic scales. Intensive agricultural zones were highlighted as well. The second dataset, mapping crop types for the country of Mali, is meant to separate signals of different crop types for improved crop yield estimation. The crop type map was used to derive detailed agricultural statistics (e.g. acreage by crop types, spatial distribution) at finer administrative scales than has previously been possible. The crop fraction information by crop type extracted from the map, gives additional details on farmers preferences by regions, and the natural adaptability of different crop types. The final analysis of this dissertation explores the use of ensemble machine learning techniques to predict maize yield in Mali (Chapter 5). Climate data (precipitation and temperature), and vegetation indices (Normalized Difference Vegetation Index, NDVI, the Enhanced Vegetation Index, EVI, and the Normalized Difference Water Index, NDWI) are used as predictors, while actual yields collected in 2017 by the Malian Ministry of Agriculture are the reference data. Random forest presented better predictive performance as compared to boosted regression trees (BRT). Results showed that climate variables have more predictive power for maize yield compared to vegetation indices. Among vegetation indices, the NDWI appeared to be the most influential predictor, maybe because of water requirement of maize and the sensitivity of this index to water in semi-arid regions. Tested with two different independent datasets, one constituted by 20% of the reference information, and another including observed yields for year 2018 (a one-year-left analysis), maize yield predictions were promising for year 2017 (RMSE = 362 kg/ha), but showed higher error for 2018 (RMSE = 707 kg/ha). That is, the fitted model may not capture accurately year to year variabilities in predicted maize yield. In this analysis, predictions were limited to field samples (~600 fields) across the country of Mali. It would be valuable in the future to predict maize yield for each pixel of the new developed crop type map. That will lead to a detailed spatial analysis of maize yield, allowing identification of low yielding regions for targeted interventions which could improve food security

Rapid land cover classification using a 36-year time series of multi-source remote sensing data

Long-time series land cover classification information is the basis for scientific research on urban sprawl, vegetation change, and the carbon cycle. The rapid development of cloud computing platforms such as the Google Earth Engine (GEE) and access to multi-source satellite imagery from Landsat and Sentinel-2 enables the application of machine learning algorithms for image classification. Here, we used the Random Forest algorithm to quickly achieve a time series land cover classification at different scales based on the fixed land classification sample points selected from images acquired in 2022, and the year-by-year spectral differences of sample points. The classification accuracy was enhanced by using multi-source remote sensing data, such as synthetic aperture radar (SAR) and digital elevation model (DEM) data. The results showed that: (i) the maximum difference (threshold) of sample points without land class change determined by counting the sample points of each band of landsat time series from 1986 to 2022 was 0.25; (ii) the kappa coefficient and observed accuracy of the same sensor from Landsat 8 are higher than the results of TM and ETM+ sensor data from 2013 to 2022; (iii) the addition of a mining land cover type increase the kappa coefficient and overall accuracy mean values of the Sentinel 2 image classification for a complex mining and -forest area. Among the land classifications by multi-source remote sensing, the combined variables spectral band + index + topography + SAR result in the highest accuracy, but the overall improvement is limited. The method proposed is applicable to remotely sensed images at different scales and using sensors under complex terrain conditions. The use of GEE cloud computing platform enabled rapid analysis of remotely sensed data to produce land cover maps with high-accuracy and a long time series

A review of satellite-based global agricultural monitoring systems available for Africa

Abstract The increasing frequency and severity of extreme climatic events and their impacts are being realized in many regions of the world, particularly in smallholder crop and livestock production systems in Sub-Saharan Africa (SSA). These events underscore the need for timely early warning. Satellite Earth Observation (EO) availability, rapid developments in methodology to archive and process them through cloud services and advanced computational capabilities, continue to generate new opportunities for providing accurate, reliable, and timely information for decision-makers across multiple cropping systems and for resource-constrained institutions. Today, systems and tools that leverage these developments to provide open access actionable early warning information exist. Some have already been employed by early adopters and are currently operational in selecting national monitoring programs in Angola, Kenya, Rwanda, Tanzania, and Uganda. Despite these capabilities, many governments in SSA still rely on traditional crop monitoring systems, which mainly rely on sparse and long latency in situ reports with little to no integration of EO-derived crop conditions and yield models. This study reviews open-access operational agricultural monitoring systems available for Africa. These systems provide the best-available open-access EO data that countries can readily take advantage of, adapt, adopt, and leverage to augment national systems and make significant leaps (timeliness, spatial coverage and accuracy) of their monitoring programs. Data accessible (vegetation indices, crop masks) in these systems are described showing typical outputs. Examples are provided including crop conditions maps, and damage assessments and how these have integrated into reporting and decision-making. The discussion compares and contrasts the types of data, assessments and products can expect from using these systems. This paper is intended for individuals and organizations seeking to access and use EO to assess crop conditions who might not have the technical skill or computing facilities to process raw data into informational products

Continental-scale land cover mapping at 10 m resolution over Europe (ELC10)

Widely used European land cover maps such as CORINE are produced at medium spatial resolutions (100 m) and rely on diverse data with complex workflows requiring significant institutional capacity. We present a high resolution (10 m) land cover map (ELC10) of Europe based on a satellite-driven machine learning workflow that is annually updatable. A Random Forest classification model was trained on 70K ground-truth points from the LUCAS (Land Use/Cover Area frame Survey) dataset. Within the Google Earth Engine cloud computing environment, the ELC10 map can be generated from approx. 700 TB of Sentinel imagery within approx. 4 days from a single research user account. The map achieved an overall accuracy of 90% across 8 land cover classes and could account for statistical unit land cover proportions within 3.9% (R2 = 0.83) of the actual value. These accuracies are higher than that of CORINE (100 m) and other 10-m land cover maps including S2GLC and FROM-GLC10. We found that atmospheric correction of Sentinel-2 and speckle filtering of Sentinel-1 imagery had minimal effect on enhancing classification accuracy (< 1%). However, combining optical and radar imagery increased accuracy by 3% compared to Sentinel-2 alone and by 10% compared to Sentinel-1 alone. The conversion of LUCAS points into homogenous polygons under the Copernicus module increased accuracy by <1%, revealing that Random Forests are robust against contaminated training data. Furthermore, the model requires very little training data to achieve moderate accuracies - the difference between 5K and 50K LUCAS points is only 3% (86 vs 89%). At 10-m resolution, the ELC10 map can distinguish detailed landscape features like hedgerows and gardens, and therefore holds potential for aerial statistics at the city borough level and monitoring property-level environmental interventions (e.g. tree planting)