LandQ(upsilon 2): A MapReduce-Based System for Processing Arable Land Quality Big Data

Arable land quality (ALQ) data are a foundational resource for national food security. With the rapid development of spatial information technologies, the annual acquisition and update of ALQ data covering the country have become more accurate and faster. ALQ data are mainly vector-based spatial big data in the ESRI (Environmental Systems Research Institute) shapefile format. Although the shapefile is the most common GIS vector data format, unfortunately, the usage of ALQ data is very constrained due to its massive size and the limited capabilities of traditional applications. To tackle the above issues, this paper introduces LandQv2, which is a MapReduce-based parallel processing system for ALQ big data. The core content of LandQv2 is composed of four key technologies including data preprocessing, the distributed R-tree index, the spatial range query, and the map tile pyramid model-based visualization. According to the functions in LandQv2, firstly, ALQ big data are transformed by a MapReduce-based parallel algorithm from the ESRI Shapefile format to the GeoCSV file format in HDFS (Hadoop Distributed File System), and then, the spatial coding-based partition and R-tree index are executed for the spatial range query operation. In addition, the visualization of ALQ big data with a GIS (Geographic Information System) web API (Application Programming Interface) uses the MapReduce program to generate a single image or pyramid tiles for big data display. Finally, a set of experiments running on a live system deployed on a cluster of machines shows the efficiency and scalability of the proposed system. All of these functions supported by LandQv2 are integrated into SpatialHadoop, and it is also able to efficiently support any other distributed spatial big data systems

Big spatial vector data management: a review

Spatial vector data with high-precision and wide-coverage has exploded globally, such as land cover, social media, and other data-sets, which provides a good opportunity to enhance the national macroscopic decision-making, social supervision, public services, and emergency capabilities. Simultaneously, it also brings great challenges in management technology for big spatial vector data (BSVD). In recent years, a large number of new concepts, parallel algorithms, processing tools, platforms, and applications have been proposed and developed to improve the value of BSVD from both academia and industry. To better understand BSVD and take advantage of its value effectively, this paper presents a review that surveys recent studies and research work in the data management field for BSVD. In this paper, we discuss and itemize this topic from three aspects according to different information technical levels of big spatial vector data management. It aims to help interested readers to learn about the latest research advances and choose the most suitable big data technologies and approaches depending on their system architectures. To support them more fully, firstly, we identify new concepts and ideas from numerous scholars about geographic information system to focus on BSVD scope in the big data era. Then, we conclude systematically not only the most recent published literatures but also a global view of main spatial technologies of BSVD, including data storage and organization, spatial index, processing methods, and spatial analysis. Finally, based on the above commentary and related work, several opportunities and challenges are listed as the future research interests and directions for reference

SAR-to-Optical Image Translation and Cloud Removal Based on Conditional Generative Adversarial Networks: Literature Survey, Taxonomy, Evaluation Indicators, Limits and Future Directions

Due to the limitation of optical images that their waves cannot penetrate clouds, such images always suffer from cloud contamination, which causes missing information and limitations for subsequent agricultural applications, among others. Synthetic aperture radar (SAR) is able to provide surface information for all times and all weather. Therefore, translating SAR or fusing SAR and optical images to obtain cloud-free optical-like images are ideal ways to solve the cloud contamination issue. In this paper, we investigate the existing literature and provides two kinds of taxonomies, one based on the type of input and the other on the method used. Meanwhile, in this paper, we analyze the advantages and disadvantages while using different data as input. In the last section, we discuss the limitations of these current methods and propose several possible directions for future studies in this field

China Data Cube (CDC) for Big Earth Observation Data: Practices and Lessons Learned

In the face of tight natural resources and complex as well as volatile environments, and in order to meet the pressure brought by population growth, we need to overcome a series of challenges. As a new data management paradigm, the Earth Observation Data Cube simplifies the way that users manage and use earth observation data, and provides an analysis-ready form to access big spatiotemporal data, so as to realize the greater potential of earth observation data. Based on the Open Data Cube (ODC) framework, combined with analysis-ready data (ARD) generation technology, the design and implementation of CDC_DLTool, extending the support for data loading and the processing of international and Chinese imagery data covering China, this study eventually constructs the China Data Cube (CDC) framework. In the framework of this CDC grid, this study carried out case studies of water change monitoring based on international satellite imagery data of Landsat 8 in addition to vegetation change monitoring based on Chinese satellite imagery data of GF-1. The experimental results show that, compared with traditional scene-based data organization, the minimum management unit of this framework is a pixel, which makes the unified organization and management of multisource heterogeneous satellite imagery data more convenient and faster

An optimized hexagonal quadtree encoding and operation scheme for icosahedral hexagonal discrete global grid systems

Although research on the discrete global grid systems (DGGSs) has become an essential issue in the era of big earth data, there is still a gap between the efficiency of current encoding and operation schemes for hexagonal DGGSs and the needs of practical applications. This paper proposes a novel and efficient encoding and operation scheme of an optimized hexagonal quadtree structure (OHQS) based on aperture 4 hexagonal discrete global grid systems by translation transformation. A vector model is established to describe and calculate the aperture 4 hexagonal grid system. This paper also provides two different grid code addition algorithms based on induction and $ijk$ coordinate transformation. We implement the transformation between OHQS codes and geographic coordinates through the $ij$, $ijk$ and $IJK$ coordinate systems. Compared with existing schemes, the scheme in this paper greatly improves the efficiency of the addition operation, neighborhood retrieval and coordinate transformation, and the coding is more concise than other aperture 4 hexagonal DGGSs. The encoding operation based on the $ijk$ coordinate system is faster than the encoding operation based on the induction and addition table. Spatial modeling based OHQS DGGSs are also provided. A case study with rainstorms demonstrated the availability of this scheme

Optimization of Numerical Methods for Transforming UTM Plane Coordinates to Lambert Plane Coordinates

The rapid transformation from UTM (Universal Transverse Mecator) projection to Lambert projection helps to realize timely merging, inversion, and analysis of high-frequency partitioned remote sensing images. In this study, the transformation error and the efficiency of the linear rule approximation method, the improved linear rule approximation method, the hyperbolic transformation method, and the conformal transformation method were compared in transforming the coordinates of sample points on WGS84 (The World Geodetic System 1984)-UTM zonal projections to WGS84-Lambert projection coordinates. The effect of the grid aspect ratio on the coordinate transformation error of the conformal transformation method was examined. In addition, the conformal transformation method-based error spatial pattern of the sample points was analyzed. The results show that the conformal transformation method can better balance error and efficiency than other numerical methods. The error of the conformal transformation method is less affected by grid size. The maximum x-error is less than 0.36 m and the maximum y-error is less than 1.22 m when the grid size reaches 300 km × 300 km. The x- and y-error values decrease when square grids are used; namely, setting the grid aspect ratio close to 1 helps to weaken the effect of increasing grid area on the error. The dispersion of the error distribution and the maximum error of sample points both decrease relative to their minimum distance to the grid edge and stabilize at a minimum distance equal to 70 km. This study can support the rapid integration of massive remote sensing data over large areas

Optimization of Numerical Methods for Transforming UTM Plane Coordinates to Lambert Plane Coordinates

The rapid transformation from UTM (Universal Transverse Mecator) projection to Lambert projection helps to realize timely merging, inversion, and analysis of high-frequency partitioned remote sensing images. In this study, the transformation error and the efficiency of the linear rule approximation method, the improved linear rule approximation method, the hyperbolic transformation method, and the conformal transformation method were compared in transforming the coordinates of sample points on WGS84 (The World Geodetic System 1984)-UTM zonal projections to WGS84-Lambert projection coordinates. The effect of the grid aspect ratio on the coordinate transformation error of the conformal transformation method was examined. In addition, the conformal transformation method-based error spatial pattern of the sample points was analyzed. The results show that the conformal transformation method can better balance error and efficiency than other numerical methods. The error of the conformal transformation method is less affected by grid size. The maximum x-error is less than 0.36 m and the maximum y-error is less than 1.22 m when the grid size reaches 300 km × 300 km. The x- and y-error values decrease when square grids are used; namely, setting the grid aspect ratio close to 1 helps to weaken the effect of increasing grid area on the error. The dispersion of the error distribution and the maximum error of sample points both decrease relative to their minimum distance to the grid edge and stabilize at a minimum distance equal to 70 km. This study can support the rapid integration of massive remote sensing data over large areas

A novel method of determining the optimal polyhedral orientation for discrete global grid systems applicable to regional-scale areas of interest

The polyhedral discrete global grid system (DGGS) is a multi-resolution discrete earth reference model supporting the fusion and processing of multi-source geospatial information. The orientation of the polyhedron relative to the earth is one of its key design choices, used when constructing the grid system, as the efficiency of indexing will decrease if local areas of interest extend over multiple faces of the spherical polyhedron. To date, most research has focused on global-scale applications while almost no rigorous mathematical models have been established for determining orientation parameters. In this paper, we propose a method for determining the optimal polyhedral orientation of DGGSs for areas of interest on a regional scale. The proposed method avoids splitting local or regional target areas across multiple polyhedral faces. At the same time, it effectively handles geospatial data at a global scale because of the inherent characteristics of DGGSs. Results show that the orientation determined by this method successfully guarantees that target areas are located at the center of a single polyhedral face. The orientation process determined by this novel method reduces distortions and is more adaptable to different geographical areas, scales, and base polyhedrons than those employed by existing procedures

The Potential of 3-D Building Height Data to Characterize Socioeconomic Activities: A Case Study from 38 Cities in China

Urban forms are closely related to the urban environment, providing great potential to analyze human socioeconomic activities. However, limited studies have investigated the impacts of three-dimensional (3-D) urban forms on socioeconomic activities across cities. In this paper, we explored the relationship between urban form and socioeconomic activities using 3-D building height data from 38 cities in China. First, we aggregated the building footprint data and calculated three building indicators at the grid scale, based on which the spatial patterns of building height and road density were analyzed. Then, we examined the capacities of two-dimensional (2D)/3D urban forms in characterizing socioeconomic activities using satellite-derived nighttime light (NTL) data. Finally, we analyzed the relationship between road density distributions and building heights across 38 cities in China. Our results suggest that the building height information can improve the correlation between urban form and NTL. Different patterns of road distribution were revealed according to the distribution of road density change from the building hotspots, showing the capacity of 3-D building height data in helping characterize socioeconomic activities. Our study indicates that the 3-D building height information is of great potential to support a variety of studies in urban domains, such as population distribution and carbon emissions, with significantly improved capacities