Sub-workflow parallel implementation of aerosol optical depth retrieval from MODIS data case on a Grid platform

Aerosol Optical Depth (AOD) is an significant parameter of aerosol optical properties. Operational production of AOD datasets over long time series, large-scale coverage puts on a severe challenge to computing technologies due to both the complexity of retrieval algorithm and the huge data amounts. The Grid computing solution-Remote Sensing Service Node (RSSN) was constructed as a high-throughput platform for remote sensing applications. Taking the sub-workflow level characteristics of some remote sensing retrieval applications into consideration, a sub-workflow parallel implementation for the Synergetic Retrieval of Aerosol Properties (SRAP) algorithm from the Moderate Resolution Imaging Spectroradiometer (MODIS) sensor data was taken on the RSSN, and an initial experiment result proved that the subworkflow parallel could further reduce the runtime of data parallel solutions commonly used

Adaptive service discovery on service-oriented and spontaneous sensor systems

Service-oriented architecture, Spontaneous networks, Self-organisation, Self-configuration, Sensor systems, Social patternsNatural and man-made disasters can significantly impact both people and environments. Enhanced effect can be achieved through dynamic networking of people, systems and procedures and seamless integration of them to fulfil mission objectives with service-oriented sensor systems. However, the benefits of integration of services will not be realised unless we have a dependable method to discover all required services in dynamic environments. In this paper, we propose an Adaptive and Efficient Peer-to-peer Search (AEPS) approach for dependable service integration on service-oriented architecture based on a number of social behaviour patterns. In the AEPS network, the networked nodes can autonomously support and co-operate with each other in a peer-to-peer (P2P) manner to quickly discover and self-configure any services available on the disaster area and deliver a real-time capability by self-organising themselves in spontaneous groups to provide higher flexibility and adaptability for disaster monitoring and relief

IDEALIST control and service management solutions for dynamic and adaptive flexi-grid DWDM networks

Wavelength Switched Optical Networks (WSON) were designed with the premise that all channels in a network have the same spectrum needs, based on the ITU-T DWDM grid. However, this rigid grid-based approach is not adapted to the spectrum requirements of the signals that are best candidates for long-reach transmission and high-speed data rates of 400Gbps and beyond. An innovative approach is to evolve the fixed DWDM grid to a flexible grid, in which the optical spectrum is partitioned into fixed-sized spectrum slices. This allows facilitating the required amount of optical bandwidth and spectrum for an elastic optical connection to be dynamically and adaptively allocated by assigning the necessary number of slices of spectrum. The ICT IDEALIST project will provide the architectural design, protocol specification, implementation, evaluation and standardization of a control plane and a network and service management system. This architecture and tools are necessary to introduce dynamicity, elasticity and adaptation in flexi-grid DWDM networks. This paper provides an overview of the objectives, framework, functional requirements and use cases of the elastic control plane and the adaptive network and service management system targeted in the ICT IDEALIST project

A cloud-based remote sensing data production system

The data processing capability of existing remote sensing system has not kept pace with the amount of data typically received and need to be processed. Existing product services are not capable of providing users with a variety of remote sensing data sources for selection, either. Therefore, in this paper, we present a product generation programme using multisource remote sensing data, across distributed data centers in a cloud environment, so as to compensate for the low productive efficiency, less types and simple services of the existing system. The programme adopts “master–slave” architecture. Specifically, the master center is mainly responsible for the production order receiving and parsing, as well as task and data scheduling, results feedback, and so on; the slave centers are the distributed remote sensing data centers, which storage one or more types of remote sensing data, and mainly responsible for production task execution. In general, each production task only runs on one data center, and the data scheduling among centers adopts a “minimum data transferring” strategy. The logical workflow of each production task is organized based on knowledge base, and then turned into the actual executed workflow by Kepler. In addition, the scheduling strategy of each production task mainly depends on the Ganglia monitoring results, thus the computing resources can be allocated or expanded adaptively. Finally, we evaluated the proposed programme using test experiments performed at global, regional and local areas, and the results showed that our proposed cloud-based remote sensing production system could deal with massive remote sensing data and different products generating, as well as on-demand remote sensing computing and information service

Building the IDECi-UIB: the scientific spatial data infrastructure node for the Balearic Islands University

Technical and methodological enhancements in Information Technologies (IT) and Geographical Information Systems (GIS) has permitted the growth in Spatial Data Infrastructures (SDI) performance. In this way, their uses and applications have grown very rapidly. In the scientific and educational working fields, different institutions and organisations have bet for its use enforcing information exchange that allows researchers to improve their studies as well as give a better dissemination within the scientific community. Therefore, the GIS and Remote Sensing Service (SSIGT) at the Balearic Islands University (UIB) has decided to build and launch its own SDI to serve scientific Geo-Information (GI) throughout the Balearic Islands society focussing on the university community. By these means it intends to boost the development of research and education focusing on the field of spatial information. This article tries to explain the background ideas that form the basic concept of the scientific SDI related to the concepts of e-Science and e-Research. Finally, it explains how these ideas are taken into practice into the new University Scientific SDI

Developing an open data portal for the ESA climate change initiative

We introduce the rationale for, and architecture of, the European Space Agency Climate Change Initiative (CCI) Open Data Portal (http://cci.esa.int/data/). The Open Data Portal hosts a set of richly diverse datasets – 13 “Essential Climate Variables” – from the CCI programme in a consistent and harmonised form and to provides a single point of access for the (>100 TB) data for broad dissemination to an international user community. These data have been produced by a range of different institutions and vary across both scientific and spatio-temporal characteristics. This heterogeneity of the data together with the range of services to be supported presented significant technical challenges. An iterative development methodology was key to tackling these challenges: the system developed exploits a workflow which takes data that conforms to the CCI data specification, ingests it into a managed archive and uses both manual and automatically generated metadata to support data discovery, browse, and delivery services. It utilises both Earth System Grid Federation (ESGF) data nodes and the Open Geospatial Consortium Catalogue Service for the Web (OGC-CSW) interface, serving data into both the ESGF and the Global Earth Observation System of Systems (GEOSS). A key part of the system is a new vocabulary server, populated with CCI specific terms and relationships which integrates OGC-CSW and ESGF search services together, developed as part of a dialogue between domain scientists and linked data specialists. These services have enabled the development of a unified user interface for graphical search and visualisation – the CCI Open Data Portal Web Presence

Landcover and crop type classification with intra-annual times series of sentinel-2 and machine learning at central Portugal

Dissertation submitted in partial fulfilment of the requirements for the degree of Master of Science in Geospatial TechnologiesLand cover and crop type mapping have benefited from a daily revisiting period of sensors such as MODIS, SPOT-VGT, NOAA-AVHRR that contains long time-series archive. However, they have low accuracy in an Area of Interest (ROI) due to their coarse spatial resolution (i.e., pixel size > 250m). The Copernicus Sentinel-2 mission from the European Spatial Agency (ESA) provides free data access for Sentinel 2-A(S2a) and B (S2b). This satellite constellation guarantees a high temporal (5-day revisit cycle) and high spatial resolution (10m), allowing frequent updates on land cover products through supervised classification. Nevertheless, this requires training samples that are traditionally collected manually via fieldwork or image interpretation. This thesis aims to implement an automatic workflow to classify land cover and crop types at 10m resolution in central Portugal using existing databases, intra-annual time series of S2a and S2b, and Random Forest, a supervised machine learning algorithm. The agricultural classes such as temporary and permanent crops as well as agricultural grasslands were extracted from the Portuguese Land Parcel Identification System (LPIS) of the Instituto de Financiamento da Agricultura e Pescas (IFAP); land cover classes like urban, forest and water were trained from the Carta de Ocupação do Solo (COS) that is the national Land Use and Land Cover (LULC) map of Portugal; and lastly, the burned areas are identified from the corresponding national map of the Instituto da Conservação da Natureza e das Florestas (ICNF). Also, a set of preprocessing steps were defined based on the implementation of ancillary data allowing to avoid the inclusion of mislabeled pixels to the classifier. Mislabeling of pixels can occur due to errors in digitalization, generalization, and differences in the Minimum Mapping Unit (MMU) between datasets. An inner buffer was applied to all datasets to reduce border overlap among classes; the mask from the ICNF was applied to remove burned areas, and NDVI rule based on Landsat 8 allowed to erase recent clear-cuts in the forest. Also, the Copernicus High-Resolution Layers (HRL) datasets from 2015 (latest available), namely Dominant Leaf Type (DLT) and Tree Cover Density (TCD) are used to distinguish between forest with more than 60% coverage (coniferous and broadleaf) such as Holm Oak and Stone Pine and between 10 and 60% (coniferous) for instance Open Maritime Pine. Next, temporal gap-filled monthly composites were created for the agricultural period in Portugal, ranging from October 2017 till September 2018. The composites provided data free of missing values in opposition to single date acquisition images. Finally, a pixel-based approach classification was carried out in the “Tejo and Sado” region of Portugal using Random Forest (RF). The resulting map achieves a 76% overall accuracy for 31 classes (17 land cover and 14 crop types). The RF algorithm captured the most relevant features for the classification from the cloud-free composites, mainly during the spring and summer and in the bands on the Red Edge, NIR and SWIR. Overall, the classification was more successful on the irrigated temporary crops whereas the grasslands presented the most complexity to classify as they were confused with other rainfed crops and burned areas