The German Copernicus Data and Exploitation platform "CODE-DE" - online data access and big data processing

Abstract

Copernicus establishes an operational European Earth Observation capacity. Data from multiple sources - first and foremost the Sentinel satellite systems - feed services and provide users with reliable and up-to-date information regarding environmental and security issues. The main Sentinel Internet based data access infrastructure is implemented by ESA. In addition, several Copernicus Participating States established national "collaborative ground segments", aiming at the additional use and distribution of Copernicus datasets focusing on their national demand. Next to that the "Copernicus Data and Information Access Services" (C-DIAS) have recently been set up (Summer 2018). The five different C-DIAS platforms are contracted by ESA and EUMETSAT in the framework of a European Commission initiative. The German Copernicus Data and Exploitation Platform (CODE-DE) is an infrastructure for data access (Sentinel data, products from the Copernicus Services and national data), on-demand processing and value-added product generation. The user-friendly online data access has been operational since March 2017 and is tapped by over 1.200 registered users (August 2018), which have downloaded around 100.000 products in that timeframe. The catalogue client provides an enhanced solution to discover, view, and download available products including time, spatial and additional filters, e.g., polarization mode or cloud cover. It features a browsing to display Sentinel-2 products in full 10 m spatial resolution. In addition other satellite data (e.g. science data from our national missions) are also integrated into the portal. An on-demand processing environment for high-performance cloud processing has been operational since November 2017. The algorithms currently available are mainly those of the Sentinel tool boxes (SNAP) such as generic band math or dedicated time series algorithms. Furthermore, the results are optionally mosaicked to cover larger areas or a larger period. The capacity of the processing environment has been continuously exploited since March 2018 by selected applications, e.g. in the agricultural, maritime or snow and ice context, based on dedicated Virtual Machines (VM) using "docker". This flexible processing approach supports different use cases - allowing developers, processing experts and data experts to work on the platform in support of their own processing scenarios