New Enabling Technologies to Observe and Characterise Urban Environments with Big Data from Space – the Urban Thematic Exploitation Platform

Modern Earth Observation (EO) satellite missions provide valuable opportunities to support sustainable urban planning and management by delivering dedicated information on the spatiotemporal development of the built environment and its key morphological and physical characteristics such as imperviousness, greenness, built-up density, building volume, albedo – from global down to local scale. However, the transformation of the raw EO imagery into ready-to-use thematic data and indicators for scientist or planners on the one hand and actionable information for decision makers on the other hand requires detailed technical expert knowledge. Moreover, the imagery collected by satellite missions such as the US Landsat program or the European fleet of Sentinel satellites, but also by airborne systems or drones, rapidly adds up to a multiple of the data volume that can effectively be handled with standard work stations and software solutions. Hence, this contribution introduces the Urban Thematic Exploitation Platform (https://urban-tep.eo.esa.int) that utilizes modern information and communication technology to bridge the gap between the mass data collections of the technology-driven EO sector and the demand of science, planning, and policy for up-to-date information on the status, properties and dynamics of the urban system. Key components of the Urban Thematic Exploitation Platform (U-TEP) are an open, web-based portal that is connected to distributed high-level computing clusters and clouds and that also provides key functionalities for i) high-performance data access, analysis and visualization, ii) customized development and sharing of algorithms, products and services, and iii) networking, communication and exchange of data and information. The overarching objective here is to enable any interested (non-expert) user to easily generate actionable indicators and information for effective sustainable urban development based on a joint analysis of various data sources such as official survey data, EO mission data, socio-economic statistics, and data collected via social media or citizen science. So far more than 3.5 PB of data have been processed and analyzed by means of the U-TEP to finally provide a broad spectrum of urban information products and related services for visualization and analytics that have yet successfully been used by more than 240 institutions (science, planning, NGOs, policy) from 41 countries (i.a. World Bank Group, United Nations, Organisation for Economic Cooperation and Development, World Food Programme, Bill and Melinda Gates Foundation, Group on Earth Observation, Global Platform for Sustainable Cities)

Exploiting Big Earth Data from Space – First Experiences with the TimeScan Processing Chain

The European Sentinel missions and the latest generation of the United States Landsat satellites provide new opportunities for global environmental monitoring. They acquire imagery at spatial resolutions between 10 and 60 m in a temporal and spatial coverage that could before only be realized on the basis of lower resolution Earth observation data (>250 m). However, images gathered by these modern missions rapidly add up to data volume that can no longer be handled with standard work stations and software solutions. Hence, this contribution introduces the TimeScan concept which combines pre-existing tools to an exemplary modular pipeline for the flexible and scalable processing of massive image data collections on a variety of (private or public) computing clusters. The TimeScan framework covers solutions for data access to arbitrary mission archives (with different data provisioning policies) and data ingestion into a processing environment EO2Data module), mission specific pre-processing of multi-temporal data collections (Data2TimeS module), and the generation of a final TimeScan baseline product (TimeS2Stats module) providing a spectrally and temporally harmonized representation of the observed surfaces. Technically, a TimeScan layer aggregates the information content of hundreds or thousands of single images available for the area and time period of interest (i.e. up to hundreds of TBs or even PBs of data) into a higher level product with significantly reduced volume. In first test, the TimeScan pipeline has been used to process a global coverage of 452,799 multispectral Landsat–8 scenes acquired from 2013 to 2015, a global data-set of 25,550 Envisat ASAR radar images collected 2010–2012, and regional Sentinel–1 and Sentinel–2 collections of ∼1500 images acquired from 2014 to 2016. The resulting TimeScan products have already been successfully used in various studies related to the large-scale monitoring of environmental processes and their temporal dynamics

The Urban Thematic Exploitation Platform – New Enabling Technologies to Observe and Characterize Urban Environments from Space

Modern Earth Observation (EO) satellite missions provide valuable opportunities to support sustainable urban planning and management by delivering dedicated information on the spatiotemporal development of the built environment and its key morphological and physical characteristics such as imperviousness, greenness, built-up density, building volume, albedo – from global down to local scale. However, the transformation of the raw EO imagery into ready-to-use thematic data and indicators for scientist or planners on the one hand and actionable information for decision makers on the other hand requires detailed technical expert knowledge. Moreover, the imagery collected by satellite missions such as the US Landsat program or the European fleet of Sentinel satellites, but also by airborne systems or drones, rapidly adds up to a multiple of the data volume that can effectively be handled with standard work stations and software solutions. Hence, this contribution introduces the Urban Thematic Exploitation Platform (https://urban-tep.eo.esa.int) that utilizes modern information and communication technology to bridge the gap between the mass data collections of the technology-driven EO sector and the demand of science, planning, and policy for up-to-date information on the status, properties and dynamics of the urban system. Key components of the Urban Thematic Exploitation Platform (U-TEP) are an open, web-based portal that is connected to distributed high-level computing clusters and clouds and that also provides key functionalities for i) high-performance data access, analysis and visualization, ii) customized development and sharing of algorithms, products and services, and iii) networking, communication and exchange of data and information. The overarching objective here is to enable any interested (non-expert) user to easily generate actionable indicators and information for effective sustainable urban development based on a joint analysis of various data sources such as official survey data, EO mission data, socio-economic statistics, and data collected via social media or citizen science. So far more than 3.5 PB of data have been processed and analyzed by means of the U-TEP to finally provide a broad spectrum of urban information products and related services for visualization and analytics that have yet successfully been used by more than 240 institutions from 41 countries (i.a. World Bank Group, United Nations, Organisation for Economic Cooperation and Development, World Food Programme, Bill and Melinda Gates Foundation, Group on Earth Observation, Global Platform for Sustainable Cities)

Urban Thematic Exploitation Platform - supporting urban research with EO data processing, integrative data analysis and reporting

Urbanization is among the most relevant global trends that affects climate, environment, as well as health and socio-economic development of a majority of the global population. As such, it poses a major challenge for the current urban population and the well-being of the next generation. To understand how to take advantage of opportunities and properly mitigate to the negative impacts of this change, we need precise and up-to-date information of the urban areas. The Urban Thematic Exploitation Platform (UrbanTEP) is a collaborative system, which focuses on the processing of earth observation (EO) data and delivering multi-source information on trans-sectoral urban challenges. The U-TEP is developed to provide end-to-end and ready-to-use solutions for a broad spectrum of users (service providers, experts and non-experts) to extract unique information/ indicators required for urban management and sustainability. Key components of the system are an open, web-based portal connected to distributed high-level computing infrastructures and providing key functionalities for i) high-performance data access and processing, ii) modular and generic state-of-the art pre-processing, analysis, and visualization, iii) customized development and sharing of algorithms, products and services, and iv) networking and communication. The service and product portfolio provides access to the archives of Copernicus and Landsat missions, Datacube technology, DIAS processing environments, as well as premium products like the World Settlement Footprint (WSF). External service providers, as well as researchers can make use of on-demand processing of new data products and the possibility of developing and deploying new processors. The onboarding of service providers, developers and researchers is supported by the Network of Resources program of the European Space Agency (ESA) and the OCRE initiative of the European Commission. In order to provide end-to-end solutions, the VISAT tool on UrbanTEP allows analyzing and visualizing project-related geospatial content and to develop storylines to enhance the transport of research output to customers and stakeholders effectively. Multiple visualizations (scopes) are already predefined. One available scope exemplary illustrates the exploitation of the WSF-Evolution dataset by analyzing the settlement and population development for South-East Asian countries from 1985 to 2015 in the context of the Sustainable Development Goal (SDG) 11.3.1 indicator. Other open scopes focus on urban green, functional urban areas, land-use and urban heat island modelling (e.g.)

UrbanTEP – a Collaborative Work Environment for EO Service Providers, Urban Researchers and User Communities

Urbanization is one of the key global challenges and affects the environment, as well as health and socio-economic development of a majority of the global population. It is also a major contributor to climate change. Collaborative efforts are needed in order to mitigate the negative impacts of urbanization on the one hand, but on the other hand to understand how to take advantage of opportunities that arise out of this process. Platform based initiatives have proven to be game changing technologies, which are capable of revolutionizing service provision, workflows and products. The Urban Thematic Exploitation Platform (UrbanTEP; urban-tep.eu) is a collaborative system, which focuses on the processing of earth observation (EO) data and delivering multi-source information on trans-sectoral urban challenges. The UrbanTEP is developed to provide end-to-end and ready-to-use solutions for a broad spectrum of users (service providers, expert and non-expert users and researchers) to extract unique information/ indicators required for urban planning, urban management and sustainable development. Key components of the system are an open, web-based portal connected to distributed and scalable high-level computing infrastructures and providing key functionalities for i) high-performance data access and processing (IaaS – Infrastructure as a Service), ii) modular and generic state-of-the art pre-processing, analysis, and visualization tools and algorithms (SaaS – Software as a Service), iii) customized development and sharing of algorithms, products and services (PaaS – Platform as a Service), and iv) networking and communication. The service and product portfolio of UrbanTEP provides access to several EO product archives (e.g. Copernicus, Landsat, etc.), as well as premium products like the World Settlement Footprint (WSF) and of the European Space Agency (ESA) Climate Change Initiative (CCI). The platform provides access to the Copernicus Data and Information Access Services (DIAS) processing environments. A cooperation with the Euro Data Cube allows users and service providers to make use of data cube technology and customized City Data Cubes enable user communities to derive actionable information from extensive EO-datasets. External service providers, as well as researchers can make use of on-demand processing of new data products and the possibility of developing and deploying new processors. The onboarding of service providers, developers and researchers is supported by the Network of Resources program of ESA and the OCRE initiative of the European Commission. In order to provide end-to-end solutions, the Visualization and Analysis Tool (VISAT) on UrbanTEP allows analyzing and visualizing project-related geospatial content and to develop storylines to enhance the transport of research output to customers and stakeholders effectively. Multiple visualizations (scopes) are already predefined and cover a wide range of applications like land-use monitoring, urban climate modelling, functional urban areas, population development and ecosystem accounting. One available scope exemplary illustrates the exploitation of the WSF-Evolution dataset by analyzing the settlement and population development for South-East Asian countries from 1985 to 2015 in the context of the Sustainable Development Goal (SDG) 11.3.1 indicator

UrbanTEP – a Collaborative Work Environment for EO Service Providers, Urban Researchers and User Communities

Urbanization is one of the key global challenges and affects the environment, as well as health and socio-economic development of a majority of the global population. It is also a major contributor to climate change. Collaborative efforts are needed in order to mitigate the negative impacts of urbanization on the one hand, but on the other hand to understand how to take advantage of opportunities that arise out of this process. Platform based initiatives have proven to be game changing technologies, which are capable of revolutionizing service provision, workflows and products. The Urban Thematic Exploitation Platform (UrbanTEP; urban-tep.eu) is a collaborative system, which focuses on the processing of earth observation (EO) data and delivering multi-source information on trans-sectoral urban challenges. The UrbanTEP is developed to provide end-to-end and ready-to-use solutions for a broad spectrum of users (service providers, expert and non-expert users and researchers) to extract unique information/ indicators required for urban planning, urban management and sustainable development. Key components of the system are an open, web-based portal connected to distributed and scalable high-level computing infrastructures and providing key functionalities for i) high-performance data access and processing (IaaS – Infrastructure as a Service), ii) modular and generic state-of-the art pre-processing, analysis, and visualization tools and algorithms (SaaS – Software as a Service), iii) customized development and sharing of algorithms, products and services (PaaS – Platform as a Service), and iv) networking and communication. The service and product portfolio of UrbanTEP provides access to several EO product archives (e.g. Copernicus, Landsat, etc.), as well as premium products like the World Settlement Footprint (WSF) and of the European Space Agency (ESA) Climate Change Initiative (CCI). The platform provides access to the Copernicus Data and Information Access Services (DIAS) processing environments. A cooperation with the Euro Data Cube allows users and service providers to make use of data cube technology and customized City Data Cubes enable user communities to derive actionable information from extensive EO-datasets. External service providers, as well as researchers can make use of on-demand processing of new data products and the possibility of developing and deploying new processors. The onboarding of service providers, developers and researchers is supported by the Network of Resources program of ESA and the OCRE initiative of the European Commission. In order to provide end-to-end solutions, the Visualization and Analysis Tool (VISAT) on UrbanTEP allows analyzing and visualizing project-related geospatial content and to develop storylines to enhance the transport of research output to customers and stakeholders effectively. Multiple visualizations (scopes) are already predefined and cover a wide range of applications like land-use monitoring, urban climate modelling, functional urban areas, population development and ecosystem accounting. One available scope exemplary illustrates the exploitation of the WSF-Evolution dataset by analyzing the settlement and population development for South-East Asian countries from 1985 to 2015 in the context of the Sustainable Development Goal (SDG) 11.3.1 indicator