New Enabling Technologies to Observe and Characterise Urban Environments with Big Data from Space – the Urban Thematic Exploitation Platform

Modern Earth Observation (EO) satellite missions provide valuable opportunities to support sustainable urban planning and management by delivering dedicated information on the spatiotemporal development of the built environment and its key morphological and physical characteristics such as imperviousness, greenness, built-up density, building volume, albedo – from global down to local scale. However, the transformation of the raw EO imagery into ready-to-use thematic data and indicators for scientist or planners on the one hand and actionable information for decision makers on the other hand requires detailed technical expert knowledge. Moreover, the imagery collected by satellite missions such as the US Landsat program or the European fleet of Sentinel satellites, but also by airborne systems or drones, rapidly adds up to a multiple of the data volume that can effectively be handled with standard work stations and software solutions. Hence, this contribution introduces the Urban Thematic Exploitation Platform (https://urban-tep.eo.esa.int) that utilizes modern information and communication technology to bridge the gap between the mass data collections of the technology-driven EO sector and the demand of science, planning, and policy for up-to-date information on the status, properties and dynamics of the urban system. Key components of the Urban Thematic Exploitation Platform (U-TEP) are an open, web-based portal that is connected to distributed high-level computing clusters and clouds and that also provides key functionalities for i) high-performance data access, analysis and visualization, ii) customized development and sharing of algorithms, products and services, and iii) networking, communication and exchange of data and information. The overarching objective here is to enable any interested (non-expert) user to easily generate actionable indicators and information for effective sustainable urban development based on a joint analysis of various data sources such as official survey data, EO mission data, socio-economic statistics, and data collected via social media or citizen science. So far more than 3.5 PB of data have been processed and analyzed by means of the U-TEP to finally provide a broad spectrum of urban information products and related services for visualization and analytics that have yet successfully been used by more than 240 institutions (science, planning, NGOs, policy) from 41 countries (i.a. World Bank Group, United Nations, Organisation for Economic Cooperation and Development, World Food Programme, Bill and Melinda Gates Foundation, Group on Earth Observation, Global Platform for Sustainable Cities)

Creation of interface for creating apriori association rules

This work aims at finding or creating web application, which allows user to easily create and edit constructs similar to association rules. I summarized and compared existing solutions in the first part and decided that none of these solutions meets the requirements of SEWEBAR project. In the second part I focus on the formats, used for communication with world outside the application and for communication betweeen parts of the applications. This is followed by definition of the web application, which I created and in the end of this part I explain in deep the architecture of the application. The resulting application is part of the SEWEBAR project, but it is written in the way, which makes it easy to deploy it to any other web projects, which needs to create or edit constructs similar to association rules. In the work I also cover possible configurations for this deployment

Creation of interface for creating apriori association rules

Cílem práce bylo objevit nebo vytvořit webovou aplikaci umožňující snadnou a přehlednou tvorbu konstruktů podobných asociačním pravidlům. V první části práce shrnuji a porovnávám existující řešení. Došel jsem k závěru, že žádné z existujících řešení nevyhovuje požadavkům projektu SEWEBAR. V druhé části se věnuji formátům, které byly použity při komunikaci aplikace s okolním světem a při komunikaci mezi částmi aplikace, na což navazuji popisem webové aplikace, kterou jsem vytvořil, a v závěru této části rozebírám přesnou architekturu aplikace. Výsledná aplikace slouží jako součást projektu SEWEBAR, nicméně je napsána tak, aby byla snadno nasaditelná i do jiných webových projektů, které potřebují zobrazovat nebo upravovat konstrukty podobné asociačním pravidlům. V práci se věnuji i možnostem tohoto nasazení.This work aims at finding or creating web application, which allows user to easily create and edit constructs similar to association rules. I summarized and compared existing solutions in the first part and decided that none of these solutions meets the requirements of SEWEBAR project. In the second part I focus on the formats, used for communication with world outside the application and for communication betweeen parts of the applications. This is followed by definition of the web application, which I created and in the end of this part I explain in deep the architecture of the application. The resulting application is part of the SEWEBAR project, but it is written in the way, which makes it easy to deploy it to any other web projects, which needs to create or edit constructs similar to association rules. In the work I also cover possible configurations for this deployment

The Urban Thematic Exploitation Platform – New Enabling Technologies to Observe and Characterize Urban Environments from Space

Modern Earth Observation (EO) satellite missions provide valuable opportunities to support sustainable urban planning and management by delivering dedicated information on the spatiotemporal development of the built environment and its key morphological and physical characteristics such as imperviousness, greenness, built-up density, building volume, albedo – from global down to local scale. However, the transformation of the raw EO imagery into ready-to-use thematic data and indicators for scientist or planners on the one hand and actionable information for decision makers on the other hand requires detailed technical expert knowledge. Moreover, the imagery collected by satellite missions such as the US Landsat program or the European fleet of Sentinel satellites, but also by airborne systems or drones, rapidly adds up to a multiple of the data volume that can effectively be handled with standard work stations and software solutions. Hence, this contribution introduces the Urban Thematic Exploitation Platform (https://urban-tep.eo.esa.int) that utilizes modern information and communication technology to bridge the gap between the mass data collections of the technology-driven EO sector and the demand of science, planning, and policy for up-to-date information on the status, properties and dynamics of the urban system. Key components of the Urban Thematic Exploitation Platform (U-TEP) are an open, web-based portal that is connected to distributed high-level computing clusters and clouds and that also provides key functionalities for i) high-performance data access, analysis and visualization, ii) customized development and sharing of algorithms, products and services, and iii) networking, communication and exchange of data and information. The overarching objective here is to enable any interested (non-expert) user to easily generate actionable indicators and information for effective sustainable urban development based on a joint analysis of various data sources such as official survey data, EO mission data, socio-economic statistics, and data collected via social media or citizen science. So far more than 3.5 PB of data have been processed and analyzed by means of the U-TEP to finally provide a broad spectrum of urban information products and related services for visualization and analytics that have yet successfully been used by more than 240 institutions from 41 countries (i.a. World Bank Group, United Nations, Organisation for Economic Cooperation and Development, World Food Programme, Bill and Melinda Gates Foundation, Group on Earth Observation, Global Platform for Sustainable Cities)

Urban Thematic Exploitation Platform - supporting urban research with EO data processing, integrative data analysis and reporting

Urbanization is among the most relevant global trends that affects climate, environment, as well as health and socio-economic development of a majority of the global population. As such, it poses a major challenge for the current urban population and the well-being of the next generation. To understand how to take advantage of opportunities and properly mitigate to the negative impacts of this change, we need precise and up-to-date information of the urban areas. The Urban Thematic Exploitation Platform (UrbanTEP) is a collaborative system, which focuses on the processing of earth observation (EO) data and delivering multi-source information on trans-sectoral urban challenges. The U-TEP is developed to provide end-to-end and ready-to-use solutions for a broad spectrum of users (service providers, experts and non-experts) to extract unique information/ indicators required for urban management and sustainability. Key components of the system are an open, web-based portal connected to distributed high-level computing infrastructures and providing key functionalities for i) high-performance data access and processing, ii) modular and generic state-of-the art pre-processing, analysis, and visualization, iii) customized development and sharing of algorithms, products and services, and iv) networking and communication. The service and product portfolio provides access to the archives of Copernicus and Landsat missions, Datacube technology, DIAS processing environments, as well as premium products like the World Settlement Footprint (WSF). External service providers, as well as researchers can make use of on-demand processing of new data products and the possibility of developing and deploying new processors. The onboarding of service providers, developers and researchers is supported by the Network of Resources program of the European Space Agency (ESA) and the OCRE initiative of the European Commission. In order to provide end-to-end solutions, the VISAT tool on UrbanTEP allows analyzing and visualizing project-related geospatial content and to develop storylines to enhance the transport of research output to customers and stakeholders effectively. Multiple visualizations (scopes) are already predefined. One available scope exemplary illustrates the exploitation of the WSF-Evolution dataset by analyzing the settlement and population development for South-East Asian countries from 1985 to 2015 in the context of the Sustainable Development Goal (SDG) 11.3.1 indicator. Other open scopes focus on urban green, functional urban areas, land-use and urban heat island modelling (e.g.)

New Enabling Technologies to Observe and Characterize Urban Environments with Big Data from Space - The Urban Thematic Exploitation Platform

Modern Earth Observation (EO) satellite missions provide valuable opportunities to support sustainable urban planning and management by delivering dedicated information on the spatiotemporal development of the built environment and its key morphological and physical characteristics such as imperviousness, greenness, built-up density, building volume, albedo – from global down to local scale. However, the transformation of the raw EO imagery into ready-to-use thematic data and indicators for scientist or planners on the one hand and actionable information for decision makers on the other hand requires detailed technical expert knowledge. Moreover, the imagery collected by satellite missions such as the US Landsat program or the European fleet of Sentinel satellites, but also by airborne systems or drones, rapidly adds up to a multiple of the data volume that can effectively be handled with standard work stations and software solutions. Hence, this contribution introduces the Urban Thematic Exploitation Platform (https://urban-tep.eo.esa.int) that utilizes modern information and communication technology to bridge the gap between the mass data collections of the technology-driven EO sector and the demand of science, planning, and policy for up-to-date information on the status, properties and dynamics of the urban system. Key components of the Urban Thematic Exploitation Platform (U-TEP) are an open, web-based portal that is connected to distributed high-level computing clusters and clouds and that also provides key functionalities for i) high-performance data access, analysis and visualization, ii) customized development and sharing of algorithms, products and services, and iii) networking, communication and exchange of data and information. The overarching objective here is to enable any interested (non-expert) user to easily generate actionable indicators and information for effective sustainable urban development based on a joint analysis of various data sources such as official survey data, EO mission data, socio-economic statistics, and data collected via social media or citizen science. So far more than 3.5 PB of data have been processed and analyzed by means of the U-TEP to finally provide a broad spectrum of urban information products and related services for visualization and analytics that have yet successfully been used by more than 240 institutions (science, planning, NGOs, policy) from 41 countries (i.a. World Bank Group, United Nations, Organisation for Economic Cooperation and Development, World Food Programme, Bill and Melinda Gates Foundation, Group on Earth Observation, Global Platform for Sustainable Cities)

Urban Thematic Exploitation Platform (U-TEP) for SDG Monitoring

Urbanization and climate change, representing two of the most relevant global trends related to the human presence on the planet, challenge our environmental, societal and economic development. Detailed and reliable information on global human settlements can directly contribute to the monitoring and decision-making regarding the 2030 Agenda for Sustainable Development. Here, the provision of relevant data for the Sustainable Development Goals (SDG), specifically SDG 11, as well as for other global and national policy frameworks is of particular interest. Scientists, analysts, planners and decision makers need capabilities to effectively and efficiently access, process, and jointly analyze the constantly increasing, but often heterogeneous and large-volume data collections on the built environment. The Urban Thematic Exploitation Platform (U-TEP) is developed to provide end-to-end and ready-to-use solutions for the above mentioned users. Key components of the system are an open, web-based portal connected to distributed high-level computing infrastructures and providing key functionalities for i) high-performance data access and processing, ii) modular and generic state-of-the art pre-processing, analysis, and visualization, iii) customized development and sharing of algorithms, products and services, and iv) networking and communication. In addition, it provides access to earth observation (EO) data (Landsat and Copernicus archives) and a heterogeneous ensemble of ready-to-use products from various providers and sources, like geotagged tweets, TimeScan layer, WorldPop population distribution layer, Global Urban Footprint (GUF) and World Settlement Footprint (WSF) products. The Visualization and Analysis Tool (VISAT) on U-TEP allows experts and non-experts to conduct geospatial analyses and queries. Users can create their individual compositions of data and related analysis and visualization functionalities (scopes). In order to allow for the calculation and provision of SDG 11.3.1 ratio of land consumption rate to population growth rate, a dedicated scope was prepared and demonstrated for the region of SE-Asia. The WSF Evolution product, which delineates the development of urban settlements from 1985 to 2015 (derived from advanced Landsat-based analysis), as well as annual population data for four SE-Asian countries were used to track and compare their development path. All countries show an unbalanced development, demonstrated by a higher increase of land up-taken by urban settlement compared to an increase in population. The population of Cambodia doubled in the period of 1985 to 2015, while the settlement area is five times the area of 1985. In Thailand, the settlement area doubled, but the population grew only by 32 %. All results can be combined with other EO-based products or statistical values and can be visualized using several types of multi-temporal graph/chart modes. The demonstration study exemplifies that U-TEP is a powerful tool that can be utilized by administration practitioners and analysts in monitoring and reporting selected SDG indicators such as the SDG 11.3.1. The presentation of the platform will include a live demonstration of the capabilities to analyze urban information relevant for the SDGs with the above-mentioned examples

Adding Value to the Urban Data Value Chain With the Urban-TEP

Urbanization is among the most relevant global trends related to the human presence on the planet. As such, it poses a major challenge for the current urban population and the well-being of the next generation. To fully understand and properly mitigate the impact of this change, we need precise and up-to-date information of the urban areas. The Urban Thematic Exploitation Platform (U-TEP) focuses on delivering multi-source information on trans-sectoral urban challenges. In 2019, U-TEP functionalities and user-interface of were streamlined, new products and services added. With these developments, the platform entered the operational phase. The U-TEP is developed to provide end-to-end and ready-to-use solutions for a broad spectrum of users (experts and non-experts) to extract unique information/ indicators required for urban management and sustainability. Key components of the system are an open, web-based portal connected to distributed high-level computing infrastructures and providing key functionalities for i) high-performance data access and processing, ii) modular and generic state-of-the art pre-processing, analysis, and visualization, iii) customized development and sharing of algorithms, products and services, and iv) networking and communication. Besides the product portfolio, that provides access to the archives of Copernicus and Landsat missions, as well as products like the World Settlement Footprint (WSF), the on-demand processing of new data products and the possibility of developing and deploying new processors, U-TEP allows analyzing and visualizing project-related geospatial content with the VISAT tool. Multiple visualizations (scopes) are available for registered users, but can also be newly set up. One available scope exemplary illustrates the exploitation of the WSF-Evolution dataset by analyzing the settlement and population development for South-East Asian countries from 1985 to 2015 in the context of the Sustainable Development Goal (SDG) 11.3.1 indicator (Fig. 1). Other accessible scopes focus on green public space (SDG 11.7) or the interactive urban heat island modelling by entering new land-use scenarios by the user