Advanced Data Chain Technologies for the Next Generation of Earth Observation Satellites Supporting On-Board Processing for Rapid Civil Alerts

The growing number of planned Earth Observation (EO) satellites, together with the increase in payload resolution and swath, brings to the fore the generation of unprecedented volumes of data that needs to be downloaded, processed and distributed with low latency. This creates a severe bottleneck problem, which overloads ground infrastructure, communications to ground, and hampers the provision of EO products to the End User with the required performances. The European H2020 EO-ALERT project (http://eo-alert-h2020.eu/), proposes the definition of next-generation EO missions by developing an on-board high speed EO data processing chain, based on a novel flight segment architecture that moves optimised key EO data processing elements from the ground segment to on-board the satellite. EO-ALERT achieves, globally, latencies below five minutes for EO products delivery, reaching latencies below 1 minute in some scenarios. The proposed architecture solves the above challenges through a combination of innovations in the on-board elements of the data chain and the communications link. Namely, the architecture introduces innovative technological solutions, including on-board reconfigurable data handling, on-board image generation and processing for generation of alerts (EO products) using Artificial Intelligence (AI), high-speed on-board avionics, on-board data compression and encryption using AI and reconfigurable high data rate communication links to ground including a separate chain for alerts with minimum latency and global coverage. Those key technologies have been studied, developed, implemented in software/hardware (SW/HW) and verified against previously established technologies requirements to meet the identified user needs. The paper presents an overview of the development of the innovative solutions defined during the project for each of the above mentioned technological areas and the results of the testing campaign of the individual SW/HW implementations within the context of two operational scenarios: ship detection and extreme weather observation (nowcasting), both requiring a high responsiveness to events to reduce the response time to few hours, or even to minutes, after an emergency situation arises. The technologies have been experimentally evaluated during the project using relevant EO historical sensor data. The results demonstrate the maturity of the technologies, having now reached TRL 4-5. Generally, the results show that, when implemented using COTS components and available communication links, the proposed architecture can generate and delivery globally EO products/alerts with a latency lower than five minutes, which demonstrates the viability of the EO-ALERT concept. The paper also discusses the implementation on an Avionic Test Bench (ATB) for the validation of the integrated technologies chain

Advances in Object and Activity Detection in Remote Sensing Imagery

The recent revolution in deep learning has enabled considerable development in the fields of object and activity detection. Visual object detection tries to find objects of target classes with precise localisation in an image and assign each object instance a corresponding class label. At the same time, activity recognition aims to determine the actions or activities of an agent or group of agents based on sensor or video observation data. It is a very important and challenging problem to detect, identify, track, and understand the behaviour of objects through images and videos taken by various cameras. Together, objects and their activity recognition in imaging data captured by remote sensing platforms is a highly dynamic and challenging research topic. During the last decade, there has been significant growth in the number of publications in the field of object and activity recognition. In particular, many researchers have proposed application domains to identify objects and their specific behaviours from air and spaceborne imagery. This Special Issue includes papers that explore novel and challenging topics for object and activity detection in remote sensing images and videos acquired by diverse platforms

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Automated High-resolution Earth Observation Image Interpretation: Outcome of the 2020 Gaofen Challenge

In this article, we introduce the 2020 Gaofen Challenge and relevant scientific outcomes. The 2020 Gaofen Challenge is an international competition, which is organized by the China High-Resolution Earth Observation Conference Committee and the Aerospace Information Research Institute, Chinese Academy of Sciences and technically cosponsored by the IEEE Geoscience and Remote Sensing Society and the International Society for Photogrammetry and Remote Sensing. It aims at promoting the academic development of automated high-resolution earth observation image interpretation. Six independent tracks have been organized in this challenge, which cover the challenging problems in the field of object detection and semantic segmentation. With the development of convolutional neural networks, deep-learning-based methods have achieved good performance on image interpretation. In this article, we report the details and the best-performing methods presented so far in the scope of this challenge

Literature review of the remote sensing of natural resources

Abstracts of 596 documents related to remote sensors or the remote sensing of natural resources by satellite, aircraft, or ground-based stations are presented. Topics covered include general theory, geology and hydrology, agriculture and forestry, marine sciences, urban land use, and instrumentation. Recent documents not yet cited in any of the seven information sources used for the compilation are summarized. An author/key word index is provided

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Physics-based satellite-derived bathymetry for nearshore coastal waters in North America

Accurate bathymetric information is fundamental to safe maritime navigation and infrastructure development in the coastal zone, but is expensive to acquire with traditional methods. Satellite-derived bathymetry (SDB) has the potential to produce bathymetric maps at dramatically reduced cost per unit area and physics-based radiative transfer model inversion methods have been developed for this purpose. This thesis demonstrates the potential of physics-based SDB in North American coastal waters. First the utility of Landsat-8 data for SDB in Canadian waters was demonstrated. Given the need for precise atmospheric correction (AC) for deriving robust ocean color products such as bathymetry, the performances of different AC algorithms were then evaluated to determine the most appropriate AC algorithm for deriving ocean colour products such as bathymetry. Subsequently, an approach to minimize AC error was demonstrated for SDB in a coastal environment in Florida Keys, USA. Finally, an ensemble approach based on multiple images, with acquisitions ranging from optimal to sub-optimal conditions, was demonstrated. Based on the findings of this thesis, it was concluded that: (1) Landsat-8 data hold great promise for physics-based SDB in coastal environments, (2) the problem posed by imprecise AC can be minimized by assessing and quantifying bias as a function of environmental factors, and then removing that bias in the atmospherically corrected images, from which bathymetry is estimated, and (3) an ensemble approach to SDB can produce results that are very similar to those obtained with the best individual image, but can be used to reduce time spent on pre-screening and filtering of scenes