A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Design and Evaluation of Data Dissemination Algorithms to Improve Object Detection in Autonomous Driving Networks

In the last few years, the amount of information that is produced by an autonomous vehicle is increasing proportionally with the number and resolution of sensors that cars are equipped with. Cars can be provided with cameras and Light Detection and Ranging (LiDAR) sensors, respectively needed to obtain a two-dimensional (2D) and three-dimensional (3D) representation of the environment. Due to the huge amount of data that multiple self-driving vehicles can push over a communication network, how these data are selected, stored, and sent is crucial. Various techniques have been developed to manage vehicular data; for example, compression can be used to alleviate the burden of data transmission over bandwidth-constrained channels and facilitate real-time communications. However, aggressive levels of compression may corrupt automotive data, and prevent proper detection of critical road objects in the scene. Along these lines, in this thesis, we studied the trade-off between compression efficiency and accuracy. To do so, we considered synthetic automotive data generated from the SELMA dataset. Then, we compared the performance of several state-of-the-art algorithms, based on machine learning, for compressing and detecting objects on LiDAR point clouds. We were able to reduce the point cloud by tens to hundreds times without any significant loss in the final detection accuracy. In a second phase, we focused our attention on the optimization of the number and type of sensors that are more meaningful to object detection operations. Notably, we tested our dataset on a sensor fusion algorithm that can combine both 2D and 3D data to have a better understanding of the environment. The results show that, although sensor fusion always achieves more accurate detections, using 3D inputs only can obtain similar results for large objects while mitigating the burden on the channel.In the last few years, the amount of information that is produced by an autonomous vehicle is increasing proportionally with the number and resolution of sensors that cars are equipped with. Cars can be provided with cameras and Light Detection and Ranging (LiDAR) sensors, respectively needed to obtain a two-dimensional (2D) and three-dimensional (3D) representation of the environment. Due to the huge amount of data that multiple self-driving vehicles can push over a communication network, how these data are selected, stored, and sent is crucial. Various techniques have been developed to manage vehicular data; for example, compression can be used to alleviate the burden of data transmission over bandwidth-constrained channels and facilitate real-time communications. However, aggressive levels of compression may corrupt automotive data, and prevent proper detection of critical road objects in the scene. Along these lines, in this thesis, we studied the trade-off between compression efficiency and accuracy. To do so, we considered synthetic automotive data generated from the SELMA dataset. Then, we compared the performance of several state-of-the-art algorithms, based on machine learning, for compressing and detecting objects on LiDAR point clouds. We were able to reduce the point cloud by tens to hundreds times without any significant loss in the final detection accuracy. In a second phase, we focused our attention on the optimization of the number and type of sensors that are more meaningful to object detection operations. Notably, we tested our dataset on a sensor fusion algorithm that can combine both 2D and 3D data to have a better understanding of the environment. The results show that, although sensor fusion always achieves more accurate detections, using 3D inputs only can obtain similar results for large objects while mitigating the burden on the channel

Assessing the role of EO in biodiversity monitoring: options for integrating in-situ observations with EO within the context of the EBONE concept

The European Biodiversity Observation Network (EBONE) is a European contribution on terrestrial monitoring to GEO BON, the Group on Earth Observations Biodiversity Observation Network. EBONE’s aims are to develop a system of biodiversity observation at regional, national and European levels by assessing existing approaches in terms of their validity and applicability starting in Europe, then expanding to regions in Africa. The objective of EBONE is to deliver: 1. A sound scientific basis for the production of statistical estimates of stock and change of key indicators; 2. The development of a system for estimating past changes and forecasting and testing policy options and management strategies for threatened ecosystems and species; 3. A proposal for a cost-effective biodiversity monitoring system. There is a consensus that Earth Observation (EO) has a role to play in monitoring biodiversity. With its capacity to observe detailed spatial patterns and variability across large areas at regular intervals, our instinct suggests that EO could deliver the type of spatial and temporal coverage that is beyond reach with in-situ efforts. Furthermore, when considering the emerging networks of in-situ observations, the prospect of enhancing the quality of the information whilst reducing cost through integration is compelling. This report gives a realistic assessment of the role of EO in biodiversity monitoring and the options for integrating in-situ observations with EO within the context of the EBONE concept (cfr. EBONE-ID1.4). The assessment is mainly based on a set of targeted pilot studies. Building on this assessment, the report then presents a series of recommendations on the best options for using EO in an effective, consistent and sustainable biodiversity monitoring scheme. The issues that we faced were many: 1. Integration can be interpreted in different ways. One possible interpretation is: the combined use of independent data sets to deliver a different but improved data set; another is: the use of one data set to complement another dataset. 2. The targeted improvement will vary with stakeholder group: some will seek for more efficiency, others for more reliable estimates (accuracy and/or precision); others for more detail in space and/or time or more of everything. 3. Integration requires a link between the datasets (EO and in-situ). The strength of the link between reflected electromagnetic radiation and the habitats and their biodiversity observed in-situ is function of many variables, for example: the spatial scale of the observations; timing of the observations; the adopted nomenclature for classification; the complexity of the landscape in terms of composition, spatial structure and the physical environment; the habitat and land cover types under consideration. 4. The type of the EO data available varies (function of e.g. budget, size and location of region, cloudiness, national and/or international investment in airborne campaigns or space technology) which determines its capability to deliver the required output. EO and in-situ could be combined in different ways, depending on the type of integration we wanted to achieve and the targeted improvement. We aimed for an improvement in accuracy (i.e. the reduction in error of our indicator estimate calculated for an environmental zone). Furthermore, EO would also provide the spatial patterns for correlated in-situ data. EBONE in its initial development, focused on three main indicators covering: (i) the extent and change of habitats of European interest in the context of a general habitat assessment; (ii) abundance and distribution of selected species (birds, butterflies and plants); and (iii) fragmentation of natural and semi-natural areas. For habitat extent, we decided that it did not matter how in-situ was integrated with EO as long as we could demonstrate that acceptable accuracies could be achieved and the precision could consistently be improved. The nomenclature used to map habitats in-situ was the General Habitat Classification. We considered the following options where the EO and in-situ play different roles: using in-situ samples to re-calibrate a habitat map independently derived from EO; improving the accuracy of in-situ sampled habitat statistics, by post-stratification with correlated EO data; and using in-situ samples to train the classification of EO data into habitat types where the EO data delivers full coverage or a larger number of samples. For some of the above cases we also considered the impact that the sampling strategy employed to deliver the samples would have on the accuracy and precision achieved. Restricted access to European wide species data prevented work on the indicator ‘abundance and distribution of species’. With respect to the indicator ‘fragmentation’, we investigated ways of delivering EO derived measures of habitat patterns that are meaningful to sampled in-situ observations

Comprehensive review of vision-based fall detection systems

Vision-based fall detection systems have experienced fast development over the last years. To determine the course of its evolution and help new researchers, the main audience of this paper, a comprehensive revision of all published articles in the main scientific databases regarding this area during the last five years has been made. After a selection process, detailed in the Materials and Methods Section, eighty-one systems were thoroughly reviewed. Their characterization and classification techniques were analyzed and categorized. Their performance data were also studied, and comparisons were made to determine which classifying methods best work in this field. The evolution of artificial vision technology, very positively influenced by the incorporation of artificial neural networks, has allowed fall characterization to become more resistant to noise resultant from illumination phenomena or occlusion. The classification has also taken advantage of these networks, and the field starts using robots to make these systems mobile. However, datasets used to train them lack real-world data, raising doubts about their performances facing real elderly falls. In addition, there is no evidence of strong connections between the elderly and the communities of researchers

Operationalization of Remote Sensing Solutions for Sustainable Forest Management

The great potential of remote sensing technologies for operational use in sustainable forest management is addressed in this book, which is the reprint of papers published in the Remote Sensing Special Issue “Operationalization of Remote Sensing Solutions for Sustainable Forest Management”. The studies come from three continents and cover multiple remote sensing systems (including terrestrial mobile laser scanning, unmanned aerial vehicles, airborne laser scanning, and satellite data acquisition) and a diversity of data processing algorithms, with a focus on machine learning approaches. The focus of the studies ranges from identification and characterization of individual trees to deriving national- or even continental-level forest attributes and maps. There are studies carefully describing exercises on the case study level, and there are also studies introducing new methodologies for transdisciplinary remote sensing applications. Even though most of the authors look forward to continuing their research, nearly all studies introduced are ready for operational use or have already been implemented in practical forestry

Automation and Control

Advances in automation and control today cover many areas of technology where human input is minimized. This book discusses numerous types and applications of automation and control. Chapters address topics such as building information modeling (BIM)–based automated code compliance checking (ACCC), control algorithms useful for military operations and video games, rescue competitions using unmanned aerial-ground robots, and stochastic control systems