Image super-resolution with dense-sampling residual channel-spatial attention networks for multi-temporal remote sensing image classification

Image super-resolution (SR) techniques can benefit a wide range of applications in the remote sensing (RS) community, including image classification. This issue is particularly relevant for image classification on time series data, considering RS datasets that feature long temporal coverage generally have a limited spatial resolution. Recent advances in deep learning brought new opportunities for enhancing the spatial resolution of historic RS data. Numerous convolutional neural network (CNN)-based methods showed superior performance in terms of developing efficient end-to-end SR models for natural images. However, such models were rarely exploited for promoting image classification based on multispectral RS data. This paper proposes a novel CNNbased framework to enhance the spatial resolution of time series multispectral RS images. Thereby, the proposed SR model employs Residual Channel Attention Networks (RCAN) as a backbone structure, whereas based on this structure the proposed models uniquely integrate tailored channel-spatial attention and dense-sampling mechanisms for performance improvement. Subsequently, state-of-the-art CNN-based classifiers are incorporated to produce classification maps based on the enhanced time series data. The experiments proved that the proposed SR model can enable unambiguously better performance compared to RCAN and other (deep learning-based) SR techniques, especially in a domain adaptation context, i.e., leveraging Sentinel-2 images for generating SR Landsat images. Furthermore, the experimental results confirmed that the enhanced multi-temporal RS images can bring substantial improvement on fine-grained multi-temporal land use classification

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Feasibility of Remote Sensing Based Deep Learning in Crop Yield Prediction

In this dissertation the applicability of novel machine learning methods with remote sensing data was studied in the context of agricultural decision support systems in smart farming. The main focus was the utilization of high-resolution unmanned aerial vehicle (UAV) data to perform in-season crop yield estimation with spatial and spatio-temporal deep learning model architectures in a Finnish coastal habitat. While open-access satellite data has already been utilized in crop-related modelling, such as crop type classification and yield prediction, intra-field scale prediction for the smaller fields common in the Nordic countries requires images with higher resolution than currently available from open-access satellite systems. In addition to using UAV remote sensing data, various combinations of crop field related sensor data, data from open-access sources and satellite data were evaluated. Data quality is also an important aspect with remote sensing data, with high altitude satellite-based earth observation suffering from occasional obstructions by the cloud canopy. A decision tree model was employed to estimate cloud coverage by using UAV data as cloudless ground truth. In this dissertation it is shown that crop yield prediction with convolutional neural networks (CNNs) is feasible with high-resolution UAV data and produces results accurate enough for performing corrective farming actions in-season. Using UAV data time series not only improves the modelling performance (post-season prediction) with high-resolution UAV RGB data but also improves the predictive capabilities (in-season prediction). Furthermore, the use of various data sources for crop yield prediction in addition to UAV RGB data is shown to improve the predictive capabilities of the model. In summary, the use of deep learning techniques can be seen to improve the smart farming decision support pipeline by providing performant and reliable decision engines

Air passenger demand forecast through the use of Artificial Neural Network algorithms

Airport planning depends to a large extent on the levels of activity that are anticipated. To plan the facilities and infrastructures of an airport system and to be able to satisfy future needs, it is essential to predict the level and distribution of demand. This document presents a short- and medium-term forecast of the demand for air passengers carried out through a specific case study (Colombia), in which the impact of the pandemic period due to COVID-19 on air traffic was taken into account. To make the forecast, an algorithm that implements techniques based on Artificial Neural Networks (ANN) (Machine Learning (ML)) was developed. In particular, for the analysis of the available time series, techniques of encoder-decoder networks of the type ConvLSTM2D have been applied. These architectures are a hybrid between Convolutional Neural Networks (CNN), very useful for the extraction of invariant patterns in their spatial position, and Recurrent Neural Networks (RNN), appropriate for the extraction of patterns within their temporal context (time series). The most relevant result of the present research is that the recovery in demand (volume and trend) to the levels reported before the pandemic is forecast for the period between the end of 2022 and the beginning of 2024 (depending on the type of traffic and scenario considered). Finally, the application of the forecasting model based on Machine Learning/Deep Learning (DL) presents, as a metric performance, a Mean Absolute Percentage Error (MAPE) value from 3% to 9% (depending on the scenario), which enables predictions of relative precision and introduces a new alternative technical approach to develop reliable air traffic forecasts, at least in the short and medium term

A Review of the Challenges of Using Deep Learning Algorithms to Support Decision-Making in Agricultural Activities

Deep Learning has been successfully applied to image recognition, speech recognition, and natural language processing in recent years. Therefore, there has been an incentive to apply it in other fields as well. The field of agriculture is one of the most important fields in which the application of deep learning still needs to be explored, as it has a direct impact on human well-being. In particular, there is a need to explore how deep learning models can be used as a tool for optimal planting, land use, yield improvement, production/disease/pest control, and other activities. The vast amount of data received from sensors in smart farms makes it possible to use deep learning as a model for decision-making in this field. In agriculture, no two environments are exactly alike, which makes testing, validating, and successfully implementing such technologies much more complex than in most other industries. This paper reviews some recent scientific developments in the field of deep learning that have been applied to agriculture, and highlights some challenges and potential solutions using deep learning algorithms in agriculture. The results in this paper indicate that by employing new methods from deep learning, higher performance in terms of accuracy and lower inference time can be achieved, and the models can be made useful in real-world applications. Finally, some opportunities for future research in this area are suggested.This work is supported by the R&D Project BioDAgro—Sistema operacional inteligente de informação e suporte á decisão em AgroBiodiversidade, project PD20-00011, promoted by Fundação La Caixa and Fundação para a Ciência e a Tecnologia, taking place at the C-MAST-Centre for Mechanical and Aerospace Sciences and Technology, Department of Electromechanical Engineering of the University of Beira Interior, Covilhã, Portugal.info:eu-repo/semantics/publishedVersio

Air passenger demand forecast through the use of Artificial Neural Network algorithms

Airport planning depends to a large extent on the levels of activity that are anticipated. In order to plan facilities and infrastructures of an airport system and to be able to satisfy future needs, it is essential to predict the level and distribution of demand. This document presents a short- and medium-term forecast of the demand for air passengers carried out through a specific case study (Colombia), in which the impact of the pandemic period due to COVID-19 on air traffic was taken into account. To make the forecast, an algorithm that implements techniques based on Artificial Neural Networks (ANN) and Machine Learning (ML) was developed. In particular, for the analysis of the available time series, techniques of encoder-decoder networks of the type ConvLSTM2D have been applied. These architectures are a hybrid between Convolutional Neural Networks (CNN), very useful for the extraction of invariant patterns in their spatial position, and Recurrent Neural Networks (RNN), appropriate for the extraction of patterns within their temporal context (time series). The most relevant result of the present research is that the recovery in demand (volume and trend) to the levels reported before the pandemic is forecast for the period between the end of 2022 and the beginning of 2024 (depending on the type of traffic and scenario considered). Finally, the application of the forecasting model based on ML/Deep Learning (DL) presents, as a metric performance, a Mean Absolute Percentage Error (MAPE) values from 3% to 9% (depending on the scenario), which enables predictions of relative precision and introduces a new alternative technical approach to develop reliable air traffic forecasts, at least in the short and medium term

A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery

Semantic segmentation (classification) of Earth Observation imagery is a crucial task in remote sensing. This paper presents a comprehensive review of technical factors to consider when designing neural networks for this purpose. The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and transformer models, discussing prominent design patterns for these ANN families and their implications for semantic segmentation. Common pre-processing techniques for ensuring optimal data preparation are also covered. These include methods for image normalization and chipping, as well as strategies for addressing data imbalance in training samples, and techniques for overcoming limited data, including augmentation techniques, transfer learning, and domain adaptation. By encompassing both the technical aspects of neural network design and the data-related considerations, this review provides researchers and practitioners with a comprehensive and up-to-date understanding of the factors involved in designing effective neural networks for semantic segmentation of Earth Observation imagery.Comment: 145 pages with 32 figure

Unveiling the frontiers of deep learning: innovations shaping diverse domains

Deep learning (DL) enables the development of computer models that are capable of learning, visualizing, optimizing, refining, and predicting data. In recent years, DL has been applied in a range of fields, including audio-visual data processing, agriculture, transportation prediction, natural language, biomedicine, disaster management, bioinformatics, drug design, genomics, face recognition, and ecology. To explore the current state of deep learning, it is necessary to investigate the latest developments and applications of deep learning in these disciplines. However, the literature is lacking in exploring the applications of deep learning in all potential sectors. This paper thus extensively investigates the potential applications of deep learning across all major fields of study as well as the associated benefits and challenges. As evidenced in the literature, DL exhibits accuracy in prediction and analysis, makes it a powerful computational tool, and has the ability to articulate itself and optimize, making it effective in processing data with no prior training. Given its independence from training data, deep learning necessitates massive amounts of data for effective analysis and processing, much like data volume. To handle the challenge of compiling huge amounts of medical, scientific, healthcare, and environmental data for use in deep learning, gated architectures like LSTMs and GRUs can be utilized. For multimodal learning, shared neurons in the neural network for all activities and specialized neurons for particular tasks are necessary.Comment: 64 pages, 3 figures, 3 table

An Introduction to Deep Morphological Networks

Over the past decade, Convolutional Networks (ConvNets) have renewed the perspectives of the research and industrial communities. Although this deep learning technique may be composed of multiple layers, its core operation is the convolution, an important linear filtering process. Easy and fast to implement, convolutions actually play a major role, not only in ConvNets, but in digital image processing and analysis as a whole, being effective for several tasks. However, aside from convolutions, researchers also proposed and developed non-linear filters, such as operators provided by mathematical morphology. Even though these are not so computationally efficient as the linear filters, in general, they are able to capture different patterns and tackle distinct problems when compared to the convolutions. In this paper, we propose a new paradigm for deep networks where convolutions are replaced by non-linear morphological filters. Aside from performing the operation, the proposed Deep Morphological Network (DeepMorphNet) is also able to learn the morphological filters (and consequently the features) based on the input data. While this process raises challenging issues regarding training and actual implementation, the proposed DeepMorphNet proves to be able to extract features and solve problems that traditional architectures with standard convolution filters cannot