113 research outputs found
A Review on Deep Learning in UAV Remote Sensing
Deep Neural Networks (DNNs) learn representation from data with an impressive
capability, and brought important breakthroughs for processing images,
time-series, natural language, audio, video, and many others. In the remote
sensing field, surveys and literature revisions specifically involving DNNs
algorithms' applications have been conducted in an attempt to summarize the
amount of information produced in its subfields. Recently, Unmanned Aerial
Vehicles (UAV) based applications have dominated aerial sensing research.
However, a literature revision that combines both "deep learning" and "UAV
remote sensing" thematics has not yet been conducted. The motivation for our
work was to present a comprehensive review of the fundamentals of Deep Learning
(DL) applied in UAV-based imagery. We focused mainly on describing
classification and regression techniques used in recent applications with
UAV-acquired data. For that, a total of 232 papers published in international
scientific journal databases was examined. We gathered the published material
and evaluated their characteristics regarding application, sensor, and
technique used. We relate how DL presents promising results and has the
potential for processing tasks associated with UAV-based image data. Lastly, we
project future perspectives, commentating on prominent DL paths to be explored
in the UAV remote sensing field. Our revision consists of a friendly-approach
to introduce, commentate, and summarize the state-of-the-art in UAV-based image
applications with DNNs algorithms in diverse subfields of remote sensing,
grouping it in the environmental, urban, and agricultural contexts.Comment: 38 pages, 10 figure
Unlocking the capabilities of explainable fewshot learning in remote sensing
Recent advancements have significantly improved the efficiency and
effectiveness of deep learning methods for imagebased remote sensing tasks.
However, the requirement for large amounts of labeled data can limit the
applicability of deep neural networks to existing remote sensing datasets. To
overcome this challenge, fewshot learning has emerged as a valuable approach
for enabling learning with limited data. While previous research has evaluated
the effectiveness of fewshot learning methods on satellite based datasets,
little attention has been paid to exploring the applications of these methods
to datasets obtained from UAVs, which are increasingly used in remote sensing
studies. In this review, we provide an up to date overview of both existing and
newly proposed fewshot classification techniques, along with appropriate
datasets that are used for both satellite based and UAV based data. Our
systematic approach demonstrates that fewshot learning can effectively adapt to
the broader and more diverse perspectives that UAVbased platforms can provide.
We also evaluate some SOTA fewshot approaches on a UAV disaster scene
classification dataset, yielding promising results. We emphasize the importance
of integrating XAI techniques like attention maps and prototype analysis to
increase the transparency, accountability, and trustworthiness of fewshot
models for remote sensing. Key challenges and future research directions are
identified, including tailored fewshot methods for UAVs, extending to unseen
tasks like segmentation, and developing optimized XAI techniques suited for
fewshot remote sensing problems. This review aims to provide researchers and
practitioners with an improved understanding of fewshot learnings capabilities
and limitations in remote sensing, while highlighting open problems to guide
future progress in efficient, reliable, and interpretable fewshot methods.Comment: Under review, once the paper is accepted, the copyright will be
transferred to the corresponding journa
Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works
Deep learning has achieved great success in learning features from massive
remote sensing images (RSIs). To better understand the connection between
feature learning paradigms (e.g., unsupervised feature learning (USFL),
supervised feature learning (SFL), and self-supervised feature learning
(SSFL)), this paper analyzes and compares them from the perspective of feature
learning signals, and gives a unified feature learning framework. Under this
unified framework, we analyze the advantages of SSFL over the other two
learning paradigms in RSIs understanding tasks and give a comprehensive review
of the existing SSFL work in RS, including the pre-training dataset,
self-supervised feature learning signals, and the evaluation methods. We
further analyze the effect of SSFL signals and pre-training data on the learned
features to provide insights for improving the RSI feature learning. Finally,
we briefly discuss some open problems and possible research directions.Comment: 24 pages, 11 figures, 3 table
A Survey on Evolutionary Computation for Computer Vision and Image Analysis: Past, Present, and Future Trends
Computer vision (CV) is a big and important field
in artificial intelligence covering a wide range of applications.
Image analysis is a major task in CV aiming to extract, analyse
and understand the visual content of images. However, imagerelated
tasks are very challenging due to many factors, e.g., high
variations across images, high dimensionality, domain expertise
requirement, and image distortions. Evolutionary computation
(EC) approaches have been widely used for image analysis with
significant achievement. However, there is no comprehensive
survey of existing EC approaches to image analysis. To fill
this gap, this paper provides a comprehensive survey covering
all essential EC approaches to important image analysis tasks
including edge detection, image segmentation, image feature
analysis, image classification, object detection, and others. This
survey aims to provide a better understanding of evolutionary
computer vision (ECV) by discussing the contributions of different
approaches and exploring how and why EC is used for
CV and image analysis. The applications, challenges, issues, and
trends associated to this research field are also discussed and
summarised to provide further guidelines and opportunities for
future research
Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery
Deep-learning frameworks have made remarkable progress thanks to the creation of large annotated datasets such as ImageNet, which has over one million training images. Although this works well for color (RGB) imagery, labeled datasets for other sensor modalities (e.g., multispectral and hyperspectral) are minuscule in comparison. This is because annotated datasets are expensive and man-power intensive to complete; and since this would be impractical to accomplish for each type of sensor, current state-of-the-art approaches in computer vision are not ideal for remote sensing problems. The shortage of annotated remote sensing imagery beyond the visual spectrum has forced researchers to embrace unsupervised feature extracting frameworks. These features are learned on a per-image basis, so they tend to not generalize well across other datasets. In this dissertation, we propose three new strategies for learning feature extracting frameworks with only a small quantity of annotated image data; including 1) self-taught feature learning, 2) domain adaptation with synthetic imagery, and 3) semi-supervised classification. ``Self-taught\u27\u27 feature learning frameworks are trained with large quantities of unlabeled imagery, and then these networks extract spatial-spectral features from annotated data for supervised classification. Synthetic remote sensing imagery can be used to boot-strap a deep convolutional neural network, and then we can fine-tune the network with real imagery. Semi-supervised classifiers prevent overfitting by jointly optimizing the supervised classification task along side one or more unsupervised learning tasks (i.e., reconstruction). Although obtaining large quantities of annotated image data would be ideal, our work shows that we can make due with less cost-prohibitive methods which are more practical to the end-user
A generic self-supervised learning (SSL) framework for representation learning from spectra-spatial feature of unlabeled remote sensing imagery
Remote sensing data has been widely used for various Earth Observation (EO)
missions such as land use and cover classification, weather forecasting,
agricultural management, and environmental monitoring. Most existing remote
sensing data-based models are based on supervised learning that requires large
and representative human-labelled data for model training, which is costly and
time-consuming. Recently, self-supervised learning (SSL) enables the models to
learn a representation from orders of magnitude more unlabelled data. This
representation has been proven to boost the performance of downstream tasks and
has potential for remote sensing applications. The success of SSL is heavily
dependent on a pre-designed pretext task, which introduces an inductive bias
into the model from a large amount of unlabelled data. Since remote sensing
imagery has rich spectral information beyond the standard RGB colour space, the
pretext tasks established in computer vision based on RGB images may not be
straightforward to be extended to the multi/hyperspectral domain. To address
this challenge, this work has designed a novel SSL framework that is capable of
learning representation from both spectra-spatial information of unlabelled
data. The framework contains two novel pretext tasks for object-based and
pixel-based remote sensing data analysis methods, respectively. Through two
typical downstream tasks evaluation (a multi-label land cover classification
task on Sentienl-2 multispectral datasets and a ground soil parameter retrieval
task on hyperspectral datasets), the results demonstrate that the
representation obtained through the proposed SSL achieved a significant
improvement in model performance
Self-supervised Learning in Remote Sensing: A Review
In deep learning research, self-supervised learning (SSL) has received great
attention triggering interest within both the computer vision and remote
sensing communities. While there has been a big success in computer vision,
most of the potential of SSL in the domain of earth observation remains locked.
In this paper, we provide an introduction to, and a review of the concepts and
latest developments in SSL for computer vision in the context of remote
sensing. Further, we provide a preliminary benchmark of modern SSL algorithms
on popular remote sensing datasets, verifying the potential of SSL in remote
sensing and providing an extended study on data augmentations. Finally, we
identify a list of promising directions of future research in SSL for earth
observation (SSL4EO) to pave the way for fruitful interaction of both domains.Comment: Accepted by IEEE Geoscience and Remote Sensing Magazine. 32 pages, 22
content page
ZeroWaste Dataset: Towards Deformable Object Segmentation in Extreme Clutter
Less than 35% of recyclable waste is being actually recycled in the US, which
leads to increased soil and sea pollution and is one of the major concerns of
environmental researchers as well as the common public. At the heart of the
problem are the inefficiencies of the waste sorting process (separating paper,
plastic, metal, glass, etc.) due to the extremely complex and cluttered nature
of the waste stream. Automated waste detection has great potential to enable
more efficient, reliable, and safe waste sorting practices, but it requires
label-efficient detection of deformable objects in extremely cluttered scenes.
This challenging computer vision task currently lacks suitable datasets or
methods in the available literature. In this paper, we take a step towards
computer-aided waste detection and present the first in-the-wild
industrial-grade waste detection and segmentation dataset, ZeroWaste. This
dataset contains over 1800 fully segmented video frames collected from a real
waste sorting plant along with waste material labels for training and
evaluation of the segmentation methods, as well as over 6000 unlabeled frames
that can be further used for semi-supervised and self-supervised learning
techniques, as well as frames of the conveyor belt before and after the sorting
process, comprising a novel setup that can be used for weakly-supervised
segmentation. Our experimental results demonstrate that state-of-the-art
segmentation methods struggle to correctly detect and classify target objects
which suggests the challenging nature of our proposed real-world task of
fine-grained object detection in cluttered scenes. We believe that ZeroWaste
will catalyze research in object detection and semantic segmentation in extreme
clutter as well as applications in the recycling domain.
Our project page can be found at http://ai.bu.edu/zerowaste/
- âŠ