902 research outputs found
Leveraging Vision Reconstruction Pipelines for Satellite Imagery
Reconstructing 3D geometry from satellite imagery is an important topic of
research. However, disparities exist between how this 3D reconstruction problem
is handled in the remote sensing context and how multi-view reconstruction
pipelines have been developed in the computer vision community. In this paper,
we explore whether state-of-the-art reconstruction pipelines from the vision
community can be applied to the satellite imagery. Along the way, we address
several challenges adapting vision-based structure from motion and multi-view
stereo methods. We show that vision pipelines can offer competitive speed and
accuracy in the satellite context.Comment: Project Page: https://kai-46.github.io/VisSat
Refined Equivalent Pinhole Model for Large-scale 3D Reconstruction from Spaceborne CCD Imagery
In this study, we present a large-scale earth surface reconstruction pipeline
for linear-array charge-coupled device (CCD) satellite imagery. While
mainstream satellite image-based reconstruction approaches perform
exceptionally well, the rational functional model (RFM) is subject to several
limitations. For example, the RFM has no rigorous physical interpretation and
differs significantly from the pinhole imaging model; hence, it cannot be
directly applied to learning-based 3D reconstruction networks and to more novel
reconstruction pipelines in computer vision. Hence, in this study, we introduce
a method in which the RFM is equivalent to the pinhole camera model (PCM),
meaning that the internal and external parameters of the pinhole camera are
used instead of the rational polynomial coefficient parameters. We then derive
an error formula for this equivalent pinhole model for the first time,
demonstrating the influence of the image size on the accuracy of the
reconstruction. In addition, we propose a polynomial image refinement model
that minimizes equivalent errors via the least squares method. The experiments
were conducted using four image datasets: WHU-TLC, DFC2019, ISPRS-ZY3, and GF7.
The results demonstrated that the reconstruction accuracy was proportional to
the image size. Our polynomial image refinement model significantly enhanced
the accuracy and completeness of the reconstruction, and achieved more
significant improvements for larger-scale images.Comment: 24 page
Assessment of IBM and NASA's geospatial foundation model in flood inundation mapping
Vision foundation models are a new frontier in GeoAI research because of
their potential to enable powerful image analysis by learning and extracting
important image features from vast amounts of geospatial data. This paper
evaluates the performance of the first-of-its-kind geospatial foundation model,
IBM-NASA's Prithvi, to support a crucial geospatial analysis task: flood
inundation mapping. This model is compared with popular convolutional neural
network and vision transformer-based architectures in terms of mapping accuracy
for flooded areas. A benchmark dataset, Sen1Floods11, is used in the
experiments, and the models' predictability, generalizability, and
transferability are evaluated based on both a test dataset and a dataset that
is completely unseen by the model. Results show the impressive transferability
of the Prithvi model, highlighting its performance advantages in segmenting
flooded areas in previously unseen regions. The findings also suggest areas for
improvement for the Prithvi model in terms of adopting multi-scale
representation learning, developing more end-to-end pipelines for high-level
image analysis tasks, and offering more flexibility in terms of input data
bands.Comment: 11 pages, 4 figure
Adaptively Lossy Image Compression for Onboard Processing
More efficient image-compression codecs are an emerging requirement for spacecraft because increasingly complex, onboard image sensors can rapidly saturate downlink bandwidth of communication transceivers. While these codecs reduce transmitted data volume, many are compute-intensive and require rapid processing to sustain sensor data rates. Emerging next-generation small satellite (SmallSat) computers provide compelling computational capability to enable more onboard processing and compression than previously considered. For this research, we apply two compression algorithms for deployment on modern flight hardware: (1) end-to-end, neural-network-based, image compression (CNN-JPEG); and (2) adaptive image compression through feature-point detection (FPD-JPEG). These algorithms rely on intelligent data-processing pipelines that adapt to sensor data to compress it more effectively, ensuring efficient use of limited downlink bandwidths. The first algorithm, CNN-JPEG, employs a hybrid approach adapted from literature combining convolutional neural networks (CNNs) and JPEG; however, we modify and tune the training scheme for satellite imagery to account for observed training instabilities. This hybrid CNN-JPEG approach shows 23.5% better average peak signal-to-noise ratio (PSNR) and 33.5% better average structural similarity index (SSIM) versus standard JPEG on a dataset collected on the Space Test Program – Houston 5 (STP-H5-CSP) mission onboard the International Space Station (ISS). For our second algorithm, we developed a novel adaptive image-compression pipeline based upon JPEG that leverages the Oriented FAST and Rotated BRIEF (ORB) feature-point detection algorithm to adaptively tune the compression ratio to allow for a tradeoff between PSNR/SSIM and combined file size over a batch of STP-H5-CSP images. We achieve a less than 1% drop in average PSNR and SSIM while reducing the combined file size by 29.6% compared to JPEG using a static quality factor (QF) of 90
RSL-Net: Localising in Satellite Images From a Radar on the Ground
This paper is about localising a vehicle in an overhead image using FMCW
radar mounted on a ground vehicle. FMCW radar offers extraordinary promise and
efficacy for vehicle localisation. It is impervious to all weather types and
lighting conditions. However the complexity of the interactions between
millimetre radar wave and the physical environment makes it a challenging
domain. Infrastructure-free large-scale radar-based localisation is in its
infancy. Typically here a map is built and suitable techniques, compatible with
the nature of sensor, are brought to bear. In this work we eschew the need for
a radar-based map; instead we simply use an overhead image -- a resource
readily available everywhere. This paper introduces a method that not only
naturally deals with the complexity of the signal type but does so in the
context of cross modal processing.Comment: Accepted to IEEE Robotics and Automation Letters (RA-L
- …