34 research outputs found
Learning representations in the hyperspectral domain in aerial imagery
We establish two new datasets with baselines and network architectures for the task of hyperspectral image analysis. The first dataset, AeroRIT, is a moving camera static scene captured from a flight and contains per pixel labeling across five categories for the task of semantic segmentation. The second dataset, RooftopHSI, helps design and interpret learnt features on hyperspectral object detection on scenes captured from an university rooftop. This dataset accounts for static camera, moving scene hyperspectral imagery. We further broaden the scope of our understanding of neural networks with the development of two novel algorithms - S4AL and S4AL+. We develop these frameworks on natural (color) imagery, by combining semi-supervised learning and active learning, and display promising results for learning with limited amount of labeled data, which can be extended to hyperspectral imagery. In this dissertation, we curated two new datasets for hyperspectral image analysis, significantly larger than existing datasets and broader in terms of categories for classification. We then adapt existing neural network architectures to function on the increased channel information, in a smart manner, to leverage all hyperspectral information. We also develop novel active learning algorithms on natural (color) imagery, and discuss the hope for expanding their functionality to hyperspectral imagery
HPFormer: Hyperspectral image prompt object tracking
Hyperspectral imagery contains abundant spectral information beyond the
visible RGB bands, providing rich discriminative details about objects in a
scene. Leveraging such data has the potential to enhance visual tracking
performance. While prior hyperspectral trackers employ CNN or hybrid
CNN-Transformer architectures, we propose a novel approach HPFormer on
Transformers to capitalize on their powerful representation learning
capabilities. The core of HPFormer is a Hyperspectral Hybrid Attention (HHA)
module which unifies feature extraction and fusion within one component through
token interactions. Additionally, a Transform Band Module (TBM) is introduced
to selectively aggregate spatial details and spectral signatures from the full
hyperspectral input for injecting informative target representations. Extensive
experiments demonstrate state-of-the-art performance of HPFormer on benchmark
NIR and VIS tracking datasets. Our work provides new insights into harnessing
the strengths of transformers and hyperspectral fusion to advance robust object
tracking
Object Tracking Based on Satellite Videos: A Literature Review
Video satellites have recently become an attractive method of Earth observation, providing consecutive images of the Earth’s surface for continuous monitoring of specific events. The development of on-board optical and communication systems has enabled the various applications of satellite image sequences. However, satellite video-based target tracking is a challenging research topic in remote sensing due to its relatively low spatial and temporal resolution. Thus, this survey systematically investigates current satellite video-based tracking approaches and benchmark datasets, focusing on five typical tracking applications: traffic target tracking, ship tracking, typhoon tracking, fire tracking, and ice motion tracking. The essential aspects of each tracking target are summarized, such as the tracking architecture, the fundamental characteristics, primary motivations, and contributions. Furthermore, popular visual tracking benchmarks and their respective properties are discussed. Finally, a revised multi-level dataset based on WPAFB videos is generated and quantitatively evaluated for future development in the satellite video-based tracking area. In addition, 54.3% of the tracklets with lower Difficulty Score (DS) are selected and renamed as the Easy group, while 27.2% and 18.5% of the tracklets are grouped into the Medium-DS group and the Hard-DS group, respectively
Correlation Filters for Unmanned Aerial Vehicle-Based Aerial Tracking: A Review and Experimental Evaluation
Aerial tracking, which has exhibited its omnipresent dedication and splendid
performance, is one of the most active applications in the remote sensing
field. Especially, unmanned aerial vehicle (UAV)-based remote sensing system,
equipped with a visual tracking approach, has been widely used in aviation,
navigation, agriculture,transportation, and public security, etc. As is
mentioned above, the UAV-based aerial tracking platform has been gradually
developed from research to practical application stage, reaching one of the
main aerial remote sensing technologies in the future. However, due to the
real-world onerous situations, e.g., harsh external challenges, the vibration
of the UAV mechanical structure (especially under strong wind conditions), the
maneuvering flight in complex environment, and the limited computation
resources onboard, accuracy, robustness, and high efficiency are all crucial
for the onboard tracking methods. Recently, the discriminative correlation
filter (DCF)-based trackers have stood out for their high computational
efficiency and appealing robustness on a single CPU, and have flourished in the
UAV visual tracking community. In this work, the basic framework of the
DCF-based trackers is firstly generalized, based on which, 23 state-of-the-art
DCF-based trackers are orderly summarized according to their innovations for
solving various issues. Besides, exhaustive and quantitative experiments have
been extended on various prevailing UAV tracking benchmarks, i.e., UAV123,
UAV123@10fps, UAV20L, UAVDT, DTB70, and VisDrone2019-SOT, which contain 371,903
frames in total. The experiments show the performance, verify the feasibility,
and demonstrate the current challenges of DCF-based trackers onboard UAV
tracking.Comment: 28 pages, 10 figures, submitted to GRS