Smart environment monitoring through micro unmanned aerial vehicles

In recent years, the improvements of small-scale Unmanned Aerial Vehicles (UAVs) in terms of flight time, automatic control, and remote transmission are promoting the development of a wide range of practical applications. In aerial video surveillance, the monitoring of broad areas still has many challenges due to the achievement of different tasks in real-time, including mosaicking, change detection, and object detection. In this thesis work, a small-scale UAV based vision system to maintain regular surveillance over target areas is proposed. The system works in two modes. The first mode allows to monitor an area of interest by performing several flights. During the first flight, it creates an incremental geo-referenced mosaic of an area of interest and classifies all the known elements (e.g., persons) found on the ground by an improved Faster R-CNN architecture previously trained. In subsequent reconnaissance flights, the system searches for any changes (e.g., disappearance of persons) that may occur in the mosaic by a histogram equalization and RGB-Local Binary Pattern (RGB-LBP) based algorithm. If present, the mosaic is updated. The second mode, allows to perform a real-time classification by using, again, our improved Faster R-CNN model, useful for time-critical operations. Thanks to different design features, the system works in real-time and performs mosaicking and change detection tasks at low-altitude, thus allowing the classification even of small objects. The proposed system was tested by using the whole set of challenging video sequences contained in the UAV Mosaicking and Change Detection (UMCD) dataset and other public datasets. The evaluation of the system by well-known performance metrics has shown remarkable results in terms of mosaic creation and updating, as well as in terms of change detection and object detection

UNSUPERVISED CHANGE DETECTION IN OPTICAL SATELLITE IMAGERY USING SIFT FLOW

The process of identifying change in remote sensing images has been a focal point of research for decades now. Many classical algorithms exist, and many new modern ones are still being developed. These algorithms can be divided into supervised and unsupervised. In this work an unsupervised method is presented. This method relies on the scene alignment algorithm SIFT flow. It is shown that building upon simple principles an accurate change map can be obtained from the SIFT descriptor flow of the two input images. Furthermore, it is shown that this method despite its simplicity exceeds other unsupervised methods and comes close to supervised ones, even exceeding them in some metrics. Lastly, the advantages of SIFT flow in comparison to the supervised methods are highlighted alongside its own downsides

Mutual superimposing of SAR and ground-level shooting images mediated by intermediate multi-altitude images

When satellite-based SAR (Synthetic Aperture Radar) images and images acquired from the ground are registered, they offer a wealth of information such as topographic, vegetation or water surface to be extracted from the ground-level shooting images. Simultaneously, high temporal-resolution and high spatial-resolution information obtained by the ground-level shooting images can be superimposed on satellite images. However, due to the differences in imaging modality, spatial resolutions, and observation angle, it was not easy to directly extract the corresponding points between them. This paper proposes an image registration method to estimate the correspondence between SAR images and ground-level shooting images through a set of multi-altitude images taken at different heights

Change Detection Using Synthetic Aperture Radar Videos

Many researches have been carried out for change detection using temporal SAR images. In this paper an algorithm for change detection using SAR videos has been proposed. There are various challenges related to SAR videos such as high level of speckle noise, rotation of SAR image frames of the video around a particular axis due to the circular movement of airborne vehicle, non-uniform back scattering of SAR pulses. Hence conventional change detection algorithms used for optical videos and SAR temporal images cannot be directly utilized for SAR videos. We propose an algorithm which is a combination of optical flow calculation using Lucas Kanade (LK) method and blob detection. The developed method follows a four steps approach: image filtering and enhancement, applying LK method, blob analysis and combining LK method with blob analysis. The performance of the developed approach was tested on SAR videos available on Sandia National Laboratories website and SAR videos generated by a SAR simulator

Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotation

Pretraining CNN models (i.e., UNet) through self-supervision has become a powerful approach to facilitate medical image segmentation under low annotation regimes. Recent contrastive learning methods encourage similar global representations when the same image undergoes different transformations, or enforce invariance across different image/patch features that are intrinsically correlated. However, CNN-extracted global and local features are limited in capturing long-range spatial dependencies that are essential in biological anatomy. To this end, we present a keypoint-augmented fusion layer that extracts representations preserving both short- and long-range self-attention. In particular, we augment the CNN feature map at multiple scales by incorporating an additional input that learns long-range spatial self-attention among localized keypoint features. Further, we introduce both global and local self-supervised pretraining for the framework. At the global scale, we obtain global representations from both the bottleneck of the UNet, and by aggregating multiscale keypoint features. These global features are subsequently regularized through image-level contrastive objectives. At the local scale, we define a distance-based criterion to first establish correspondences among keypoints and encourage similarity between their features. Through extensive experiments on both MRI and CT segmentation tasks, we demonstrate the architectural advantages of our proposed method in comparison to both CNN and Transformer-based UNets, when all architectures are trained with randomly initialized weights. With our proposed pretraining strategy, our method further outperforms existing SSL methods by producing more robust self-attention and achieving state-of-the-art segmentation results. The code is available at https://github.com/zshyang/kaf.git.Comment: Camera ready for NeurIPS 2023. Code available at https://github.com/zshyang/kaf.gi

LAND COVER CHANGES DETECTION IN POLARIMETRIC SAR DATA USING ALGEBRA, SIMILARITY AND DISTANCE BASED METHODS

Monitoring and surveillance changes around the world need powerful methods, so detection, visualization, and assessment of significant changes are essential for planning and management. Incorporating polarimetric SAR images due to interactions between electromagnetic waves and target and because of the high spatial resolution almost one meter can be used to study changes in the Earth's surface. Full polarized radar images comparing to single polarized radar images use amplitude and phase information of the surface in different available polarization (HH, HV, VH, and VV). This study is based on the decomposition of full polarized airborne UAVSAR images and integration of these features with algebra method involves Image Differencing (ID) and Image Ratio (IR) algorithms with the mathematical nature and distance-based method involves Canberra (CA) and Euclidean (ED) algorithms with measuring distance between corresponding vector and similarity-based method involves Taminoto (TA) and Kulczynski (KU) algorithms with dependence corresponding vector for change detecting purposes on two real PolSAR datasets. Assessment of incorporated methods is implemented using ground truth data and different criteria for evaluating such as overall accuracy (OA), area under ROC curve (AUC) and false alarms rate (FAR). The output results show that ID, IR, and CA have superiority to detect changes comparing to other implemented algorithms. Also, numerical results show that the highest performance in two datasets has OA more than 90%. In other assessment criteria, mention algorithms have low FAR and high AUC value indices to detect changes in PolSAR images

HSI-MSER: Hyperspectral Image Registration Algorithm based on MSER and SIFT

Image alignment is an essential task in many applications of hyperspectral remote sensing images. Before any processing, the images must be registered. The Maximally Stable Extremal Regions (MSER) is a feature detection algorithm which extracts regions by thresholding the image at different grey levels. These extremal regions are invariant to image transformations making them ideal for registration. The Scale-Invariant Feature Transform (SIFT) is a well-known keypoint detector and descriptor based on the construction of a Gaussian scale-space. This article presents a hyperspectral remote sensing image registration method based on MSER for feature detection and SIFT for feature description. It efficiently exploits the information contained in the different spectral bands to improve the image alignment. The experimental results over nine hyperspectral images show that the proposed method achieves a higher number of correct registration cases using less computational resources than other hyperspectral registration methods. Results are evaluated in terms of accuracy of the registration and also in terms of execution timeMinisterio de Ciencia e Innovación, Government of Spain PID2019-104834GB-I00; Consellería de Cultura, Educación e Universidade (Grant Number: ED431C 2018/19 and 2019-2022 ED431G-2019/04); Junta de Castilla y León under Project VA226P20; 10.13039/501100008530-European Regional Development Fund; Ministerio de Universidades, Government of Spain (Grant Number: FPU16/03537)S