Data Driven Multispectral Image Registration Framework

Multispectral imaging is widely used in remote sensing applications from UAVs and ground-based platforms. Multispectral cameras often use a physically different camera for each wavelength causing misalignment in the images for different imaging bands. This misalignment must be corrected prior to concurrent multi-band image analysis. The traditional approach for multispectral image registration process is to select a target channel and register all other image channels to the target. There is no objective evidence-based method to select a target channel. The possibility of registration to some intermediate channel before registering to the target is not usually considered, but could be beneficial if there is no target channel for which direct registration performs well for every other channel. In this paper, we propose an automatic data-driven multispectral image registration framework that determines a target channel, and possible intermediate registration steps based on the assumptions that 1) some reasonable minimum number of control-points correspondences between two channels is needed to ensure a low-error registration; 2) a greater number of such correspondences generally results in higher registration performance. Our prototype is tested on five multispectral datasets captured with UAV-mounted multispectral cameras. The output of the prototype is a registration scheme in the form of a directed acyclic graph (actually a tree) that represents the target channel and the process to register other image channels. The resulting registration schemes had more control point correspondences on average than the traditional register-all-to-one-targetchannel approach. Data-driven registration scheme consistently showed low back-projection error across all the image channel pairs in most of the experiments. Our data-driven framework has generated registration schemes with the best control point extraction algorithm for each image channel pair and registering images in a data-driven approach. The data-driven image registration framework is dataset independent, and it performs on datasets with any number of image channels. With the growing need of remote sensing and the lack of a proper evidence-based method to register multispectral image channels, a data-driven registration framework is an essential tool in the field of image registration and multispectral imaging

UAV-Multispectral Sensed Data Band Co-Registration Framework

Precision farming has greatly benefited from new technologies over the years. The use of multispectral and hyperspectral sensors coupled to Unmanned Aerial Vehicles (UAV) has enabled farms to monitor crops, improve the use of resources and reduce costs. Despite being widely used, multispectral images present a natural misalignment among the various spectra due to the use of different sensors. The variation of the analyzed spectrum also leads to a loss of characteristics among the bands which hinders the feature detection process among the bands, which makes the alignment process complex. In this work, we propose a new framework for the band co-registration process based on two premises: i) the natural misalignment is an attribute of the camera, so it does not change during the acquisition process; ii) the speed of displacement of the UAV when compared to the speed between the acquisition of the first to the last band, is not sufficient to create significant distortions. We compared our results with the ground-truth generated by a specialist and with other methods present in the literature. The proposed framework had an average back-projection (BP) error of 0.425 pixels, this result being 335% better than the evaluated frameworks.CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível SuperiorDissertação (Mestrado)A agricultura de precisão se beneficiou muito das novas tecnologias ao longo dos anos. O uso de sensores multiespectrais e hiperespectrais acoplados aos Veículos Aéreos Não Tripulados (VANT) permitiu que as fazendas monitorassem as lavouras, melhorassem o uso de recursos e reduzissem os custos. Apesar de amplamente utilizadas, as imagens multiespectrais apresentam um desalinhamento natural entre os vários espectros devido ao uso de diferentes sensores. A variação do espectro analisado também leva à perda de características entre as bandas, o que dificulta o processo de detecção de atributos entre as bandas, o que torna complexo o processo de alinhamento. Neste trabalho, propomos um novo framework para o processo de alinhamento entre as bandas com base em duas premissas: i) o desalinhamento natural é um atributo da câmera, e por esse motivo ele não é alterado durante o processo de aquisição; ii) a velocidade de deslocamento do VANT, quando comparada à velocidade entre a aquisição da primeira e a última banda, não é suficiente para criar distorções significativas. Os resultados obtidos foram comparados com o padrão ouro gerado por um especialista e com outros métodos presentes na literatura. O framework proposto teve um back-projection error (BP) de 0, 425 pixels, sendo este resultado 335% melhor aos frameworks avaliados

An Evaluation of Popular Copy-Move Forgery Detection Approaches

A copy-move forgery is created by copying and pasting content within the same image, and potentially post-processing it. In recent years, the detection of copy-move forgeries has become one of the most actively researched topics in blind image forensics. A considerable number of different algorithms have been proposed focusing on different types of postprocessed copies. In this paper, we aim to answer which copy-move forgery detection algorithms and processing steps (e.g., matching, filtering, outlier detection, affine transformation estimation) perform best in various postprocessing scenarios. The focus of our analysis is to evaluate the performance of previously proposed feature sets. We achieve this by casting existing algorithms in a common pipeline. In this paper, we examined the 15 most prominent feature sets. We analyzed the detection performance on a per-image basis and on a per-pixel basis. We created a challenging real-world copy-move dataset, and a software framework for systematic image manipulation. Experiments show, that the keypoint-based features SIFT and SURF, as well as the block-based DCT, DWT, KPCA, PCA and Zernike features perform very well. These feature sets exhibit the best robustness against various noise sources and downsampling, while reliably identifying the copied regions.Comment: Main paper: 14 pages, supplemental material: 12 pages, main paper appeared in IEEE Transaction on Information Forensics and Securit

WxBS: Wide Baseline Stereo Generalizations

We have presented a new problem -- the wide multiple baseline stereo (WxBS) -- which considers matching of images that simultaneously differ in more than one image acquisition factor such as viewpoint, illumination, sensor type or where object appearance changes significantly, e.g. over time. A new dataset with the ground truth for evaluation of matching algorithms has been introduced and will be made public. We have extensively tested a large set of popular and recent detectors and descriptors and show than the combination of RootSIFT and HalfRootSIFT as descriptors with MSER and Hessian-Affine detectors works best for many different nuisance factors. We show that simple adaptive thresholding improves Hessian-Affine, DoG, MSER (and possibly other) detectors and allows to use them on infrared and low contrast images. A novel matching algorithm for addressing the WxBS problem has been introduced. We have shown experimentally that the WxBS-M matcher dominantes the state-of-the-art methods both on both the new and existing datasets.Comment: Descriptor and detector evaluation expande

HSI-MSER: Hyperspectral Image Registration Algorithm based on MSER and SIFT

Image alignment is an essential task in many applications of hyperspectral remote sensing images. Before any processing, the images must be registered. The Maximally Stable Extremal Regions (MSER) is a feature detection algorithm which extracts regions by thresholding the image at different grey levels. These extremal regions are invariant to image transformations making them ideal for registration. The Scale-Invariant Feature Transform (SIFT) is a well-known keypoint detector and descriptor based on the construction of a Gaussian scale-space. This article presents a hyperspectral remote sensing image registration method based on MSER for feature detection and SIFT for feature description. It efficiently exploits the information contained in the different spectral bands to improve the image alignment. The experimental results over nine hyperspectral images show that the proposed method achieves a higher number of correct registration cases using less computational resources than other hyperspectral registration methods. Results are evaluated in terms of accuracy of the registration and also in terms of execution timeMinisterio de Ciencia e Innovación, Government of Spain PID2019-104834GB-I00; Consellería de Cultura, Educación e Universidade (Grant Number: ED431C 2018/19 and 2019-2022 ED431G-2019/04); Junta de Castilla y León under Project VA226P20; 10.13039/501100008530-European Regional Development Fund; Ministerio de Universidades, Government of Spain (Grant Number: FPU16/03537)S