Joint-SRVDNet: Joint Super Resolution and Vehicle Detection Network

In many domestic and military applications, aerial vehicle detection and super-resolutionalgorithms are frequently developed and applied independently. However, aerial vehicle detection on super-resolved images remains a challenging task due to the lack of discriminative information in the super-resolved images. To address this problem, we propose a Joint Super-Resolution and Vehicle DetectionNetwork (Joint-SRVDNet) that tries to generate discriminative, high-resolution images of vehicles fromlow-resolution aerial images. First, aerial images are up-scaled by a factor of 4x using a Multi-scaleGenerative Adversarial Network (MsGAN), which has multiple intermediate outputs with increasingresolutions. Second, a detector is trained on super-resolved images that are upscaled by factor 4x usingMsGAN architecture and finally, the detection loss is minimized jointly with the super-resolution loss toencourage the target detector to be sensitive to the subsequent super-resolution training. The network jointlylearns hierarchical and discriminative features of targets and produces optimal super-resolution results. Weperform both quantitative and qualitative evaluation of our proposed network on VEDAI, xView and DOTAdatasets. The experimental results show that our proposed framework achieves better visual quality than thestate-of-the-art methods for aerial super-resolution with 4x up-scaling factor and improves the accuracy ofaerial vehicle detection

Image Super-Resolution Using Adaptive 2-D Gaussian Basis Function Interpolation

Digital image interpolation using Gaussian radial basis functions has been implemented by several investigators, and promising results have been obtained; however, determining the basis function variance has been problematic. Here, adaptive Gaussian basis functions fit the mean vector and covariance matrix of a non-radial Gaussian function to each pixel and its neighbors, which enables edges and other image characteristics to be more effectively represented. The interpolation is constrained to reproduce the original image mean gray level, and the mean basis function variance is determined using the expected image smoothness for the increased resolution. Test outputs from the resulting Adaptive Gaussian Interpolation algorithm are presented and compared with classical interpolation techniques

Development Of A High Performance Mosaicing And Super-Resolution Algorithm

In this dissertation, a high-performance mosaicing and super-resolution algorithm is described. The scale invariant feature transform (SIFT)-based mosaicing algorithm builds an initial mosaic which is iteratively updated by the robust super resolution algorithm to achieve the final high-resolution mosaic. Two different types of datasets are used for testing: high altitude balloon data and unmanned aerial vehicle data. To evaluate our algorithm, five performance metrics are employed: mean square error, peak signal to noise ratio, singular value decomposition, slope of reciprocal singular value curve, and cumulative probability of blur detection. Extensive testing shows that the proposed algorithm is effective in improving the captured aerial data and the performance metrics are accurate in quantifying the evaluation of the algorithm

Exploring the Internal Statistics: Single Image Super-Resolution, Completion and Captioning

Image enhancement has drawn increasingly attention in improving image quality or interpretability. It aims to modify images to achieve a better perception for human visual system or a more suitable representation for further analysis in a variety of applications such as medical imaging, remote sensing, and video surveillance. Based on different attributes of the given input images, enhancement tasks vary, e.g., noise removal, deblurring, resolution enhancement, prediction of missing pixels, etc. The latter two are usually referred to as image super-resolution and image inpainting (or completion). Image super-resolution and completion are numerically ill-posed problems. Multi-frame-based approaches make use of the presence of aliasing in multiple frames of the same scene. For cases where only one input image is available, it is extremely challenging to estimate the unknown pixel values. In this dissertation, we target at single image super-resolution and completion by exploring the internal statistics within the input image and across scales. An internal gradient similarity-based single image super-resolution algorithm is first presented. Then we demonstrate that the proposed framework could be naturally extended to accomplish super-resolution and completion simultaneously. Afterwards, a hybrid learning-based single image super-resolution approach is proposed to benefit from both external and internal statistics. This framework hinges on image-level hallucination from externally learned regression models as well as gradient level pyramid self-awareness for edges and textures refinement. The framework is then employed to break the resolution limitation of the passive microwave imagery and to boost the tracking accuracy of the sea ice movements. To extend our research to the quality enhancement of the depth maps, a novel system is presented to handle circumstances where only one pair of registered low-resolution intensity and depth images are available. High quality RGB and depth images are generated after the system. Extensive experimental results have demonstrated the effectiveness of all the proposed frameworks both quantitatively and qualitatively. Different from image super-resolution and completion which belong to low-level vision research, image captioning is a high-level vision task related to the semantic understanding of an input image. It is a natural task for human beings. However, image captioning remains challenging from a computer vision point of view especially due to the fact that the task itself is ambiguous. In principle, descriptions of an image can talk about any visual aspects in it varying from object attributes to scene features, or even refer to objects that are not depicted and the hidden interaction or connection that requires common sense knowledge to analyze. Therefore, learning-based image captioning is in general a data-driven task, which relies on the training dataset. Descriptions in the majority of the existing image-sentence datasets are generated by humans under specific instructions. Real-world sentence data is rarely directly utilized for training since it is sometimes noisy and unbalanced, which makes it ‘imperfect’ for the training of the image captioning task. In this dissertation, we present a novel image captioning framework to deal with the uncontrolled image-sentence dataset where descriptions could be strongly or weakly correlated to the image content and in arbitrary lengths. A self-guiding learning process is proposed to fully reveal the internal statistics of the training dataset and to look into the learning process in a global way and generate descriptions that are syntactically correct and semantically sound

Hyperspectral Clustering and Unmixing of Satellite Imagery for the Study of Complex Society State Formation

This project is an application of remote sensing techniques to the field of archaeology. Clustering and unmixing algorithms are applied to hyperspectral Hyperion imagery over Oaxaca, Mexico. Oaxaca is the birthplace of the Zapotec civilization, the earliest state-level society in Mesoamerica. A passionate debate is ongoing over whether the Zapotecs\u27 evolution was environmentally deterministic or socioeconomic. Previous archaeological remote sensing has focused on the difficult tasks of feature detection using low spatial resolution imagery or visual inspection of spectral data. This project attempts to learn about a civilization on the macro level, using unsupervised land classification techniques. Overlapping 158 band Hyperion data are tasked for approximately 30,000 km2, to be taken over several years. K-means and ISODATA are implemented for clustering. MaxD is used to find endmembers for stepwise spectral unmixing. Case studies are performed that provide insights into the best use of various algorithms. To produce results with spatial context, a method is devised to tile long hyperspectral flight lines, process them, then merge the tiles back into a single coherent image. Google Earth is utilized to effectively share the produced classification and abundance maps. All the processes are automated to efficiently handle the large amount of data. In summary, this project focuses on spectral over spatial exploitation for a land survey study, using open source tools to facilitate results. Classification and abundance maps are generated highlighting basic material spatial patterns (e.g., soil, vegetation and water). Additional remote sensing techniques that are potentially useful to archaeologists are briefly described for use in future work