Super-Resolution of Unmanned Airborne Vehicle Images with Maximum Fidelity Stochastic Restoration

Super-resolution (SR) refers to reconstructing a single high resolution (HR) image from a set of subsampled, blurred and noisy low resolution (LR) images. One may, then, envision a scenario where a set of LR images is acquired with sensors on a moving platform like unmanned airborne vehicles (UAV). Due to the wind, the UAV may encounter altitude change or rotational effects which can distort the acquired as well as the processed images. Also, the visual quality of the SR image is affected by image acquisition degradations, the available number of the LR images and their relative positions. This dissertation seeks to develop a novel fast stochastic algorithm to reconstruct a single SR image from UAV-captured images in two steps. First, the UAV LR images are aligned using a new hybrid registration algorithm within subpixel accuracy. In the second step, the proposed approach develops a new fast stochastic minimum square constrained Wiener restoration filter for SR reconstruction and restoration using a fully detailed continuous-discrete-continuous (CDC) model. A new parameter that accounts for LR images registration and fusion errors is added to the SR CDC model in addition to a multi-response restoration and reconstruction. Finally, to assess the visual quality of the resultant images, two figures of merit are introduced: information rate and maximum realizable fidelity. Experimental results show that quantitative assessment using the proposed figures coincided with the visual qualitative assessment. We evaluated our filter against other SR techniques and its results were found to be competitive in terms of speed and visual quality

Mathematical Model Development of Super-Resolution Image Wiener Restoration

In super-resolution (SR), a set of degraded low-resolution (LR) images are used to reconstruct a higher-resolution image that suffers from acquisition degradations. One way to boost SR images visual quality is to use restoration filters to remove reconstructed images artifacts. We propose an efficient method to optimally allocate the LR pixels on the high-resolution grid and introduce a mathematical derivation of a stochastic Wiener filter. It relies on the continuous-discrete-continuous model and is constrained by the periodic and nonperiodic interrelationships between the different frequency components of the proposed SR system. We analyze an end-to-end model and formulate the Wiener filter as a function of the parameters associated with the proposed SR system such as image gathering and display response indices, system average signal-to-noise ratio, and inter-subpixel shifts between the LR images. Simulation and experimental results demonstrate that the derived Wiener filter with the optimal allocation of LR images results in sharper reconstruction. When compared with other SR techniques, our approach outperforms them in both quality and computational time

Wavelet-based image and video super-resolution reconstruction.

Super-resolution reconstruction process offers the solution to overcome the high-cost and inherent resolution limitations of current imaging systems. The wavelet transform is a powerful tool for super-resolution reconstruction. This research provides a detailed study of the wavelet-based super-resolution reconstruction process, and wavelet-based resolution enhancement process (with which it is closely associated). It was addressed to handle an explicit need for a robust wavelet-based method that guarantees efficient utilisation of the SR reconstruction problem in the wavelet-domain, which will lead to a consistent solution of this problem and improved performance. This research proposes a novel performance assessment approach to improve the performance of the existing wavelet-based image resolution enhancement techniques. The novel approach is based on identifying the factors that effectively influence on the performance of these techniques, and designing a novel optimal factor analysis (OFA) algorithm. A new wavelet-based image resolution enhancement method, based on discrete wavelet transform and new-edge directed interpolation (DWT-NEDI), and an adaptive thresholding process, has been developed. The DWT-NEDI algorithm aims to correct the geometric errors and remove the noise for degraded satellite images. A robust wavelet-based video super-resolution technique, based on global motion is developed by combining the DWT-NEDI method, with super-resolution reconstruction methods, in order to increase the spatial-resolution and remove the noise and aliasing artefacts. A new video super-resolution framework is designed using an adaptive local motion decomposition and wavelet transform reconstruction (ALMD-WTR). This is to address the challenge of the super-resolution problem for the real-world video sequences containing complex local motions. The results show that OFA approach improves the performance of the selected wavelet-based methods. The DWT-NEDI algorithm outperforms the state-of-the art wavelet-based algorithms. The global motion-based algorithm has the best performance over the super-resolution techniques, namely Keren and structure-adaptive normalised convolution methods. ALMD-WTR framework surpass the state-of-the-art wavelet-based algorithm, namely local motion-based video super-resolution.PhD in Manufacturin

Real Time Fpga Implementation Of A Training Based Content Adaptive Video Resolution Upconversion Algorithm

Tez (Yüksek Lisans) -- İstanbul Teknik Üniversitesi, Bilişim Enstitüsü, 2007Thesis (M.Sc.) -- İstanbul Technical University, Institute of Informatics, 2007Bu çalışmada, eğitim tabanlı, içerik uyarlamalı bir video çözünürlük yükseltme algoritması için, iş hattı ve kaynak paylaşımı kullanan yüksek performanslı bir donanım mimarisi önerilmiş ve önerilen yapı, 480x720 standart çözünürlükteki videonun 720x1280 yüksek çözünürlükte videoya dönüştürülmesi uygulaması için düşük maliyetli bir sahada programlanabilir kapı dizisi (SPKD (FPGA)) kullanarak gerçeklenmiştir. Donanım yapısı önerilen ve gerçeklenen, modifiye edilmiş çözünürlük sentezi algoritması (MÇS (MRS)), alt örnekleme işlemi sürecinde video sinyalinde kaybolan yüksek frekans bileşenlerinin, geniş bir video görüntü kümesi üzerinde gerçekleştirilen eğitim sürecinde elde edilen bilgi ile geri kazanılmasını hedefler. MÇS algoritması çıkış görüntüsünü oluşturan her piksel için 137 çarpma ve 120 toplama işlemi içerir. 480x720 standart çözünürlükte videonun 720x1280 yüksek çözünürlükte videoya dönüştürülmesi problemi, 27 Mhz giriş saat çevriminde üretilen piksel datası ile gerçek zaman kısıtları içerir. Hedeflenen FPGA için, tasarım, giriş piksel saat frekansının dört katı olan 108 Mhz saat frekansında çalışacak biçimde iş hattı yapısı kurulmuştur. Bu sayede çarpma ve toplama işlemleri için kaynak paylaşımı yapılmış ve, iş hattındaki saklayıcılarda ve kontrol lojiğinde küçük bir artış ile çarpıcı ve toplayıcı sayısı dörtte birine indirilmiştir. Önerilen yapının, saklayıcı transfer seviyesindeki tanımı, VHDL dili ile yazılmış, sabit noktalı C modeli ile VHDL modeli çıktıları karşılaştırılarak donanım yapısı doğrulanmıştır. Doğrulanan tasarım, Xilinx XC3S2000 FPGA kullanılarak gerçeklenmiş ve standart çözünürlükteki videonun yüksek çözünürlükte videoya dönüştürülmesi uygulaması için likit kristal ekranlı TV üzerinde test edilmiştir. Tasarım, FPGA içerisinde 3533 dilim ve yaklaşık 60 KB blok RAM yapısı kullanmaktadır. Tasarımın lojik kapı cinsinden karmaşıklığının, literatürdeki lineer video boyutlandırma algoritmaları ile yaklaşık aynı ölçekte olduğu görülmüştür.In this study, a high performance, pipelined, resource shared hardware architecture was proposed for a training based content adaptive video resolution up-conversion algorithm, and the proposed architecture was implemented in a field programmable gate array (FPGA), for a video standards conversion application where the input is standard definition (SD) video with 480x720 resolution, and the output is high definition (HD) video with 720x1280 resolution. Modified resolution synthesis (MRS), which was implemented in this study is a method, that aims to recover the missing spectrum at the down sampled image, by using information obtained by training with large set of images. MRS requires 137 multiplications and 120 additions per output pixel. For 480x720 to 720x1280 video conversion, the design is constrained by the input pixel rate which is 27 Mhz. For the targeted FPGA, the design was pipelined to work at 108 Mhz, four times the input pixel clock rate. Number of multipliers and adders were reduced by a factor of 4, with minor increase in the pipeline stages and the control logic complexity. Register transfer level (RTL) description of the proposed architecture was written in VHDL and RTL model was verified with fixed point C model outputs. The verified design was mapped to Xilinx XC3S2000 FPGA, and was tested on TV for SD to HD video conversion. The design uses 3533 slices, and 60KByte of block RAMS available in the FPGA. The logic gate count of the design is in the order of gate counts for bicubic scalers proposed previously.Yüksek LisansM.Sc

Motion Segmentation Aided Super Resolution Image Reconstruction

This dissertation addresses Super Resolution (SR) Image Reconstruction focusing on motion segmentation. The main thrust is Information Complexity guided Gaussian Mixture Models (GMMs) for Statistical Background Modeling. In the process of developing our framework we also focus on two other topics; motion trajectories estimation toward global and local scene change detections and image reconstruction to have high resolution (HR) representations of the moving regions. Such a framework is used for dynamic scene understanding and recognition of individuals and threats with the help of the image sequences recorded with either stationary or non-stationary camera systems. We introduce a new technique called Information Complexity guided Statistical Background Modeling. Thus, we successfully employ GMMs, which are optimal with respect to information complexity criteria. Moving objects are segmented out through background subtraction which utilizes the computed background model. This technique produces superior results to competing background modeling strategies. The state-of-the-art SR Image Reconstruction studies combine the information from a set of unremarkably different low resolution (LR) images of static scene to construct an HR representation. The crucial challenge not handled in these studies is accumulating the corresponding information from highly displaced moving objects. In this aspect, a framework of SR Image Reconstruction of the moving objects with such high level of displacements is developed. Our assumption is that LR images are different from each other due to local motion of the objects and the global motion of the scene imposed by non-stationary imaging system. Contrary to traditional SR approaches, we employed several steps. These steps are; the suppression of the global motion, motion segmentation accompanied by background subtraction to extract moving objects, suppression of the local motion of the segmented out regions, and super-resolving accumulated information coming from moving objects rather than the whole scene. This results in a reliable offline SR Image Reconstruction tool which handles several types of dynamic scene changes, compensates the impacts of camera systems, and provides data redundancy through removing the background. The framework proved to be superior to the state-of-the-art algorithms which put no significant effort toward dynamic scene representation of non-stationary camera systems

Super Resolution of Wavelet-Encoded Images and Videos

In this dissertation, we address the multiframe super resolution reconstruction problem for wavelet-encoded images and videos. The goal of multiframe super resolution is to obtain one or more high resolution images by fusing a sequence of degraded or aliased low resolution images of the same scene. Since the low resolution images may be unaligned, a registration step is required before super resolution reconstruction. Therefore, we first explore in-band (i.e. in the wavelet-domain) image registration; then, investigate super resolution. Our motivation for analyzing the image registration and super resolution problems in the wavelet domain is the growing trend in wavelet-encoded imaging, and wavelet-encoding for image/video compression. Due to drawbacks of widely used discrete cosine transform in image and video compression, a considerable amount of literature is devoted to wavelet-based methods. However, since wavelets are shift-variant, existing methods cannot utilize wavelet subbands efficiently. In order to overcome this drawback, we establish and explore the direct relationship between the subbands under a translational shift, for image registration and super resolution. We then employ our devised in-band methodology, in a motion compensated video compression framework, to demonstrate the effective usage of wavelet subbands. Super resolution can also be used as a post-processing step in video compression in order to decrease the size of the video files to be compressed, with downsampling added as a pre-processing step. Therefore, we present a video compression scheme that utilizes super resolution to reconstruct the high frequency information lost during downsampling. In addition, super resolution is a crucial post-processing step for satellite imagery, due to the fact that it is hard to update imaging devices after a satellite is launched. Thus, we also demonstrate the usage of our devised methods in enhancing resolution of pansharpened multispectral images