De-velopment of Demosaicking Techniques for Multi-Spectral Imaging Using Mosaic Focal Plane Arrays

The use of mosaicked array technology in commercial digital cameras has madethem smaller, cheaper and mechanically more robust. In a mosaicked sensor, each pixel detector is covered with a wavelength-specific optical filter. Since only one spectral band is sensed per pixel location, there is an absence of information from the rest of the spectral bands. These unmeasured spectral bands are estimated by using information obtained from the neighborhood pixels. This process of estimating the unmeasured spectral band information is called demosaicking. The demosaicking process uses interpolation strategies to estimate the missing pixels. Sophisticated interpolation methods have been developed for performing this task in digital color cameras.In this thesis we propose to evaluate the adaptation of the mosaicked technol- ogy for multi-spectral cameras. Existing multi-spectral cameras use traditional methods like imaging spectrometers to capture a multi-spectral image. These methods are very expensive and delicate in nature. The objective of using the mosaicked technology for multi-spectral cameras is to reap the same benefits it offers in the commercial digital color cameras. However, the problem in using the mosaicked technology for multi-spectral images is the huge amount of missing pixels that need to be estimated in order to form the multi-spectral image. The estimation process becomes even more complicated as the number of bands in the multi-spectral image increases. Traditional demosaicking algorithms cannot be used because they have been specifically designed to suit three-band color images.This thesis focuses on developing new demosaicking algorithms for multi- spectral images. The existing demosaicking algorithms for color images have been extended for multi-spectral images. A new variation of the bilinear interpolationbased strategy has been developed to perform demosaicking. This demosaicking method uses variable neighborhood definitions to interpolate the missing spectral band values at each pixel locations in a multi-spectral image. A novel Maximum a-Posteriori (MAP) based demosaicking method has also been developed. This method treats demosaicking as an image restoration problem. It can derive op- timal estimation result that resembles the original image the best. In addition, it can simultaneously perform interpolation of missing spectral bands at pixel locations and also remove noise and degradations in the image.Extensive experimentation and comparisons have shown that the new demo- saicking methods for multi-spectral images developed in this thesis perform better than the traditional interpolation trategies. The outputs from the demosaicking methods have been shown to be better reconstructed estimates of the original im- ages and also have the ability to produce good classification results in applicationslike target recognition and discrimination

A Demonstration of Wavefront Sensing and Mirror Phasing from the Image Domain

In astronomy and microscopy, distortions in the wavefront affect the dynamic range of a high contrast imaging system. These aberrations are either imposed by a turbulent medium such as the atmosphere, by static or thermal aberrations in the optical path, or by imperfectly phased subapertures in a segmented mirror. Active and adaptive optics (AO), consisting of a wavefront sensor and a deformable mirror, are employed to address this problem. Nevertheless, the non-common-path between the wavefront sensor and the science camera leads to persistent quasi-static speckles that are difficult to calibrate and which impose a floor on the image contrast. In this paper we present the first experimental demonstration of a novel wavefront sensor requiring only a minor asymmetric obscuration of the pupil, using the science camera itself to detect high order wavefront errors from the speckle pattern produced. We apply this to correct errors imposed on a deformable microelectromechanical (MEMS) segmented mirror in a closed loop, restoring a high quality point spread function (PSF) and residual wavefront errors of order $\sim 10$ nm using 1600 nm light, from a starting point of $\sim 300$ nm in piston and $\sim 0.3$ mrad in tip-tilt. We recommend this as a method for measuring the non-common-path error in AO-equipped ground based telescopes, as well as as an approach to phasing difficult segmented mirrors such as on the \emph{James Webb Space Telescope} primary and as a future direction for extreme adaptive optics.Comment: 9 pages, 6 figure

Speckle interferometry

We have presented the basic mathematical treatment of interferometry in the optical domain. Its applications in astronomical observations using both the single aperture, as well as the diluted apertures are described in detail. We have also described about the shortcomings of this technique in the presence of Earth's atmosphere. A short descriptions of the atmospheric turbulence and its effect on the flat wavefront from a stellar source is given. The formation of speckle which acts as carrier of information is defined. Laboratory experiments with phase modulation screens, as well as the resultant intensity distributions due to point source are demonstrated. The experimental method to freeze the speckles, as well as data processing techniques for both Fourier modulus and Fourier phase are described. We have also discussed the technique of the aperture synthesis using non-redundant aperture masks at the pupil plane of the telescope, emphasizing set on the comparison with speckle interferometry. The various methods of image restoration and their comparisons are also discussed. Finally, we have touched upon certain astrophysical problems which can be tackled with the newly developed speckle interferometer using the 2.34 meter Vainu Bappu Telescope (VBT), situated at the Vainu Bappu Observatory (VBO), Kavalur, India.Comment: 32 pages tex files including figure

Astronomical use of television-type image sensors

Conference on using TV type image sensors in astronomical photometr

Hyper-Restormer: A General Hyperspectral Image Restoration Transformer for Remote Sensing Imaging

The deep learning model Transformer has achieved remarkable success in the hyperspectral image (HSI) restoration tasks by leveraging Spectral and Spatial Self-Attention (SA) mechanisms. However, applying these designs to remote sensing (RS) HSI restoration tasks, which involve far more spectrums than typical HSI (e.g., ICVL dataset with 31 bands), presents challenges due to the enormous computational complexity of using Spectral and Spatial SA mechanisms. To address this problem, we proposed Hyper-Restormer, a lightweight and effective Transformer-based architecture for RS HSI restoration. First, we introduce a novel Lightweight Spectral-Spatial (LSS) Transformer Block that utilizes both Spectral and Spatial SA to capture long-range dependencies of input features map. Additionally, we employ a novel Lightweight Locally-enhanced Feed-Forward Network (LLFF) to further enhance local context information. Then, LSS Transformer Blocks construct a Single-stage Lightweight Spectral-Spatial Transformer (SLSST) that cleverly utilizes the low-rank property of RS HSI to decompose the feature maps into basis and abundance components, enabling Spectral and Spatial SA with low computational cost. Finally, the proposed Hyper-Restormer cascades several SLSSTs in a stepwise manner to progressively enhance the quality of RS HSI restoration from coarse to fine. Extensive experiments were conducted on various RS HSI restoration tasks, including denoising, inpainting, and super-resolution, demonstrating that the proposed Hyper-Restormer outperforms other state-of-the-art methods