Image Quality Modeling and Characterization of Nyquist Sampled Framing Systems with Operational Considerations for Remote Sensing

The trade between detector and optics performance is often conveyed through the Q metric, which is defined as the ratio of detector sampling frequency and optical cutoff frequency. Historically sensors have operated at Q~1, which introduces aliasing but increases the system modulation transfer function (MTF) and signal-to-noise ratio (SNR). Though mathematically suboptimal, such designs have been operationally ideal when considering system parameters such as pointing stability and detector performance. Substantial advances in read noise and quantum efficiency of modern detectors may compensate for the negative aspects associated with balancing detector/optics performance, presenting an opportunity to revisit the potential for implementing Nyquist-sampled (Q~2) sensors. A digital image chain simulation is developed and validated against a laboratory testbed using objective and subjective assessments. Objective assessments are accomplished by comparison of the modeled MTF and measurements from slant-edge photographs. Subjective assessments are carried out by performing a psychophysical study where subjects are asked to rate simulation and testbed imagery against a Delta-NIIRS scale with the aid of a marker set. Using the validated model, additional test cases are simulated to study the effects of increased detector sampling on image quality with operational considerations. First, a factorial experiment using Q-sampling, pointing stability, integration time, and detector performance is conducted to measure the main effects and interactions of each on the response variable, Delta-NIIRS. To assess the fidelity of current models, variants of the General Image Quality Equation (GIQE) are evaluated against subject-provided ratings and two modied GIQE versions are proposed. Finally, using the validated simulation and modified IQE, trades are conducted to ascertain the feasibility of implementing Q~2 designs in future systems

Evaluation of changes in image appearance with changes in displayed image size

This research focused on the quantification of changes in image appearance when images are displayed at different image sizes on LCD devices. The final results provided in calibrated Just Noticeable Differences (JNDs) on relevant perceptual scales, allowing the prediction of sharpness and contrast appearance with changes in the displayed image size. A series of psychophysical experiments were conducted to enable appearance predictions. Firstly, a rank order experiment was carried out to identify the image attributes that were most affected by changes in displayed image size. Two digital cameras, exhibiting very different reproduction qualities, were employed to capture the same scenes, for the investigation of the effect of the original image quality on image appearance changes. A wide range of scenes with different scene properties was used as a test-set for the investigation of image appearance changes with scene type. The outcomes indicated that sharpness and contrast were the most important attributes for the majority of scene types and original image qualities. Appearance matching experiments were further conducted to quantify changes in perceived sharpness and contrast with respect to changes in the displayed image size. For the creation of sharpness matching stimuli, a set of frequency domain filters were designed to provide equal intervals in image quality, by taking into account the system’s Spatial Frequency Response (SFR) and the observation distance. For the creation of contrast matching stimuli, a series of spatial domain S-shaped filters were designed to provide equal intervals in image contrast, by gamma adjustments. Five displayed image sizes were investigated. Observers were always asked to match the appearance of the smaller version of each stimulus to its larger reference. Lastly, rating experiments were conducted to validate the derived JNDs in perceptual quality for both sharpness and contrast stimuli. Data obtained by these experiments finally converted into JND scales for each individual image attribute. Linear functions were fitted to the final data, which allowed the prediction of image appearance of images viewed at larger sizes than these investigated in this research

A Log NEQ based comparison of several silver halide and electronic pictorial imaging systems

A protocol for determining log NEQ and a new metric, the Equivalent Image Quality or EIQ, are presented for the generalized experimental evaluation of noise related monochrome pictorial image quality. These metrics are then applied to the evaluation of a pictorial CCD camera and several pictorial films. Emphasis is placed on the development and verification of experimental techniques which do not require elaborate equipment or support facilities. Data analysis is conducted using only available software packages and personal computers. Conclusions are drawn concerning the performance of CCD based and silver halide imaging systems which allow for the objective comparison of the images they produce, and the fundamental differences in the characteristics and requirements of the two systems as applied to pictorial imaging are noted

Design And Assessment Of Compact Optical Systems Towards Special Effects Imaging

A main challenge in the field of special effects is to create special effects in real time in a way that the user can preview the effect before taking the actual picture or movie sequence. There are many techniques currently used to create computer-simulated special effects, however current techniques in computer graphics do not provide the option for the creation of real-time texture synthesis. Thus, while computer graphics is a powerful tool in the field of special effects, it is neither portable nor does it provide work in real-time capabilities. Real-time special effects may, however, be created optically. Such approach will provide not only real-time image processing at the speed of light but also a preview option allowing the user or the artist to preview the effect on various parts of the object in order to optimize the outcome. The work presented in this dissertation was inspired by the idea of optically created special effects, such as painterly effects, encoded in images captured by photographic or motion picture cameras. As part of the presented work, compact relay optics was assessed, developed, and a working prototype was built. It was concluded that even though compact relay optics can be achieved, further push for compactness and cost-effectiveness was impossible in the paradigm of bulk macro-optics systems. Thus, a paradigm for imaging with multi-aperture micro-optics was proposed and demonstrated for the first time, which constitutes one of the key contributions of this work. This new paradigm was further extended to the most general case of magnifying multi-aperture micro-optical systems. Such paradigm allows an extreme reduction in size of the imaging optics by a factor of about 10 and a reduction in weight by a factor of about 500. Furthermore, an experimental quantification of the feasibility of optically created special effects was completed, and consequently raytracing software was developed, which was later commercialized by SmARTLens(TM). While the art forms created via raytracing were powerful, they did not predict all effects acquired experimentally. Thus, finally, as key contribution of this work, the principles of scalar diffraction theory were applied to optical imaging of extended objects under quasi-monochromatic incoherent illumination in order to provide a path to more accurately model the proposed optical imaging process for special effects obtained in the hardware. The existing theoretical framework was generalized to non-paraxial in- and out-of-focus imaging and results were obtained to verify the generalized framework. In the generalized non-paraxial framework, even the most complex linear systems, without any assumptions for shift invariance, can be modeled and analyzed

Characterization of digital film scanner systems for use with digital scene algorithms

Digital film scanners have been used in the photographic industry for more than a decade. The existence of digital image data has made possible the use of computer-based scene enhancement algorithms to improve image quality. These algorithms are usually device-dependent, functioning properly only for data generated by one scanner system. The complexity of most enhancement algorithms make them costly to develop, thus device-independent scene enhancement algorithms would be valuable. The computation of mathematical transformations to convert scanner data to a device-independent space is possible. The data created using these transformations should serve as the input for device-independent enhancement algorithms. A study to determine scanner data space transformations was performed. This study evaluated a subset of Operational Characteristics for three film scanners. These scanner characteristics were used to determine transformations to convert between scanner data spaces. These results were used as part of a system prototype to test the performance of scanner data space transformations

ATM experiment S-056 image processing requirements definition

A plan is presented for satisfying the image data processing needs of the S-056 Apollo Telescope Mount experiment. The report is based on information gathered from related technical publications, consultation with numerous image processing experts, and on the experience that was in working on related image processing tasks over a two-year period

Earth resources data processing center study. Volume 2 - Study findings Final report

Basic objectives and requirements of Earth Resources Progra

New Test Set for Video Quality Benchmarking

A new test set design and benchmarking approach (US Patent pending) allows a standard observer to assess the end-to-end image quality characteristics of video imaging systems operating in day time or low-light conditions. It uses randomized targets based on extensive application of Photometry, Geometrical Optics, and Digital Media. The benchmarking takes into account the target’s contrast sensitivity, its color characteristics, and several aspects of human vision such as visual acuity and dynamic response. The standard observer is part of the extended video imaging system (EVIS). The new test set allows image quality benchmarking by a panel of standard observers at the same time. The new approach shows that an unbiased assessment can be guaranteed. Manufacturers, system integrators, and end users will assess end-to-end performance by simulating a choice of different colors, luminance levels, and dynamic conditions in the laboratory or in permanent video systems installations

Camera Spatial Frequency Response Derived from Pictorial Natural Scenes

Camera system performance is a prominent part of many aspects of imaging science and computer vision. There are many aspects to camera performance that determines how accurately the image represents the scene, including measurements of colour accuracy, tone reproduction, geometric distortions, and image noise evaluation. The research conducted in this thesis focuses on the Modulation Transfer Function (MTF), a widely used camera performance measurement employed to describe resolution and sharpness. Traditionally measured under controlled conditions with characterised test charts, the MTF is a measurement restricted to laboratory settings. The MTF is based on linear system theory, meaning the input to output must follow a straightforward correlation. Established methods for measuring the camera system MTF include the ISO12233:2017 for measuring the edge-based Spatial Frequency Response (e-SFR), a sister measure of the MTF designed for measuring discrete systems. Many modern camera systems incorporate non-linear, highly adaptive image signal processing (ISP) to improve image quality. As a result, system performance becomes scene and processing dependant, adapting to the scene contents captured by the camera. Established test chart based MTF/SFR methods do not describe this adaptive nature; they only provide the response of the camera to a test chart signal. Further, with the increased use of Deep Neural Networks (DNN) for image recognition tasks and autonomous vision systems, there is an increased need for monitoring system performance outside laboratory conditions in real-time, i.e. live-MTF. Such measurements would assist in monitoring the camera systems to ensure they are fully operational for decision critical tasks. This thesis presents research conducted to develop a novel automated methodology that estimates the standard e-SFR directly from pictorial natural scenes. This methodology has the potential to produce scene dependant and real-time camera system performance measurements, opening new possibilities in imaging science and allowing live monitoring/calibration of systems for autonomous computer vision applications. The proposed methodology incorporates many well-established image processes, as well as others developed for specific purposes. It is presented in two parts. Firstly, the Natural Scene derived SFR (NS-SFR) are obtained from isolated captured scene step-edges, after verifying that these edges have the correct profile for implementing into the slanted-edge algorithm. The resulting NS-SFRs are shown to be a function of both camera system performance and scene contents. The second part of the methodology uses a series of derived NS-SFRs to estimate the system e-SFR, as per the ISO12233 standard. This is achieved by applying a sequence of thresholds to segment the most likely data corresponding to the system performance. These thresholds a) group the expected optical performance variation across the imaging circle within radial distance segments, b) obtain the highest performance NS-SFRs per segment and c) select the NS-SFRs with input edge and region of interest (ROI) parameter ranges shown to introduce minimal e-SFR variation. The selected NS-SFRs are averaged per radial segment to estimate system e-SFRs across the field of view. A weighted average of these estimates provides an overall system performance estimation. This methodology is implemented for e-SFR estimation of three characterised camera systems, two near-linear and one highly non-linear. Investigations are conducted using large, diverse image datasets as well as restricting scene content and the number of images used for the estimation. The resulting estimates are comparable to ISO12233 e-SFRs derived from test chart inputs for the near-linear systems. Overall estimate stays within one standard deviation of the equivalent test chart measurement. Results from the highly non-linear system indicate scene and processing dependency, potentially leading to a more representative SFR measure than the current chart-based approaches for such systems. These results suggest that the proposed method is a viable alternative to the ISO technique