Image gathering, coding, and processing: End-to-end optimization for efficient and robust acquisition of visual information

Researchers are concerned with the end-to-end performance of image gathering, coding, and processing. The applications range from high-resolution television to vision-based robotics, wherever the resolution, efficiency and robustness of visual information acquisition and processing are critical. For the presentation at this workshop, it is convenient to divide research activities into the following two overlapping areas: The first is the development of focal-plane processing techniques and technology to effectively combine image gathering with coding, with an emphasis on low-level vision processing akin to the retinal processing in human vision. The approach includes the familiar Laplacian pyramid, the new intensity-dependent spatial summation, and parallel sensing/processing networks. Three-dimensional image gathering is attained by combining laser ranging with sensor-array imaging. The second is the rigorous extension of information theory and optimal filtering to visual information acquisition and processing. The goal is to provide a comprehensive methodology for quantitatively assessing the end-to-end performance of image gathering, coding, and processing

Electro-Optical Design for Efficient Visual Communication

Visual communication, in the form of telephotography and television, for example, can be regarded as efficient only if the amount of information that it conveys about the scene to the observer approaches the maximum possible and the associated cost approaches the minimum possible. Elsewhere we have addressed the problem of assessing the end to end performance of visual communication systems in terms of their efficiency in this sense by integrating the critical limiting factors that constrain image gathering into classical communications theory. We use this approach to assess the electro-optical design of image gathering devices as a function of the f number and apodization of the objective lens and the aperture size and sampling geometry of the phot-detection mechanism. Results show that an image gathering device that is designed to optimize information capacity performs similarly to the human eye. For both, the performance approaches the maximum possible, in terms of the efficiency with which the acquired information can be transmitted as decorrelated data, and the fidelity, sharpness, and clearity with which fine detail can be restored

On The Information-Theoretic Assessment Of Visual Communication

This paper deals with the extension of information theory to the assessment of visual communication from scene to observer. The mathematical development rigorously unites the electro-optical design of image gathering and display devices with the digital processing algorithms for image coding and restoration. Results show that: ffl End-to-end system analysis closely correlates with measurable and perceptual performance characteristics, such as data rate and image quality, respectively. ffl The goal of producing the best possible image at the lowest data rate can be realized only if (a) the electro-optical design of the image-gathering device is optimized for the maximum-realizable information rate and (b) the image-restoration algorithm properly accounts for the perturbations in the visual communication channel. 1. INTRODUCTION Modern visual communication channels increasingly combine image gathering and display with digital image coding and restoration (Fig. 1). So far, however, the..

Informationally Optimized Image Gathering and Restoration

The goal of image gathering and restoration often is to produce the best possible picture in terms of fidelity, sharpness and clarity. However, this goal cannot be attained, at it has been pursued in the past, by treating image gathering and restoration as independent tasks. Instead, in a clean departure from the mores of traditional image processing, we present an approach that rigorously uses modern communication theory to optimally combine the electro-optical design of the image gathering device with the digital processing algorithm for image restoration. Extensive simulations have shown that there exists a strong correlation between the information rate that is produced by the image gathering device and the image quality with which an image can be restored. Introduction Modern visual communication channels increasingly combine image gathering and display with digital image coding and restoration (Fig. 1). So far, however, the imagegathering devices are still designed to produce t..