A simplified HDR image processing pipeline for digital photography

High Dynamic Range (HDR) imaging has revolutionized the digital imaging. It allows capture, storage, manipulation, and display of full dynamic range of the captured scene. As a result, it has spawned whole new possibilities for digital photography, from photorealistic to hyper-real. With all these advantages, the technique is expected to replace the conventional 8-bit Low Dynamic Range (LDR) imaging in the future. However, HDR results in an even more complex imaging pipeline including new techniques for capturing, encoding, and displaying images. The goal of this thesis is to bridge the gap between conventional imaging pipeline to the HDR’s in as simple a way as possible. We make three contributions. First we show that a simple extension of gamma encoding suffices as a representation to store HDR images. Second, gamma as a control for image contrast can be ‘optimally’ tuned on a per image basis. Lastly, we show a general tone curve, with detail preservation, suffices to tone map an image (there is only a limited need for the expensive spatially varying tone mappers). All three of our contributions are evaluated psychophysically. Together they support our general thesis that an HDR workflow, similar to that already used in photography, might be used. This said, we believe the adoption of HDR into photography is, perhaps, less difficult than it is sometimes posed to be

Optimization of video capturing and tone mapping in video camera systems

Image enhancement techniques are widely employed in many areas of professional and consumer imaging, machine vision and computational imaging. Image enhancement techniques used in surveillance video cameras are complex systems involving controllable lenses, sensors and advanced signal processing. In surveillance, a high output image quality with very robust and stable operation under difficult imaging conditions are essential, combined with automatic, intelligent camera behavior without user intervention. The key problem discussed in this thesis is to ensure this high quality under all conditions, which specifically addresses the discrepancy of the dynamic range of input scenes and displays. For example, typical challenges are High Dynamic Range (HDR) and low-dynamic range scenes with strong light-dark differences and overall poor visibility of details, respectively. The detailed problem statement is as follows: (1) performing correct and stable image acquisition for video cameras in variable dynamic range environments, and (2) finding the best image processing algorithms to maximize the visualization of all image details without introducing image distortions. Additionally, the solutions should satisfy complexity and cost requirements of typical video surveillance cameras. For image acquisition, we develop optimal image exposure algorithms that use a controlled lens, sensor integration time and camera gain, to maximize SNR. For faster and more stable control of the camera exposure system, we remove nonlinear tone-mapping steps from the level control loop and we derive a parallel control strategy that prevents control delays and compensates for the non-linearity and unknown transfer characteristics of the used lenses. For HDR imaging we adopt exposure bracketing that merges short and long exposed images. To solve the involved non-linear sensor distortions, we apply a non-linear correction function to the distorted sensor signal, implementing a second-order polynomial with coefficients adaptively estimated from the signal itself. The result is a good, dynamically controlled match between the long- and short-exposed image. The robustness of this technique is improved for fluorescent light conditions, preventing serious distortions by luminance flickering and color errors. To prevent image degradation we propose both fluorescent light detection and fluorescence locking, based on measurements of the sensor signal intensity and color errors in the short-exposed image. The use of various filtering steps increases the detector robustness and reliability for scenes with motion and the appearance of other light sources. In the alternative algorithm principle of fluorescence locking, we ensure that light integrated during the short exposure time has a correct intensity and color by synchronizing the exposure measurement to the mains frequency. The second area of research is to maximize visualization of all image details. This is achieved by both global and local tone mapping functions. The largest problem of Global Tone Mapping Functions (GTMF) is that they often significantly deteriorate the image contrast. We have developed a new GTMF and illustrate, both analytically and perceptually, that it exhibits only a limited amount of compression, compared to conventional solutions. Our algorithm splits GTMF into two tasks: (1) compressing HDR images (DRC transfer function) and (2) enhancing the (global) image contrast (CHRE transfer function). The DRC subsystem adapts the HDR video signal to the remainder of the system, which can handle only a fraction of the original dynamic range. Our main contribution is a novel DRC function shape which is adaptive to the image, so that details in the dark image parts are enhanced simultaneously while only moderately compressing details in the bright areas. Also, the DRC function shape is well matched with the sensor noise characteristics in order to limit the noise amplification. Furthermore, we show that the image quality can be significantly improved in DRC compression if a local contrast preservation step is included. The second part of GTMF is a CHRE subsystem that fine-tunes and redistributes the luminance (and color) signal in the image, to optimize global contrast of the scene. The contribution of the proposed CHRE processing is that unlike standard histogram equalization, it can preserve details in statistically unpopulated but visually relevant luminance regions. One of the important cornerstones of the GTMF is that both DRC and CHRE algorithms are performed in the perceptually uniform space and optimized for the salient regions obtained by the improved salient-region detector, to maximize the relevant information transfer to the HVS. The proposed GTMF solution offers a good processing quality, but cannot sufficiently preserve local contrast for extreme HDR signals and it gives limited improvement low-contrast scenes. The local contrast improvement is based on the Locally Adaptive Contrast Enhancement (LACE) algorithm. We contribute by using multi-band frequency decomposition, to set up the complete enhancement system. Four key problems occur with real-time LACE processing: (1) "halo" artifacts, (2) clipping of the enhancement signal, (3) noise degradation and (4) the overall system complexity. "Halo" artifacts are eliminated by a new contrast gain specification using local energy and contrast measurements. This solution has a low complexity and offers excellent performance in terms of higher contrast and visually appealing performance. Algorithms preventing clipping of the output signal and reducing noise amplification give a further enhancement. We have added a supplementary discussion on executing LACE in the logarithmic domain, where we have derived a new contrast gain function solving LACE problems efficiently. For the best results, we have found that LACE processing should be performed in the logarithmic domain for standard and HDR images, and in the linear domain for low-contrast images. Finally, the complexity of the contrast gain calculation is reduced by a new local energy metric, which can be calculated efficiently in a 2D-separable fashion. Besides the complexity benefit, the proposed energy metric gives better performance compared to the conventional metrics. The conclusions of our work are summarized as follows. For acquisition, we need to combine an optimal exposure algorithm, giving both improved dynamic performance and maximum image contrast/SNR, with robust exposure bracketing that can handle difficult conditions such as fluorescent lighting. For optimizing visibility of details in the scene, we have split the GTMF in two parts, DRC and CHRE, so that a controlled optimization can be performed offering less contrast compression and detail loss than in the conventional case. Local contrast is enhanced with the known LACE algorithm, but the performance is significantly improved by individually addressing "halo" artifacts, signal clipping and noise degradation. We provide artifact reduction by new contrast gain function based on local energy, contrast measurements and noise estimation. Besides the above arguments, we have contributed feasible performance metrics and listed ample practical evidence of the real-time implementation of our algorithms in FPGAs and ASICs, used in commercially available surveillance cameras, which obtained awards for their image quality

Põhjalik uuring ülisuure dünaamilise ulatusega piltide toonivastendamisest koos subjektiivsete testidega

A high dynamic range (HDR) image has a very wide range of luminance levels that traditional low dynamic range (LDR) displays cannot visualize. For this reason, HDR images are usually transformed to 8-bit representations, so that the alpha channel for each pixel is used as an exponent value, sometimes referred to as exponential notation [43]. Tone mapping operators (TMOs) are used to transform high dynamic range to low dynamic range domain by compressing pixels so that traditional LDR display can visualize them. The purpose of this thesis is to identify and analyse differences and similarities between the wide range of tone mapping operators that are available in the literature. Each TMO has been analyzed using subjective studies considering different conditions, which include environment, luminance, and colour. Also, several inverse tone mapping operators, HDR mappings with exposure fusion, histogram adjustment, and retinex have been analysed in this study. 19 different TMOs have been examined using a variety of HDR images. Mean opinion score (MOS) is calculated on those selected TMOs by asking the opinion of 25 independent people considering candidates’ age, vision, and colour blindness

Digital Camera Workflow for High Dynamic Range Images Using a Model of Retinal Processing

We propose a complete digital camera workflow to capture and render high dynamic range (HDR) static scenes, from RAW sensor data to an output- referred encoded image. In traditional digital camera processing, demosaicing is one of the first operations done after scene analysis. It is followed by rendering operations, such as color correction and tone mapping. Our approach is based on a model of retinal processing of the human visual system (HVS). In the HVS, rendering operations, including adaptation, are performed directly on the cone responses, which corresponds to a mosaic image. Our workflow conforms more closely to the retinal processing model, performing all rendering before demosaicing.. This reduces the complexity of the computation, as only one third of the pixels are processed. This is especially important as our tone mapping operator applies local and global tone corrections, which is usually needed to well render high dynamic scenes. Our algorithms efficiently process HDR images with different keys and different content

Tone mapping for high dynamic range images

Tone mapping is an essential step for the reproduction of "nice looking" images. It provides the mapping between the luminances of the original scene to the output device's display values. When the dynamic range of the captured scene is smaller or larger than that of the display device, tone mapping expands or compresses the luminance ratios. We address the problem of tone mapping high dynamic range (HDR) images to standard displays (CRT, LCD) and to HDR displays. With standard displays, the dynamic range of the captured HDR scene must be compressed significantly, which can induce a loss of contrast resulting in a loss of detail visibility. Local tone mapping operators can be used in addition to the global compression to increase the local contrast and thus improve detail visibility, but this tends to create artifacts. We developed a local tone mapping method that solves the problems generally encountered by local tone mapping algorithms. Namely, it does not create halo artifacts, nor graying-out of low contrast areas, and provides good color rendition. We then investigated specifically the rendition of color and confirmed that local tone mapping algorithms must be applied to the luminance channel only. We showed that the correlation between luminance and chrominance plays a role in the appearance of the final image but a perfect decorrelation is not necessary. Recently developed HDR monitors enable the display of HDR images with hardly any compression of their dynamic range. The arrival of these displays on the market create the need for new tone mapping algorithms. In particular, legacy images that were mapped to SDR displays must be re-rendered to HDR displays, taking best advantage of the increase in dynamic range. This operation can be seen as the reverse of the tone mapping to SDR. We propose a piecewise linear tone scale function that enhances the brightness of specular highlights so that the sensation of naturalness is improved. Our tone scale algorithm is based on the segmentation of the image into its diffuse and specular components as well as on the range of display luminance that is allocated to the specular component and the diffuse component, respectively. We performed a psychovisual experiment to validate the benefit of our tone scale. The results showed that, with HDR displays, allocating more luminance range to the specular component than what was allocated in the image rendered to SDR displays provides more natural looking images

Mixing tone mapping operators on the GPU by differential zone mapping based on psychophysical experiments

© 2016 In this paper, we present a new technique for displaying High Dynamic Range (HDR) images on Low Dynamic Range (LDR) displays in an efficient way on the GPU. The described process has three stages. First, the input image is segmented into luminance zones. Second, the tone mapping operator (TMO) that performs better in each zone is automatically selected. Finally, the resulting tone mapping (TM) outputs for each zone are merged, generating the final LDR output image. To establish the TMO that performs better in each luminance zone we conducted a preliminary psychophysical experiment using a set of HDR images and six different TMOs. We validated our composite technique on several (new) HDR images and conducted a further psychophysical experiment, using an HDR display as the reference that establishes the advantages of our hybrid three-stage approach over a traditional individual TMO. Finally, we present a GPU version, which is perceptually equal to the standard version but with much improved computational performance

Fast Local Tone Mapping, Summed-Area Tables and Mesopic Vision Simulation

広島大学(Hiroshima University)博士(工学)Engineeringdoctora