Deep Bilateral Learning for Real-Time Image Enhancement

Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images, we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher.Comment: 12 pages, 14 figures, Siggraph 201

DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks

Despite a rapid rise in the quality of built-in smartphone cameras, their physical limitations - small sensor size, compact lenses and the lack of specific hardware, - impede them to achieve the quality results of DSLR cameras. In this work we present an end-to-end deep learning approach that bridges this gap by translating ordinary photos into DSLR-quality images. We propose learning the translation function using a residual convolutional neural network that improves both color rendition and image sharpness. Since the standard mean squared loss is not well suited for measuring perceptual image quality, we introduce a composite perceptual error function that combines content, color and texture losses. The first two losses are defined analytically, while the texture loss is learned in an adversarial fashion. We also present DPED, a large-scale dataset that consists of real photos captured from three different phones and one high-end reflex camera. Our quantitative and qualitative assessments reveal that the enhanced image quality is comparable to that of DSLR-taken photos, while the methodology is generalized to any type of digital camera

An information assistant system for the prevention of tunnel vision in crisis management

In the crisis management environment, tunnel vision is a set of bias in decision makers’ cognitive process which often leads to incorrect understanding of the real crisis situation, biased perception of information, and improper decisions. The tunnel vision phenomenon is a consequence of both the challenges in the task and the natural limitation in a human being’s cognitive process. An information assistant system is proposed with the purpose of preventing tunnel vision. The system serves as a platform for monitoring the on-going crisis event. All information goes through the system before arrives at the user. The system enhances the data quality, reduces the data quantity and presents the crisis information in a manner that prevents or repairs the user’s cognitive overload. While working with such a system, the users (crisis managers) are expected to be more likely to stay aware of the actual situation, stay open minded to possibilities, and make proper decisions

Evaluation of the effectiveness of HDR tone-mapping operators for photogrammetric applications

[EN] The ability of High Dynamic Range (HDR) imaging to capture the full range of lighting in a scene has meant that it is being increasingly used for Cultural Heritage (CH) applications. Photogrammetric techniques allow the semi-automatic production of 3D models from a sequence of images. Current photogrammetric methods are not always effective in reconstructing images under harsh lighting conditions, as significant geometric details may not have been captured accurately within under- and over-exposed regions of the image. HDR imaging offers the possibility to overcome this limitation, however the HDR images need to be tone mapped before they can be used within existing photogrammetric algorithms. In this paper we evaluate four different HDR tone-mapping operators (TMOs) that have been used to convert raw HDR images into a format suitable for state-of-the-art algorithms, and in particular keypoint detection techniques. The evaluation criteria used are the number of keypoints, the number of valid matches achieved and the repeatability rate. The comparison considers two local and two global TMOs. HDR data from four CH sites were used: Kaisariani Monastery (Greece), Asinou Church (Cyprus), Château des Baux (France) and Buonconsiglio Castle (Italy).We would like to thank Kurt Debattista, Timothy Bradley, Ratnajit Mukherjee, Diego Bellido Castañeda and TomBashford Rogers for their suggestions, help and encouragement. We would like to thank the hosting institutions: 3D Optical Metrology Group, FBK (Trento, Italy) and UMR 3495 MAP CNRS/MCC (Marseille, France), for their support during the data acquisition campaigns. This project has received funding from the European Union’s 7 th Framework Programme for research, technological development and demonstration under grant agreement No. 608013, titled “ITN-DCH: Initial Training Network for Digital Cultural Heritage: Projecting our Past to the Future”.Suma, R.; Stavropoulou, G.; Stathopoulou, EK.; Van Gool, L.; Georgopoulos, A.; Chalmers, A. (2016). Evaluation of the effectiveness of HDR tone-mapping operators for photogrammetric applications. Virtual Archaeology Review. 7(15):54-66. https://doi.org/10.4995/var.2016.6319SWORD546671

Estudi comparatiu del comportament perceptual de diferents algorismes Tone Mapping

D'experiments diferents per avaluar "tone mapping operators" se n'ha fet de molts tipus diferents i amb molts objectius diferents. Nosaltres, en aquest treball, hem realitzat experiments psicofísics sobre 15 algorismes "tone mapping" diferents per tal d'obtenir un ranking des del punt de vista de la percepció humana. Per aconseguir aquest ranking, hem realitzat dos experiments diferents: un per estudiar la relació de tonalitats de grisos en la imatge "tone mapped" i en l'escena HDR real i un "pairwise comparison" dels 15 algorismes, observant l'escena HDR real just al costat del monitor CRT calibrat. Els resultats dels experiments han demostrat que, pel primer experiment, iCAM és el millor amb diferència, mentre que pel segon experiment, els millors han estat Kraw i KimK.There are a lot of different experiments to evaluate tone mapping operators. In this work, we performed psycophysics experiments with 15 different tone mapping algorithms to obtain a human perceptual ranking. To achieve this ranking, we performed two different experiments: one to study the relationship between gray tones, and the other performed a pairwise comparison of these 15 algorithms. In the first one, we matched the gray tones in the tone mapped image and in the HDR real scene. In the second one, we performed a pairwise comparison of tone mapped images, while watching the HDR real scene next to the CRT calibrated display. We built a ranking from the pairwise comparison results using the Balanced Rank Estimation method implemented by Wauthier et al.(2013). The results of the experiments showed that iCAM is the best tone mapping operator for the first experiment, while Kraw and KimK are the best for the second one