CNN based Learning using Reflection and Retinex Models for Intrinsic Image Decomposition

Most of the traditional work on intrinsic image decomposition rely on deriving priors about scene characteristics. On the other hand, recent research use deep learning models as in-and-out black box and do not consider the well-established, traditional image formation process as the basis of their intrinsic learning process. As a consequence, although current deep learning approaches show superior performance when considering quantitative benchmark results, traditional approaches are still dominant in achieving high qualitative results. In this paper, the aim is to exploit the best of the two worlds. A method is proposed that (1) is empowered by deep learning capabilities, (2) considers a physics-based reflection model to steer the learning process, and (3) exploits the traditional approach to obtain intrinsic images by exploiting reflectance and shading gradient information. The proposed model is fast to compute and allows for the integration of all intrinsic components. To train the new model, an object centered large-scale datasets with intrinsic ground-truth images are created. The evaluation results demonstrate that the new model outperforms existing methods. Visual inspection shows that the image formation loss function augments color reproduction and the use of gradient information produces sharper edges. Datasets, models and higher resolution images are available at https://ivi.fnwi.uva.nl/cv/retinet.Comment: CVPR 201

Recent Progress in Image Deblurring

This paper comprehensively reviews the recent development of image deblurring, including non-blind/blind, spatially invariant/variant deblurring techniques. Indeed, these techniques share the same objective of inferring a latent sharp image from one or several corresponding blurry images, while the blind deblurring techniques are also required to derive an accurate blur kernel. Considering the critical role of image restoration in modern imaging systems to provide high-quality images under complex environments such as motion, undesirable lighting conditions, and imperfect system components, image deblurring has attracted growing attention in recent years. From the viewpoint of how to handle the ill-posedness which is a crucial issue in deblurring tasks, existing methods can be grouped into five categories: Bayesian inference framework, variational methods, sparse representation-based methods, homography-based modeling, and region-based methods. In spite of achieving a certain level of development, image deblurring, especially the blind case, is limited in its success by complex application conditions which make the blur kernel hard to obtain and be spatially variant. We provide a holistic understanding and deep insight into image deblurring in this review. An analysis of the empirical evidence for representative methods, practical issues, as well as a discussion of promising future directions are also presented.Comment: 53 pages, 17 figure

Scalable, Detailed and Mask-Free Universal Photometric Stereo

In this paper, we introduce SDM-UniPS, a groundbreaking Scalable, Detailed, Mask-free, and Universal Photometric Stereo network. Our approach can recover astonishingly intricate surface normal maps, rivaling the quality of 3D scanners, even when images are captured under unknown, spatially-varying lighting conditions in uncontrolled environments. We have extended previous universal photometric stereo networks to extract spatial-light features, utilizing all available information in high-resolution input images and accounting for non-local interactions among surface points. Moreover, we present a new synthetic training dataset that encompasses a diverse range of shapes, materials, and illumination scenarios found in real-world scenes. Through extensive evaluation, we demonstrate that our method not only surpasses calibrated, lighting-specific techniques on public benchmarks, but also excels with a significantly smaller number of input images even without object masks.Comment: CVPR 2023 (Highlight). The source code will be available at https://github.com/satoshi-ikehata/SDM-UniPS-CVPR202

Tehnike zrcaljenja u Real-Time računalnoj grafici

Reflections have a long history in computer graphics, as they are important for conveying a sense of realism as well as depth and proportion. Their implementations come with a multitude of difficulties, and each solution typically has various trade-offs. Approaches highly depend on the geometry of the reflective surface since curved reflectors are usually more difficult to portray accurately. Techniques can typically be categorized by whether they work with the actual geometry of the reflected objects or with an image of these objects. For curved surfaces, image-based techniques are usually preferred, whereas for planar surfaces the reflected geometry can be used more easily because of the lack of distortion. With current advances in graphics hardware technology, ray tracing is also becoming more viable for real-time applications. Many modern solutions often combine multiple approaches to form a hybrid technique. In this paper, we give an overview of the techniques used in computer graphics applications to create real-time reflections. We highlight the trade-offs that have to be dealt with when choosing a particular technique, as well as their ability to produce interreflections. Finally, we describe how contemporary state-of-the-art rendering engines deal with reflections.Zrcaljenja imaju dugu povijest primjene u računalnoj grafici zbog njihove važnosti u prenošenju realističnosti prikaza te prikaza dubine i omjera na slikama. Pri implementaciji zrcaljenja dolazimo do raznih teškoća i svako novo rješenje često imaju svoju cijenu. Pristupi implementacije ovise o geometriji plohe na kojoj leži prikaz, Što je ploha zakrivljenija, to je teže postići vjerni prikaz. Tehnike možemo kategorizirati u one koje rade sa stvarnom geometrijom zrcaljenih objekata te one koje rade samo sa slikama objekata. Kod zakrivljenih ploha koriste se tehnike bazirane na slikama, dok se kod ravninskih ploha koristi zrcaljena geometrija jer nema iskrivljenja. Zahvaljujući trenutnom razvoju tehnologije grafičkih hardvera, metoda praćenja zraka (ray tracing) postaje sve isplativija u real-time primjeni. Mnoga moderna rješenja kombiniraju razne pristupe i dolazi do hibridnih tehnika. U ovom radu dajemo pregled tehnika korištenih u primjeni računalne grafike za postizanje real-time zrcalnih slika. Naglašavamo probleme koji nastaju pri korištenju određene tehnike te njihove mogućnosti u pogledu stvaranja međuzrcaljenja. Naposljetku, opisujemo kako moderni alati za renderiranje rješavaju probleme zrcaljenj