Object-based Illumination Estimation with Rendering-aware Neural Networks

We present a scheme for fast environment light estimation from the RGBD appearance of individual objects and their local image areas. Conventional inverse rendering is too computationally demanding for real-time applications, and the performance of purely learning-based techniques may be limited by the meager input data available from individual objects. To address these issues, we propose an approach that takes advantage of physical principles from inverse rendering to constrain the solution, while also utilizing neural networks to expedite the more computationally expensive portions of its processing, to increase robustness to noisy input data as well as to improve temporal and spatial stability. This results in a rendering-aware system that estimates the local illumination distribution at an object with high accuracy and in real time. With the estimated lighting, virtual objects can be rendered in AR scenarios with shading that is consistent to the real scene, leading to improved realism.Comment: ECCV 202

Hierarchical learning classifier systems for polymorphism in heterogeneous niches

Learning classifier systems (LCSs) have been successfully adapted to real-world domains with the claim of human-readable rule populations. However, due to the inherent rich characteristic of the employed representation, it is possible to represent the underlying patterns in multiple (polymorphic) ways, which obscures the most informative patterns. A novel rule reduction algorithm is proposed based on ensembles of multiple trained LCSs populations in a hierarchical learning architecture to reduce the local diversity and global polymorphism. The primary aim of this project is to interrogate the hidden patterns in LCSs’ trained population rather than improve the predictive power on test sets. This enables successful visualization of the importance of features in data groups (niches) that can contain heterogeneous patterns, i.e. even if different patterns result in the same class the importance of features can be found.</p