98 research outputs found
Accurate and Efficient Filtering using Anisotropic Filter Decomposition
Efficient filtering remains an important challenge in computer graphics, particularly when filters are spatially-varying, have large extent, and/or exhibit complex anisotropic profiles. We present an efficient filtering approach for these difficult cases based on anisotropic filter decomposition (IFD). By decomposing complex filters into linear combinations of simpler, displaced isotropic kernels, and precomputing a compact prefiltered dataset, we are able to interactively apply any number of---potentially transformed---filters to a signal. Our performance scales linearly with the size of the decomposition, not the size nor the dimensionality of the filter, and our prefiltered data requires reasonnable storage, comparing favorably to the state-of-the-art. We apply IFD to interesting problems in image processing and realistic rendering.Les opérations de filtrage en synthèse/analyse d'images sont coûteuses à effectuer lorsque les filtres varient spatialement, sont très étendus et/ou très anisotropes. Nous présentons dans ce cas précis une méthode pour rendre le filtrage efficace, basée sur une décomposition du filtre en une combinaison linéaire de filtres isotropes, en translation. Le coût de notre méthode est linéaire par rapport au nombre de filtres utilisés dans la décomposition, et ne dépend pas de la taille des données filtrées. Nous en présentons différentes applications, en analyses d'images et en rendu
Attention-based Neural Cellular Automata
Recent extensions of Cellular Automata (CA) have incorporated key ideas from
modern deep learning, dramatically extending their capabilities and catalyzing
a new family of Neural Cellular Automata (NCA) techniques. Inspired by
Transformer-based architectures, our work presents a new class of
NCAs formed using a spatially
localized\unicode{x2014}yet globally organized\unicode{x2014}self-attention
scheme. We introduce an instance of this class named (ViTCA). We present quantitative and
qualitative results on denoising autoencoding across six benchmark datasets,
comparing ViTCA to a U-Net, a U-Net-based CA baseline (UNetCA), and a Vision
Transformer (ViT). When comparing across architectures configured to similar
parameter complexity, ViTCA architectures yield superior performance across all
benchmarks and for nearly every evaluation metric. We present an ablation study
on various architectural configurations of ViTCA, an analysis of its effect on
cell states, and an investigation on its inductive biases. Finally, we examine
its learned representations via linear probes on its converged cell state
hidden representations, yielding, on average, superior results when compared to
our U-Net, ViT, and UNetCA baselines.Comment: NeurIPS 202
A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem
Training multiple agents to coordinate is an important problem with
applications in robotics, game theory, economics, and social sciences. However,
most existing Multi-Agent Reinforcement Learning (MARL) methods are online and
thus impractical for real-world applications in which collecting new
interactions is costly or dangerous. While these algorithms should leverage
offline data when available, doing so gives rise to the offline coordination
problem. Specifically, we identify and formalize the strategy agreement (SA)
and the strategy fine-tuning (SFT) challenges, two coordination issues at which
current offline MARL algorithms fail. To address this setback, we propose a
simple model-based approach that generates synthetic interaction data and
enables agents to converge on a strategy while fine-tuning their policies
accordingly. Our resulting method, Model-based Offline Multi-Agent Proximal
Policy Optimization (MOMA-PPO), outperforms the prevalent learning methods in
challenging offline multi-agent MuJoCo tasks even under severe partial
observability and with learned world models
Filtering Non-Linear Transfer Functions on Surfaces
International audienceApplying non-linear transfer functions and look-up tables to procedural functions (such as noise), surface attributes, or even surface geometry are common strategies used to enhance visual detail. Their simplicity and ability to mimic a wide range of realistic appearances have led to their adoption in many rendering problems. As with any textured or geometric detail, proper filtering is needed to reduce aliasing when viewed across a range of distances, but accurate and efficient transfer function filtering remains an open problem for several reasons: transfer functions are complex and non-linear, especially when mapped through procedural noise and/or geometry-dependent functions, and the effects of perspective and masking further complicate the filtering over a pixel's footprint. We accurately solve this problem by computing and sampling from specialized filtering distributions on the fly, yielding very fast performance. We investigate the case where the transfer function to filter is a color map applied to (macroscale) surface textures (like noise), as well as color maps applied according to (microscale) geometric details. We introduce a novel representation of a (potentially modulated) color map's distribution over pixel footprints using Gaussian statistics and, in the more complex case of high-resolution color mapped microsurface details, our filtering is view- and light-dependent, and capable of correctly handling masking and occlusion effects. Our approach can be generalized to filter other physical-based rendering quantities. We propose an application to shading with irradiance environment maps over large terrains. Our framework is also compatible with the case of transfer functions used to warp surface geometry, as long as the transformations can be represented with Gaussian statistics, leading to proper view- and light-dependent filtering results. Our results match ground truth and our solution is well suited to real-time applications, requires only a few lines of shader code (provided in supplemental material), is high performance, and has a negligible memory footprint
Frequency Based Radiance Cache for Rendering Animations
International audienceWe propose a method to render animation sequences with direct distant lighting that only shades a fraction of the total pixels. We leverage frequency-based analyses of light transport to determine shading and image sampling rates across an animation using a samples cache. To do so, we derive frequency bandwidths that account for the complexity of distant lights, visibility, BRDF, and temporal coherence during animation. We finaly apply a cross-bilateral filter when rendering our final images from sparse sets of shading points placed according to our frequency-based oracles (generally < 25% of the pixels, per frame)
Learning Neural Implicit Representations with Surface Signal Parameterizations
Neural implicit surface representations have recently emerged as popular
alternative to explicit 3D object encodings, such as polygonal meshes,
tabulated points, or voxels. While significant work has improved the geometric
fidelity of these representations, much less attention is given to their final
appearance. Traditional explicit object representations commonly couple the 3D
shape data with auxiliary surface-mapped image data, such as diffuse color
textures and fine-scale geometric details in normal maps that typically require
a mapping of the 3D surface onto a plane, i.e., a surface parameterization;
implicit representations, on the other hand, cannot be easily textured due to
lack of configurable surface parameterization. Inspired by this digital content
authoring methodology, we design a neural network architecture that implicitly
encodes the underlying surface parameterization suitable for appearance data.
As such, our model remains compatible with existing mesh-based digital content
with appearance data. Motivated by recent work that overfits compact networks
to individual 3D objects, we present a new weight-encoded neural implicit
representation that extends the capability of neural implicit surfaces to
enable various common and important applications of texture mapping. Our method
outperforms reasonable baselines and state-of-the-art alternatives
- …