Path Replay Backpropagation: Differentiating Light Paths using Constant Memory and Linear Time

Differentiable physically-based rendering has become an indispensable tool for solving inverse problems involving light. Most applications in this area jointly optimize a large set of scene parameters to minimize an objective function, in which case reverse-mode differentiation is the method of choice for obtaining parameter gradients.However, existing techniques that perform the necessary differentiation step suffer from either statistical bias or a prohibitive cost in terms of memory and computation time. For example, standard techniques for automatic differentiation based on program transformation or Wengert tapes lead to impracticably large memory usage when applied to physically-based rendering algorithms. A recently proposed adjoint method by Nimier-David et al. [2020] reduces this to a constant memory footprint, but the computation time for unbiased gradient estimates then becomes quadratic in the number of scattering events along a light path. This is problematic when the scene contains highly scattering materials like participating media.In this paper, we propose a new unbiased backpropagation algorithm for rendering that only requires constant memory, and whose computation time is linear in the number of scattering events (i.e., just like path tracing). Our approach builds on the invertibility of the local Jacobian at scattering interactions to recover the various quantities needed for reverse-mode differentiation. Our method also extends to specular materials such as smooth dielectrics and conductors that cannot be handled by prior work

Radiative Backpropagation: An Adjoint Method for Lightning-Fast Differentiable Rendering

Physically based differentiable rendering has recently evolved into a powerful tool for solving inverse problems involving light. Methods in this area perform a differentiable simulation of the physical process of light transport and scattering to estimate partial derivatives relating scene parameters to pixels in the rendered image. Together with gradient-based optimization, such algorithms have interesting applications in diverse disciplines, e.g., to improve the reconstruction of 3D scenes, while accounting for interreflection and transparency, or to design meta-materials with specified optical properties.The most versatile differentiable rendering algorithms rely on reverse-mode differentiation to compute all requested derivatives at once, enabling optimization of scene descriptions with millions of free parameters. However, a severe limitation of the reverse-mode approach is that it requires a detailed transcript of the computation that is subsequently replayed to back-propagate derivatives to the scene parameters. The transcript of typical renderings is extremely large, exceeding the available system memory by many orders of magnitude, hence current methods are limited to simple scenes rendered at low resolutions and sample counts.We introduce radiative backpropagation, a fundamentally different approach to differentiable rendering that does not require a transcript, greatly improving its scalability and efficiency. Our main insight is that reverse-mode propagation through a rendering algorithm can be interpreted as the solution of a continuous transport problem involving the partial derivative of radiance with respect to the optimization objective. This quantity is "emitted" by sensors, "scattered" by the scene, and eventually "received" by objects with differentiable parameters. Differentiable rendering then decomposes into two separate primal and adjoint simulation steps that scale to complex scenes rendered at high resolutions. We also investigated biased variants of this algorithm and find that they considerably improve both runtime and convergence speed. We showcase an efficient GPU implementation of radiative backpropagation and compare its performance arid the quality of its gradients to prior work

High Fidelity Visualization of Large Scale Digitally Reconstructed Brain Circuitry with Signed Distance Functions

We explore a first proof-of-concept application for visualizing large scale digitally reconstructed brain circuitry using signed distance functions. The significance of our method is demonstrated in comparison with using implicit geometry that is limited to provide the natural look of neurons or explicit geometry that requires huge amounts of memory and has limited scalability with larger circuits

BlueBrain/Brayns: 3.2.2 Release

Visualizer for large-scale and interactive ray-tracing of neuron

BlueBrain/Brayns: 3.3.0 Release

Visualizer for large-scale and interactive ray-tracing of neuron

BlueBrain/Brayns: 3.4.0 Release

Visualizer for large-scale and interactive ray-tracing of neuron

BlueBrain/Brayns: 3.5.0 Release

Visualizer for large-scale and interactive ray-tracing of neuron

BlueBrain/Brayns: 3.5.1 Release

Visualizer for large-scale and interactive ray-tracing of neuron

mitsuba-renderer/mitsuba3: v3.4.1

<p>Changes in this version:</p> <ul> <li><p>Upgrade Dr.Jit to <a href="https://github.com/mitsuba-renderer/drjit/releases/tag/v0.4.4">v0.4.4</a></p> <ul> <li>Solved threading/concurrency issues which could break loading of large scenes or long running optimizations</li> </ul> </li> <li><p>Scene's bounding box now gets updated on parameter changes <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/97d4b6ad4c1ba3471642c177cee01d3adf0bf22e">97d4b6a</a></p> </li> <li><p>Python bindings for <code>mi.lookup_ior</code> <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/d598d79a7d21c76ac9b422b3488137b1d28a33f9">d598d79</a></p> </li> <li><p>Fixes to <code>mask</code> BSDF when differentiated <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/ee87f1c01aa1b731bc58057ed9e6944046460a69">ee87f1c</a></p> </li> <li><p>Ray sampling is fixed when <code>sample_border</code> is used <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/c10b87b072634db15d55a7dbc55cc3cf8f7c844c">c10b87b</a></p> </li> <li><p>Rename OpenEXR shared library <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/9cc3bf495da10dcd28e80cc14a145fb178a5ef4c">9cc3bf4</a></p> </li> <li><p>Handle phase function differentiation in <code>prbvolpath</code> <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/5f9eebd41a3a939096d4509b1d2504586a3bf7c6">5f9eebd</a></p> </li> <li><p>Fixes to linear <code>retarder</code> <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/8033a807091f8315c5cef25f4f1a36a3766fb223">8033a80</a></p> </li> <li><p>Avoid copies to host when building 1D distributions <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/825f44f081fb43b23589b2bf0b9b7071af858f2a">825f44f</a> .. <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/8f71fe995f40923449478ee05500918710ef27f6">8f71fe9</a></p> </li> <li><p>Fixes to linear <code>retarder</code> <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/8033a807091f8315c5cef25f4f1a36a3766fb223">8033a80</a></p> </li> <li><p>Sensor's prinicpal point is now exposed throught <code>m̀i.traverse()</code> <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/f59faa51929b506608a66522dc841f5317a8d43c">f59faa5</a></p> </li> <li><p>Minor fixes to <code>ptracer</code> which could result in illegal memory accesses <a href="https://github.com/mitsuba-renderer/mitsuba3/commit/3d902a4dbf176c8c8d08e5493f23623659295197">3d902a4</a></p> </li> <li><p>Other various minor bug fixes</p> </li> </ul&gt

mitsuba-renderer/mitsuba3: v3.5.0

<p>Changes in this version:</p> <ul> <li><p>New projective sampling based integrators, see PR #997 for more details. Here's a brief overview of some of the major or breaking changes:</p> <ul> <li>New <code>prb_projective</code> and <code>direct_projective</code> integrators</li> <li>New curve/shadow optimization tutorial</li> <li>Removed reparameterizations</li> <li>Can no longer differentiate <code>instance</code>, <code>sdfgrid</code> and <code>Sensor</code>'s positions</li> </ul> </li> </ul&gt