Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications

We present an overview and evaluation of a new, systematic approach for generation of highly realistic, annotated synthetic data for training of deep neural networks in computer vision tasks. The main contribution is a procedural world modeling approach enabling high variability coupled with physically accurate image synthesis, and is a departure from the hand-modeled virtual worlds and approximate image synthesis methods used in real-time applications. The benefits of our approach include flexible, physically accurate and scalable image synthesis, implicit wide coverage of classes and features, and complete data introspection for annotations, which all contribute to quality and cost efficiency. To evaluate our approach and the efficacy of the resulting data, we use semantic segmentation for autonomous vehicles and robotic navigation as the main application, and we train multiple deep learning architectures using synthetic data with and without fine tuning on organic (i.e. real-world) data. The evaluation shows that our approach improves the neural network's performance and that even modest implementation efforts produce state-of-the-art results.Comment: The project web page at http://vcl.itn.liu.se/publications/2017/TKWU17/ contains a version of the paper with high-resolution images as well as additional materia

A radiative transfer framework for non-exponential media

We develop a new theory of volumetric light transport for media with non-exponential free-flight distributions. Recent insights from atmospheric sciences and neutron transport demonstrate that such distributions arise in the presence of correlated scatterers, which are naturally produced by processes such as cloud condensation and fractal-pattern formation. Our theory accommodates correlations by disentangling the concepts of the free-flight distribution and transmittance, which are equivalent when scatterers are statistically independent, but become distinct when correlations are present. Our theory results in a generalized path integral which allows us to handle non-exponential media using the full range of Monte Carlo rendering algorithms while enriching the range of achievable appearance. We propose parametric models for controlling the statistical correlations by leveraging work on stochastic processes, and we develop a method to combine such unresolved correlations (and the resulting non-exponential free-flight behavior) with explicitly modeled macroscopic heterogeneity. This provides a powerful authoring approach where artists can freely design the shape of the attenuation profile separately from the macroscopic heterogeneous density, while our theory provides a physically consistent interpretation in terms of a path space integral. We address important considerations for graphics including energy conservation, reciprocity, and bidirectional rendering algorithms, all in the presence of surfaces and correlated media

Procedural feature generation for volumetric terrains using voxel grammars

© 2018 Terrain generation is a fundamental requirement of many computer graphics simulations, including computer games, flight simulators and environments in feature films. There has been a considerable amount of research in this domain, which ranges between fully automated and semi-automated methods. Voxel representations of 3D terrains can create rich features that are not found in other forms of terrain generation techniques, such as caves and overhangs. In this article, we introduce a semi-automated method of generating features for volumetric terrains using a rule-based procedural generation system. Features are generated by selecting subsets of a voxel grid as input symbols to a grammar, composed of user-created operators. This results in overhangs and caves generated from a set of simple rules. The feature generation runs on the CPU and the GPU is utilised to extract a robust mesh from the volumetric dataset

NOVA: rendering virtual worlds with humans for computer vision tasks

Today, the cutting edge of computer vision research greatly depends on the availability of large datasets, which are critical for effectively training and testing new methods. Manually annotating visual data, however, is not only a labor-intensive process but also prone to errors. In this study, we present NOVA, a versatile framework to create realistic-looking 3D rendered worlds containing procedurally generated humans with rich pixel-level ground truth annotations. NOVA can simulate various environmental factors such as weather conditions or different times of day, and bring an exceptionally diverse set of humans to life, each having a distinct body shape, gender and age. To demonstrate NOVA's capabilities, we generate two synthetic datasets for person tracking. The first one includes 108 sequences, each with different levels of difficulty like tracking in crowded scenes or at nighttime and aims for testing the limits of current state-of-the-art trackers. A second dataset of 97 sequences with normal weather conditions is used to show how our synthetic sequences can be utilized to train and boost the performance of deep-learning based trackers. Our results indicate that the synthetic data generated by NOVA represents a good proxy of the real-world and can be exploited for computer vision tasks

Fluid Simulation for Visual Effects

This thesis describes a system for dealing with free surface fluid simulations, and the components needed in order to construct such a system. It builds upon recent research, but in a computer graphics context the amount of available literature is limited and difficult to implement. Because of this, the text aims at providing a solid foundation of the mathematics needed, at explaining in greater detail the steps needed to solve the problem, and lastly at improving some aspects of the animation process as it has been described in earlier works. The aim of the system itself is to provide visually plausible renditions of animated fluids in three dimensions in a manner that allows it to be usable in a visual effects production context. The novel features described include a generalized interaction layer providing greater control to artists, a new way of dealing with moving objects that interact with the fluid and a method for adding source and drain capabilities

Production volume rendering: design and implementation

Due to limited publicly available software and lack of documentation, those involved with production volume rendering often have to start from scratch creating the necessary elements to make their system work. Production Volume Rendering: Design and Implementation provides the first full account of volume rendering techniques used for feature animation and visual effects production. It covers the theoretical underpinnings as well as the implementation of a working renderer. The book offers two paths toward understanding production volume rendering. It describes: Modern production volume rendering techniques in a generic context, explaining how the techniques fit together and how the modules are used to achieve real-world goals Implementation of the techniques, showing how to translate abstract concepts into concrete, working code and how the ideas work together to create a complete system As an introduction to the field and an overview of current techniques and algorithms, this book is a valuable source of information for programmers, technical directors, artists, and anyone else interested in how production volume rendering works

Direct Transmittance Estimation in Heterogeneous Participating Media Using Approximated Taylor Expansions

Evaluating the transmittance between two points along a ray is a key component in solving the light transport through heterogeneous participating media and entails computing an intractable exponential of the integrated medium's extinction coefficient. While algorithms for estimating this transmittance exist, there is a lack of theoretical knowledge about their behaviour, which also prevent new theoretically sound algorithms from being developed. For this purpose, we introduce a new class of unbiased transmittance estimators based on random sampling or truncation of a Taylor expansion of the exponential function. In contrast to classical tracking algorithms, these estimators are non-analogous to the physical light transport process and directly sample the underlying extinction function without performing incremental advancement. We present several versions of the new class of estimators, based on either importance sampling or Russian roulette to provide finite unbiased estimators of the infinite Taylor series expansion. We also show that the well known ratio tracking algorithm can be seen as a special case of the new class of estimators. Lastly, we conduct performance evaluations on both the central processing unit (CPU) and the graphics processing unit (GPU), and the results demonstrate that the new algorithms outperform traditional algorithms for heterogeneous mediums.Funding: Knut and Alice Wallenberg Foundation (KAW) [2013-0076]; SeRC (Swedish e-Science Research Center); Wallenberg AI, Autonomous Systems and Software Program (WASP); Swedish Foundation for Strategic Research (SSF) via the project ASSEMBLE [RIT15-0012]; ELLIIT environment for strategic research in Sweden</p