Joint Learning of Intrinsic Images and Semantic Segmentation

Semantic segmentation of outdoor scenes is problematic when there are variations in imaging conditions. It is known that albedo (reflectance) is invariant to all kinds of illumination effects. Thus, using reflectance images for semantic segmentation task can be favorable. Additionally, not only segmentation may benefit from reflectance, but also segmentation may be useful for reflectance computation. Therefore, in this paper, the tasks of semantic segmentation and intrinsic image decomposition are considered as a combined process by exploring their mutual relationship in a joint fashion. To that end, we propose a supervised end-to-end CNN architecture to jointly learn intrinsic image decomposition and semantic segmentation. We analyze the gains of addressing those two problems jointly. Moreover, new cascade CNN architectures for intrinsic-for-segmentation and segmentation-for-intrinsic are proposed as single tasks. Furthermore, a dataset of 35K synthetic images of natural environments is created with corresponding albedo and shading (intrinsics), as well as semantic labels (segmentation) assigned to each object/scene. The experiments show that joint learning of intrinsic image decomposition and semantic segmentation is beneficial for both tasks for natural scenes. Dataset and models are available at: https://ivi.fnwi.uva.nl/cv/intrinsegComment: ECCV 201

Single-picture reconstruction and rendering of trees for plausible vegetation synthesis

State-of-the-art approaches for tree reconstruction either put limiting constraints on the input side (requiring multiple photographs, a scanned point cloud or intensive user input) or provide a representation only suitable for front views of the tree. In this paper we present a complete pipeline for synthesizing and rendering detailed trees from a single photograph with minimal user effort. Since the overall shape and appearance of each tree is recovered from a single photograph of the tree crown, artists can benefit from georeferenced images to populate landscapes with native tree species. A key element of our approach is a compact representation of dense tree crowns through a radial distance map. Our first contribution is an automatic algorithm for generating such representations from a single exemplar image of a tree. We create a rough estimate of the crown shape by solving a thin-plate energy minimization problem, and then add detail through a simplified shape-from-shading approach. The use of seamless texture synthesis results in an image-based representation that can be rendered from arbitrary view directions at different levels of detail. Distant trees benefit from an output-sensitive algorithm inspired on relief mapping. For close-up trees we use a billboard cloud where leaflets are distributed inside the crown shape through a space colonization algorithm. In both cases our representation ensures efficient preservation of the crown shape. Major benefits of our approach include: it recovers the overall shape from a single tree image, involves no tree modeling knowledge and minimal authoring effort, and the associated image-based representation is easy to compress and thus suitable for network streaming.Peer ReviewedPostprint (author's final draft

Reconstructing Plants in 3D from a Single Image Using Analysis-by-Synthesis

International audienceMature computer vision techniques allow the reconstruction of challenging 3D objects from images. However, due to high complexity of plant topology, dedicated methods for generating 3D plant models must be devised. We propose to generate a 3D model of a plant, using an analysis-by-synthesis method mixing information from a single image and a priori knowledge of the plant species. First, our dedicated skeletonisation algorithm generates a possible branch- ing structure from the foliage segmentation. Then, a 3D generative model, based on a parametric model of branching systems that takes into ac- count botanical knowledge is built. The resulting skeleton follows the hierarchical organisation of natural branching structures. An instance of a 3D model can be generated. Moreover, varying parameter values of the generative model (main branching structure of the plant and foliage), we produce a series of candidate models. The reconstruction is improved by selecting the model among these proposals based on a matching criterion with the image. Realistic results obtained on di erent species of plants illustrate the performance of the proposed method

An Investigation into Animating Plant Structures within Real-time Constraints

This paper is an analysis of current developments in rendering botanical structures for scientic and entertainment purposes with a focus on visualising growth. The choices of practical investigations produce a novel approach for parallel parsing of difficult bracketed L-Systems, based upon the work of Lipp, Wonka and Wimmer (2010). Alongside this is a general overview of the issues involved when looking at growing systems, technical details involving programming for the Graphics Processing Unit (GPU) and other possible solutions for further work that also could achieve the project's goals

Mesh Generating Plugin for Blender

Tato práce se zabývá automatizovaným generováním modelů stromů v prostředí open-source 3D modelovacího a animačního nástroje Blender. Program byl vytvořen v rozhraní pro programování aplikací Blenderu v jazyce Python. Práce rozebírá nejběžnější techniky generování rostlin a stromů a postup, který byl použit v programu.This work describes automatic generation of trees in open-source 3D graphics application Blender. It also concerns different techniques for generating plants and method.

Simulation levels of detail for plant motion

In this paper we describe a method for simulating motion of realistically complex plants interactively. We use a precomputation stage to generate the plant structure, along with a set of simulation levels of detail. The levels of detail are made by continuously grouping branches starting from the tips of the branches and working toward the trunk. Grouped branches are simulated as single branches that have similar simulation characteristics to the original branches. During run-time, we traverse the plant and determine the allowable error in the simulation of branch motion. This allows us to choose the appropriate simulation level of detail and we provide smooth transitions from level to level. Our level of detail approach affects only the simulation parameters, allowing geometry to be handled independently. Using this method we can significantly improve simulation times for complex trees

Space colonisation based procedural road generation

Dissertação de mestrado em Computer ScienceProcedural generation of content has been studied for quite some time and it is increasingly relevant in scientific areas and in video-game and film industries. Procedural road layout generation has been traditionally approached using L-Systems, with some works exploring alternative avenues. Although originally conceived for biological systems modelling, the adequacy of L-Systems as a base for road generation has been demonstrated in several works. In this context, this work presents an alternative approach for procedural road layout generation that is also inspired by plant generation algorithms: space colonisation. In particular, this work uses the concept of attraction points introduced in space colonisation as its base to produce road layouts, both in urban and inter-city environments. As will be shown, the usage of attraction points provides an intuitive way to parameterise a road layout. The original Space Colonization Algorithm (SCA) generates a tree like structure, but in this work, the extensions made aim to fully generate a inter-connected road network. As most previous methods the method has two phases. A first phase generates what is mostly a tree structure growing from user defined road segments. The second phase performs the inter connectivity among the roads created in the first phase. The original SCA parameters such as the killradius help to control the capillarity of the road layout, the number of attraction points used by each segment will dictate its relevance establishing a road hierarchy naturally dependent on the distribution of the attraction points on the terrain. An angle control allows the creation of grid like or more organic road layouts. The distribution of the attraction points in the terrain can be conditioned by boundary maps, containing parks, sea, rivers, and other forbidden areas. Population density maps can be used to supply an explicit probabilistic distribution to the attraction points. Flow-fields can be used to dictate the flow of the road layout. Elevation maps provide an additional restriction regarding the steepness of the roads. The tests were executed within a graphic toolbox developed simultaneously. The results are exported to a geographical information file format, GeoJSON, and then maps are rendered using a geospatial visualisation and processing framework called Mapnik. For the most part, parameter settings were intuitively reflected on the road layout and this method can be seen as a first step towards fully exploring the usage of attraction points in the context of road layout.Gradualmente a geração procedimental de conteúdo tem-se tornado cada vez mais relevante, sendo maioritariamente aplicada em industrias como a dos vídeo-jogos e cinema. No que toca à geração procedimental de redes de estradas, grande parte das abordagens em torno deste tema são baseadas em L-Systems. Embora a área de aplicação dos L-Systems tenha sido originalmente para produzir modelos de sistemas biológicos, mostrou também ser um algoritmo adequado para a geração procedimental de redes de estradas. Este trabalho apresenta uma abordagem alternativa à geração procedimental de redes de estradas que também é inspirada num algoritmo procedimental de geração de plantas, colonização espacial, utilizando o conceito de pontos de atracão como base para gerar padrões de estradas. Como será demonstrado, a utilização de pontos de atracão fornece uma maneira intuitiva de parametrizar um padrão de estradas desejado. Como a maioria dos trabalhos feitos nesta área, este método tem duas fases. A primeira fase gera uma rede semelhante a uma árvore criada a partir de um ou mais segmentos iniciais da rede determinados pelo utilizador. A segunda fase trata de interligar as estradas geradas na primeira fase. Os parâmetros iniciais do algoritmo de colonização espacial, como o kill radius, ajudam a controlar a capilaridade da rede, os pontos de atracão que influenciam cada segmento irão ditar a sua relevância na rede geral, estabelecendo a noção de hierarquia de estradas, dependendo da distribuição de pontos de atracão no terreno. O controlo do ângulo entre segmentos permite a criação de padrões de estradas tanto em forma de grelha como padrões mais orgânicos. A distribuição dos pontos de atracão no terreno pode ser influenciada por mapas de fronteira, que contem as áreas válidas e/ou inválidas, como parques, mar, rios, e outras áreas proibidas. Mapas de densidade populacional podem ser usados para fornecer uma distribuição probabilística dos pontos de atracão. Campos de forças, podem ser usados para ditar o fluxo da rede de estradas. Mapas de elevação oferecem uma restrição adicional tendo em conta a inclinação das estradas. De um modo geral, as definições de parâmetros refletiram-se de um modo intuitivo nos padrões de redes de estradas gerados, e este trabalho pode ser considerado como um primeiro passo na exploração do conceito de pontos de atracão na área da geração de redes de estradas

EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenes

Multimodal large-scale datasets for outdoor scenes are mostly designed for urban driving problems. The scenes are highly structured and semantically different from scenarios seen in nature-centered scenes such as gardens or parks. To promote machine learning methods for nature-oriented applications, such as agriculture and gardening, we propose the multimodal synthetic dataset for Enclosed garDEN scenes (EDEN). The dataset features more than 300K images captured from more than 100 garden models. Each image is annotated with various low/high-level vision modalities, including semantic segmentation, depth, surface normals, intrinsic colors, and optical flow. Experimental results on the state-of-the-art methods for semantic segmentation and monocular depth prediction, two important tasks in computer vision, show positive impact of pre-training deep networks on our dataset for unstructured natural scenes. The dataset and related materials will be available at https://lhoangan.github.io/eden.Comment: Accepted for publishing at WACV 202