Single-picture reconstruction and rendering of trees for plausible vegetation synthesis

State-of-the-art approaches for tree reconstruction either put limiting constraints on the input side (requiring multiple photographs, a scanned point cloud or intensive user input) or provide a representation only suitable for front views of the tree. In this paper we present a complete pipeline for synthesizing and rendering detailed trees from a single photograph with minimal user effort. Since the overall shape and appearance of each tree is recovered from a single photograph of the tree crown, artists can benefit from georeferenced images to populate landscapes with native tree species. A key element of our approach is a compact representation of dense tree crowns through a radial distance map. Our first contribution is an automatic algorithm for generating such representations from a single exemplar image of a tree. We create a rough estimate of the crown shape by solving a thin-plate energy minimization problem, and then add detail through a simplified shape-from-shading approach. The use of seamless texture synthesis results in an image-based representation that can be rendered from arbitrary view directions at different levels of detail. Distant trees benefit from an output-sensitive algorithm inspired on relief mapping. For close-up trees we use a billboard cloud where leaflets are distributed inside the crown shape through a space colonization algorithm. In both cases our representation ensures efficient preservation of the crown shape. Major benefits of our approach include: it recovers the overall shape from a single tree image, involves no tree modeling knowledge and minimal authoring effort, and the associated image-based representation is easy to compress and thus suitable for network streaming.Peer ReviewedPostprint (author's final draft

Large-Scale Automatic Reconstruction of Neuronal Processes from Electron Microscopy Images

Automated sample preparation and electron microscopy enables acquisition of very large image data sets. These technical advances are of special importance to the field of neuroanatomy, as 3D reconstructions of neuronal processes at the nm scale can provide new insight into the fine grained structure of the brain. Segmentation of large-scale electron microscopy data is the main bottleneck in the analysis of these data sets. In this paper we present a pipeline that provides state-of-the art reconstruction performance while scaling to data sets in the GB-TB range. First, we train a random forest classifier on interactive sparse user annotations. The classifier output is combined with an anisotropic smoothing prior in a Conditional Random Field framework to generate multiple segmentation hypotheses per image. These segmentations are then combined into geometrically consistent 3D objects by segmentation fusion. We provide qualitative and quantitative evaluation of the automatic segmentation and demonstrate large-scale 3D reconstructions of neuronal processes from a $\mathbf{27,000}$ $\mathbf{\mu m^3}$ volume of brain tissue over a cube of $\mathbf{30 \; \mu m}$ in each dimension corresponding to 1000 consecutive image sections. We also introduce Mojo, a proofreading tool including semi-automated correction of merge errors based on sparse user scribbles

Mapping Forest Regeneration from Terrestrial Laser Scans

Az erdei újulati foltok helye, kiterjedése, borítottsága és törzsszáma kulcsfontosságú tényezők az erdődinamikai folyamatok feltárásában és a többkorú faállományok kezelésében. A fatermési modellek előállítása, az üzemi gyakorlatban végzett erdőművelés valamint erdőfeltárás pontos és objektív módszereket kíván az újulat helyének meghatározására. A földi lézeres letapogatás kiválóan alkalmas törzstérképek előállítására, ám az adatok feldolgozásához szükséges eljárásokat eddig csak szálerdőkre fejlesztettek ki. A tanulmány olyan automatikus eljárást mutat be, ami 3–6 méter magasságú faegyedek lézeres letapogatás adataiból történő azonosítását teszi lehetővé. Három, különböző jellegű újulati foltban létesített mintaterületen a ponthalmaz vizuális interpretációjával azonosított törzsek 79–90%-át sikerült automatikus úton felismerni. Az eljárás teljesítményét a vizsgált állományjellemzők közül elsősorban a törzsszám befolyásolta, míg az ágak mennyiségének hatása elenyésző. Az elért eredmények rámutatnak, hogy a földi lézeres letapogatás alkalmas az újulat mennyiségének felmérésére, így a folyamatos borítású erdők leírásának ígéretes eszköze lehet

Semi-Automated Reconstruction of Neural Processes from Large Numbers of Fluorescence Images

We introduce a method for large scale reconstruction of complex bundles of neural processes from fluorescent image stacks. We imaged yellow fluorescent protein labeled axons that innervated a whole muscle, as well as dendrites in cerebral cortex, in transgenic mice, at the diffraction limit with a confocal microscope. Each image stack was digitally re-sampled along an orientation such that the majority of axons appeared in cross-section. A region growing algorithm was implemented in the open-source Reconstruct software and applied to the semi-automatic tracing of individual axons in three dimensions. The progression of region growing is constrained by user-specified criteria based on pixel values and object sizes, and the user has full control over the segmentation process. A full montage of reconstructed axons was assembled from the ∼200 individually reconstructed stacks. Average reconstruction speed is ∼0.5 mm per hour. We found an error rate in the automatic tracing mode of ∼1 error per 250 um of axonal length. We demonstrated the capacity of the program by reconstructing the connectome of motor axons in a small mouse muscle

Extraction of Unfoliaged Trees from Terrestrial Image Sequences

This thesis presents a generative statistical approach for the fully automatic three-dimensional (3D) extraction and reconstruction of unfoliaged deciduous trees from wide-baseline image sequences. Tree models improve the realism of 3D Geoinformation systems (GIS) by adding a natural touch. Unfoliaged trees are, however, difficult to reconstruct from images due to partially weak contrast, background clutter, occlusions, and particularly the possibly varying order of branches in images from different viewpoints. The proposed approach combines generative modeling by L-systems and statistical maximum a posteriori (MAP) estimation for the extraction of the 3D branching structure of trees. Background estimation is conducted by means of mathematical (gray scale) morphology as basis for generative modeling. A Gaussian likelihood function based on intensity differences is employed to evaluate the hypotheses. A mechanism has been devised to control the sampling sequence of multiple parameters in the Markov Chain considering their characteristics and the performance in the previous step. A tree is classified into three typical branching types after the extraction of the first level of branches and more specific Production Rules of L-systems are used accordingly. Generic prior distributions for parameters are refined based on already extracted branches in a Bayesian framework and integrated into the MAP estimation. By these means most of the branching structure besides tiny twigs can be reconstructed. Results are presented in the form of VRML (Virtual Reality Modeling Language) models demonstrating the potential of the approach as well as its current shortcomings.Diese Dissertationsschrift stellt einen generativen statistischen Ansatz für die vollautomatische drei-dimensionale (3D) Extraktion und Rekonstruktion unbelaubter Laubbäume aus Bildsequenzen mit großer Basis vor. Modelle für Bäume verbessern den Realismus von 3D Geoinformationssystemen (GIS), indem sie Letzteren eine natürliche Note geben. Wegen z.T. schwachem Kontrast, Störobjekten im Hintergrund, Verdeckungen und insbesondere der möglicherweise unterschiedlichen Ordnung der Äste in Bildern von verschiedenen Blickpunkten sind unbelaubte Bäume aber schwierig zu rekonstruieren. Der vorliegende Ansatz kombiniert generative Modellierung mittels L-Systemen und statistische Maximum A Posteriori (MAP) Schätzung für die Extraktion der 3D Verzweigungsstruktur von Bäumen. Hintergrund-Schätzung wird auf Grundlage von mathematischer (Grauwert) Morphologie als Basis für die generative Modellierung durchgeführt. Für die Bewertung der Hypothesen wird eine Gaußsche Likelihood-Funktion basierend auf Intensitätsunterschieden benutzt. Es wurde ein Mechanismus entworfen, der die Reihenfolge der Verwendung mehrerer Parameter für die Markoff-Kette basierend auf deren Charakteristik und Performance im letzten Schritt kontrolliert. Ein Baum wird nach der Extraktion der ersten Stufe von Ästen in drei typische Verzweigungstypen klassifiziert und es werden entsprechend Produktionsregeln von spezifischen L-Systemen verwendet. Basierend auf bereits extrahierten Ästen werden generische Prior-Verteilungen für die Parameter in einem Bayes’schen Rahmen verfeinert und in die MAP Schätzung integriert. Damit kann ein großer Teil der Verzweigungsstruktur außer kleinen Ästen extrahiert werden. Die Ergebnisse werden als VRML (Virtual Reality Modeling Language) Modelle dargestellt. Sie zeigen das Potenzial aber auch die noch vorhandenen Defizite des Ansatzes

Reconstructing Plants in 3D from a Single Image Using Analysis-by-Synthesis

International audienceMature computer vision techniques allow the reconstruction of challenging 3D objects from images. However, due to high complexity of plant topology, dedicated methods for generating 3D plant models must be devised. We propose to generate a 3D model of a plant, using an analysis-by-synthesis method mixing information from a single image and a priori knowledge of the plant species. First, our dedicated skeletonisation algorithm generates a possible branch- ing structure from the foliage segmentation. Then, a 3D generative model, based on a parametric model of branching systems that takes into ac- count botanical knowledge is built. The resulting skeleton follows the hierarchical organisation of natural branching structures. An instance of a 3D model can be generated. Moreover, varying parameter values of the generative model (main branching structure of the plant and foliage), we produce a series of candidate models. The reconstruction is improved by selecting the model among these proposals based on a matching criterion with the image. Realistic results obtained on di erent species of plants illustrate the performance of the proposed method

Accurate geometry modeling of vasculatures using implicit fitting with 2D radial basis functions

Accurate vascular geometry modeling is an essential task in computer assisted vascular surgery and therapy. This paper presents a vessel cross-section based implicit vascular modeling technique, which represents a vascular surface as a set of locally fitted implicit surfaces. In the proposed method, a cross-section based technique is employed to extract from each cross-section of the vascular surface a set of points, which are then fitted with an implicit curve represented as 2D radial basis functions. All these implicitly represented cross-section curves are then being considered as 3D cylindrical objects and combined together using a certain partial shape-preserving spline to build a complete vessel branch; different vessel branches are then blended using a extended smooth maximum function to construct the complete vascular tree. Experimental results show that the proposed method can correctly represent the morphology and topology of vascular structures with high level of smoothness. Both qualitative comparison with other methods and quantitative validations to the proposed method have been performed to verify the accuracy and smoothness of the generated vascular geometric models