3D Reconstruction of Neural Circuits from Serial EM Images

A basic requirement for reconstructing and understanding complete circuit diagrams of neuronal processing units is the availability of electron microscopic 3D data sets of large ensembles of neurons. A recently developed technique, "Serial Block Face Scanning Electron Microscopy" (SBFSEM, Denk and Horstmann 2004) allows automatic sectioning and imaging of biological tissue inside the vacuum chamber of a scanning electron microscope. Image stacks generated with this technology have a resolution sucient to distinguish different cellular compartments, including synaptic structures. Such an image stack contains thousands of images and is recorded with a voxel size of 23 nm in the x- and y-directions and 30 nm in the z-direction. Consequently a tissue block of 1 mm3 produces 63 terabytes of data. Therefore new concepts for managing large data sets and automated image processing are required. I developed an image segmentation and 3D reconstruction software, which allows precise contour tracing of cell membranes and simultaneously displays the resulting 3D structure. The software contains two stand-alone packages: Neuron2D and Neuron3D, both oering an easy-to-operate graphical user interface (GUI). The software package Neuron2D provides the following image processing functions: • Image Registration: Combination of multiple SBFSEM image tiles. • Image Preprocessing: Filtering of image stacks. Implemented are Gaussian and Non-Linear-Diusion lters in 2D and 3D. This step enhances the contrast between contour lines and image background, leading to a higher signal-to-noise ratio, thus further improving detection of membrane borders. • Image Segmentation: The implemented algorithms extract contour lines from the preceding image and automatically trace the contour lines in the following images (z-direction), taking into account the previous image segmentation. They also permit image segmentation starting at any position in the image stack. In addition, manual interaction is possible. To visualize 3D structures of neuronal circuits the additional software Neuron3D was developed. The program relies on the contour line information provided by Neuron2D to implement a surface reconstruction algorithm based on dynamic time warping. Additional rendering techniques, such as shading and texture mapping, are provided. The detailed anatomical reconstruction provides a framework for computational models of neuronal circuits. For example in ies, where moving retinal images lead to appropriate course control signals, the circuit reconstruction of motion-sensitive neurons can help to further understand the neural processing of visual motion in ies

Analysis and Parameterization of Triangulated Surfaces

This dissertation deals with the analysis and parameterization of surfaces represented by triangle meshes, that is, piecewise linear surfaces which enable a simple representation of 3D models commonly used in mathematics and computer science. Providing equivalent and high-level representations of a 3D triangle mesh M is of basic importance for approaching different computational problems and applications in the research fields of Computational Geometry, Computer Graphics, Geometry Processing, and Shape Modeling. The aim of the thesis is to show how high-level representations of a given surface M can be used to find other high-level or equivalent descriptions of M and vice versa. Furthermore, this analysis is related to the study of local and global properties of triangle meshes depending on the information that we want to capture and needed by the application context. The local analysis of an arbitrary triangle mesh M is based on a multi-scale segmentation of M together with the induced local parameterization, where we replace the common hypothesis of decomposing M into a family of disc-like patches (i.e., 0-genus and one boundary component) with a feature-based segmentation of M into regions of 0-genus without constraining the number of boundary components of each patch. This choice and extension is motivated by the necessity of identifying surface patches with features, of reducing the parameterization distortion, and of better supporting standard applications of the parameterization such as remeshing or more generally surface approximation, texture mapping, and compression. The global analysis, characterization, and abstraction of M take into account its topological and geometric aspects represented by the combinatorial structure of M (i.e., the mesh connectivity) with the associated embedding in R^3. Duality and dual Laplacian smoothing are the first characterizations of M presented with the final aim of a better understanding of the relations between mesh connectivity and geometry, as discussed by several works in this research area, and extended in the thesis to the case of 3D parameterization. The global analysis of M has been also approached by defining a real function on M which induces a Reeb graph invariant with respect to affine transformations and best suited for applications such as shape matching and comparison. Morse theory and the Reeb graph were also used for supporting a new and simple method for solving the global parameterization problem, that is, the search of a cut graph of an arbitrary triangle mesh M. The main characteristics of the proposed approach with respect to previous work are its capability of defining a family of cut graphs, instead of just one cut, of bordered and closed surfaces which are treated with a unique approach. Furthermore, each cut graph is smooth and the way it is built is based on the cutting procedure of 0-genus surfaces that was used for the local parameterization of M. As discussed in the thesis, defining a family of cut graphs provides a great flexibility and effective simplifications of the analysis, modeling, and visualization of (time-depending) scalar and vector fields; in fact, the global parameterization of M enables to reduce th

Contours in Visualization

This thesis studies the visualization of set collections either via or defines as the relations among contours. In the first part, dynamic Euler diagrams are used to communicate and improve semimanually the result of clustering methods which allow clusters to overlap arbitrarily. The contours of the Euler diagram are rendered as implicit surfaces called blobs in computer graphics. The interaction metaphor is the moving of items into or out of these blobs. The utility of the method is demonstrated on data arising from the analysis of gene expressions. The method works well for small datasets of up to one hundred items and few clusters. In the second part, these limitations are mitigated employing a GPU-based rendering of Euler diagrams and mixing textures and colors to resolve overlapping regions better. The GPU-based approach subdivides the screen into triangles on which it performs a contour interpolation, i.e. a fragment shader determines for each pixel which zones of an Euler diagram it belongs to. The rendering speed is thus increased to allow multiple hundred items. The method is applied to an example comparing different document clustering results. The contour tree compactly describes scalar field topology. From the viewpoint of graph drawing, it is a tree with attributes at vertices and optionally on edges. Standard tree drawing algorithms emphasize structural properties of the tree and neglect the attributes. Adapting popular graph drawing approaches to the problem of contour tree drawing it is found that they are unable to convey this information. Five aesthetic criteria for drawing contour trees are proposed and a novel algorithm for drawing contour trees in the plane that satisfies four of these criteria is presented. The implementation is fast and effective for contour tree sizes usually used in interactive systems and also produces readable pictures for larger trees. Dynamical models that explain the formation of spatial structures of RNA molecules have reached a complexity that requires novel visualization methods to analyze these model\''s validity. The fourth part of the thesis focuses on the visualization of so-called folding landscapes of a growing RNA molecule. Folding landscapes describe the energy of a molecule as a function of its spatial configuration; they are huge and high dimensional. Their most salient features are described by their so-called barrier tree -- a contour tree for discrete observation spaces. The changing folding landscapes of a growing RNA chain are visualized as an animation of the corresponding barrier tree sequence. The animation is created as an adaption of the foresight layout with tolerance algorithm for dynamic graph layout. The adaptation requires changes to the concept of supergraph and it layout. The thesis finishes with some thoughts on how these approaches can be combined and how the task the application should support can help inform the choice of visualization modality

Courbure discrète : théorie et applications

International audienceThe present volume contains the proceedings of the 2013 Meeting on discrete curvature, held at CIRM, Luminy, France. The aim of this meeting was to bring together researchers from various backgrounds, ranging from mathematics to computer science, with a focus on both theory and applications. With 27 invited talks and 8 posters, the conference attracted 70 researchers from all over the world. The challenge of finding a common ground on the topic of discrete curvature was met with success, and these proceedings are a testimony of this wor

ICASE/LaRC Workshop on Adaptive Grid Methods

Solution-adaptive grid techniques are essential to the attainment of practical, user friendly, computational fluid dynamics (CFD) applications. In this three-day workshop, experts gathered together to describe state-of-the-art methods in solution-adaptive grid refinement, analysis, and implementation; to assess the current practice; and to discuss future needs and directions for research. This was accomplished through a series of invited and contributed papers. The workshop focused on a set of two-dimensional test cases designed by the organizers to aid in assessing the current state of development of adaptive grid technology. In addition, a panel of experts from universities, industry, and government research laboratories discussed their views of needs and future directions in this field

Robust surface modelling of visual hull from multiple silhouettes

Reconstructing depth information from images is one of the actively researched themes in computer vision and its application involves most vision research areas from object recognition to realistic visualisation. Amongst other useful vision-based reconstruction techniques, this thesis extensively investigates the visual hull (VH) concept for volume approximation and its robust surface modelling when various views of an object are available. Assuming that multiple images are captured from a circular motion, projection matrices are generally parameterised in terms of a rotation angle from a reference position in order to facilitate the multi-camera calibration. However, this assumption is often violated in practice, i.e., a pure rotation in a planar motion with accurate rotation angle is hardly realisable. To address this problem, at first, this thesis proposes a calibration method associated with the approximate circular motion. With these modified projection matrices, a resulting VH is represented by a hierarchical tree structure of voxels from which surfaces are extracted by the Marching cubes (MC) algorithm. However, the surfaces may have unexpected artefacts caused by a coarser volume reconstruction, the topological ambiguity of the MC algorithm, and imperfect image processing or calibration result. To avoid this sensitivity, this thesis proposes a robust surface construction algorithm which initially classifies local convex regions from imperfect MC vertices and then aggregates local surfaces constructed by the 3D convex hull algorithm. Furthermore, this thesis also explores the use of wide baseline images to refine a coarse VH using an affine invariant region descriptor. This improves the quality of VH when a small number of initial views is given. In conclusion, the proposed methods achieve a 3D model with enhanced accuracy. Also, robust surface modelling is retained when silhouette images are degraded by practical noise

Robust surface modelling of visual hull from multiple silhouettes

Reconstructing depth information from images is one of the actively researched themes in computer vision and its application involves most vision research areas from object recognition to realistic visualisation. Amongst other useful vision-based reconstruction techniques, this thesis extensively investigates the visual hull (VH) concept for volume approximation and its robust surface modelling when various views of an object are available. Assuming that multiple images are captured from a circular motion, projection matrices are generally parameterised in terms of a rotation angle from a reference position in order to facilitate the multi-camera calibration. However, this assumption is often violated in practice, i.e., a pure rotation in a planar motion with accurate rotation angle is hardly realisable. To address this problem, at first, this thesis proposes a calibration method associated with the approximate circular motion. With these modified projection matrices, a resulting VH is represented by a hierarchical tree structure of voxels from which surfaces are extracted by the Marching cubes (MC) algorithm. However, the surfaces may have unexpected artefacts caused by a coarser volume reconstruction, the topological ambiguity of the MC algorithm, and imperfect image processing or calibration result. To avoid this sensitivity, this thesis proposes a robust surface construction algorithm which initially classifies local convex regions from imperfect MC vertices and then aggregates local surfaces constructed by the 3D convex hull algorithm. Furthermore, this thesis also explores the use of wide baseline images to refine a coarse VH using an affine invariant region descriptor. This improves the quality of VH when a small number of initial views is given. In conclusion, the proposed methods achieve a 3D model with enhanced accuracy. Also, robust surface modelling is retained when silhouette images are degraded by practical noise

Computer vision

The field of computer vision is surveyed and assessed, key research issues are identified, and possibilities for a future vision system are discussed. The problems of descriptions of two and three dimensional worlds are discussed. The representation of such features as texture, edges, curves, and corners are detailed. Recognition methods are described in which cross correlation coefficients are maximized or numerical values for a set of features are measured. Object tracking is discussed in terms of the robust matching algorithms that must be devised. Stereo vision, camera control and calibration, and the hardware and systems architecture are discussed