Sparse reconstruction from a limited projection number of the coronary artery tree in X-ray rotational imaging

International audienceThis paper deals with the 3D reconstruction of sparse data in X-ray rotational imaging. Due to the cardiac motion, the number of available projections for this reconstruction is equal to four, which leads to a strongly undersampled reconstruction problem. We address thus this illness problem through a regularized iterative method. The whole algorithm is divided into two steps. Firstly, a minimal path segmentation step extracts artery tree boundaries. Secondly, a MAP reconstruction comparing L0-norm and L1-norm priors is applied on this extracted coronary tree. The reconstruction optimization process relies on a separable paraboloidal (SPS) algorithm. Some preliminary results are provided on simulated rotational angiograms

Theorems and algorithms for multiple view geometry with applications to electron tomography

The thesis considers both theory and algorithms for geometric computer vision. The framework of the work is built around the application of autonomous transmission electron microscope image registration. The theoretical part of the thesis first develops a consistent robust estimator that is evaluated in estimating two view geometry with both affine and projective camera models. The uncertainty of the fundamental matrix is similarly estimated robustly, and the previous observation whether the covariance matrix of the fundamental matrix contains disparity information of the scene is explained and its utilization in matching is discussed. For point tracking purposes, a reliable wavelet-based matching technique and two EM algorithms for the maximum likelihood affine reconstruction under missing data are proposed. The thesis additionally discusses identification of degeneracy as well as affine bundle adjustment. The application part of the thesis considers transmission electron microscope image registration, first with fiducial gold markers and thereafter without markers. Both methods utilize the techniques proposed in the theoretical part of the thesis and, in addition, a graph matching method is proposed for matching gold markers. Conversely, alignment without markers is disposed by tracking interest points of the intensity surface of the images. At the present level of development, the former method is more accurate but the latter is appropriate for situations where fiducial markers cannot be used. Perhaps the most significant result of the thesis is the proposed robust estimator because of consistence proof and its many application areas, which are not limited to the computer vision field. The other algorithms could be found useful in multiple view applications in computer vision that have to deal with uncertainty, matching, tracking, and reconstruction. From the viewpoint of image registration, the thesis further achieved its aims since two accurate image alignment methods are suggested for obtaining the most exact reconstructions in electron tomography.reviewe

Multi-camera reconstruction and rendering for free-viewpoint video

While virtual environments in interactive entertainment become more and more lifelike and sophisticated, traditional media like television and video have not yet embraced the new possibilities provided by the rapidly advancing processing power. In particular, they remain as non-interactive as ever, and do not allow the viewer to change the camera perspective to his liking. The goal of this work is to advance in this direction, and provide essential ingredients for a free-viewpoint video system, where the viewpoint can be chosen interactively during playback. Knowledge of scene geometry is required to synthesize novel views. Therefore, we describe 3D reconstruction methods for two distinct kinds of camera setups. The first one is depth reconstruction for camera arrays with parallel optical axes, the second one surface reconstruction, in the case that the cameras are distributed around the scene. Another vital part of a 3D video system is the interactive rendering from different viewpoints, which has to perform in real-time. We cover this topic in the last part of this thesis.Während die virtuellen Welten in interaktiven Unterhaltungsmedien immer realitätsnäher werden, machen traditionellere Medien wie Fernsehen und Video von den neuen Möglichkeiten der rasant wachsenden Rechenkapazität bisher kaum Gebrauch. Insbesondere mangelt es ihnen immer noch an Interaktivität, und sie erlauben dem Konsumenten nicht, elementare Parameter wie zum Beispiel die Kameraperspektive seinen Wünschen anzupassen. Ziel dieser Arbeit ist es, die Entwicklung in diese Richtung voranzubringen und essentielle Bausteine für ein Videosystem bereitzustellen, bei dem der Blickpunkt während der Wiedergabe jederzeit völlig frei gewählt werden kann. Um neue Ansichten synthetisieren zu können, ist zunächst Kenntnis von der 3D Geometrie der Szene notwendig. Wir entwickeln daher Rekonstruktionsalgorithmen für zwei verschiedene Anordnungen von Kameras. Falls die Kameras eng beieinanderliegen und parallele optische Achsen haben, können lediglich Tiefenkarten geschätzt werden. Sind die Kameras jedoch im einer Halbkugel um die Szene herum montiert, so rekonstruieren wir sogar echte Oberflächengeometrie. Ein weiterer wichtiger Aspekt ist die interaktive Darstellung der Szene aus neuen Blickwinkeln, die wir im letzten Teil der Arbeit in Angriff nehmen

Automatically Recovering Geometry and Texture from Large Sets of Calibrated Images

Three-dimensional models which contain both geometry and texture have numerous applications such as urban planning, physical simulation, and virtual environments. A major focus of computer vision (and recently graphics) research is the automatic recovery of three-dimensional models from two-dimensional images. After many years of research this goal is yet to be achieved. Most practical modeling systems require substantial human input and unlike automatic systems are not scalable. This thesis presents a novel method for automatically recovering dense surface patches using large sets (1000's) of calibrated images taken from arbitrary positions within the scene. Physical instruments, such as Global Positioning System (GPS), inertial sensors, and inclinometers, are used to estimate the position and orientation of each image. Essentially, the problem is to find corresponding points in each of the images. Once a correspondence has been established, calculating its three-dimensional position is simply a matter of geometry. Long baseline images improve the accuracy. Short baseline images and the large number of images greatly simplifies the correspondence problem. The initial stage of the algorithm is completely local and scales linearly with the number of images. Subsequent stages are global in nature, exploit geometric constraints, and scale quadratically with the complexity of the underlying scene. We describe techniques for: 1) detecting and localizing surface patches; 2) refining camera calibration estimates and rejecting false positive surfels; and 3) grouping surface patches into surfaces and growing the surface along a two-dimensional manifold. We also discuss a method for producing high quality, textured three-dimensional models from these surfaces. Some of the most important characteristics of this approach are that it: 1) uses and refines noisy calibration estimates; 2) compensates for large variations in illumination; 3) tolerates significant soft occlusion (e.g. tree branches); and 4) associates, at a fundamental level, an estimated normal (i.e. no frontal-planar assumption) and texture with each surface patch

The feasibility of high resolution, three-dimensional reconstruction of metal-coated surfaces in structural biology

>Magister Scientiae - MScLife is an emergent property of a complex network of interacting cellular-machines. Three-dimensional (3D), cellular structure captured at supra-atomic resolution has the potential to revolutionise our understanding of the interactions, dynamics and structure of these machines: proteins, organelles and other cellular constituents, in their normal functional states. Techniques, capable of acquiring 3D cellular structure at sufficient resolution to enable identification and interpretation of individual macromolecules in the cellular milieu, have the potential to provide this data. Advances in cryo-preservation, preparation and metal-coating techniques allow images of the surfaces of in situ macromolecules to be obtained in a life-like state by field emission scanning – and transmission electron microscopy (FE/SEM, FE/TEM) at a resolution of 2-4 nm. A large body of macromolecular structural information has been obtained using these techniques, but while the images produced provide a qualitative impression of three-dimensionality, computational methods are required to extract quantitative 3D structure. In order to test the feasibility of applying various photogrammetric and tomographic algorithms to micrographs of well-preserved metal-coated biological surfaces, several algorithms were attempted on a variety of FE/SEM and TEM micrographs. A stereoscopic algorithm was implemented and applied to FESEM stereo images of the nuclear pore basket, resulting in a high quality digital elevation map. A SEM rotation series of an object of complicated topology (ant) was reconstructed volumetrically by silhouette-intersection. Finally, the iterative helical real-space reconstruction technique as applied to cryo-TEM micrographs of unidirectionally heavy-metal shadowed. These preliminary results confirm that 3D information obtained from multiple TEM or SEM surface images could be applied to the problem of 3D macromolecular imaging in the cellular context. However, each of the various methods described here comes with peculiar topological, resolution and geometrical limitations, some of which are inherent shortcomings of the methodologies described; others might be overcome with improved algorithms. Combined with carefully designed surface experiments, some of the methods investigated here could provide novel insights and extend current surface-imaging studies. Docking of atomic resolution structures into low-resolution maps derived from surface imaging experiments is a particularly exciting prospect