High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inference

We propose a data-driven method for recovering miss-ing parts of 3D shapes. Our method is based on a new deep learning architecture consisting of two sub-networks: a global structure inference network and a local geometry refinement network. The global structure inference network incorporates a long short-term memorized context fusion module (LSTM-CF) that infers the global structure of the shape based on multi-view depth information provided as part of the input. It also includes a 3D fully convolutional (3DFCN) module that further enriches the global structure representation according to volumetric information in the input. Under the guidance of the global structure network, the local geometry refinement network takes as input lo-cal 3D patches around missing regions, and progressively produces a high-resolution, complete surface through a volumetric encoder-decoder architecture. Our method jointly trains the global structure inference and local geometry refinement networks in an end-to-end manner. We perform qualitative and quantitative evaluations on six object categories, demonstrating that our method outperforms existing state-of-the-art work on shape completion.Comment: 8 pages paper, 11 pages supplementary material, ICCV spotlight pape

Cardiovascular function in ectotherm sauropsids

Although the anatomy of the heart and its outflow tract differs between crocodiles and all other extant ectotherm sauropsids (i.e. Chelonia, Rhynchocephalia, and Squamata), they share one unique feature: All ectotherm sauropsids are able to bypass the pulmonary circulation and to redirect blood volumes returning from the systemic circulation directly back into the systemic circulation. Lepidosauria and Testudines use the interconnected three chambers of their ventricle for pulmonary bypass, and they are also able to bypass the systemic circulation. All ectotherm sauropsids possess two aortae. In crocodiles, they emerge from different ventricles, which is the mechanism that enables a pulmonary bypass. Their aortae are interconnected by two structures: The Foramen Panizzae and the aortic anastomosis. They enable exchange of blood volumes between the two aortae carrying blood from the two ventricles. This thesis evaluates the functional significance of bypass and blood flow through the Foramen Panizzae and the anastomosis from two sides: First, it examines the morphology of bypass related structures in crocodiles and snakes using a variety of histological and bioimaging methods. These methods are also compared regarding their strengths and weaknesses for the examination of the structures of interest in. Second, a computational multi-compartment model was developed to simulate blood volume distribution in dependence from pulmonary bypass and the blood flow through the Foramen Panizzae and the aortic anastomosis in the crocodilian body. Additionally, I examined the impact of these parameters on physiological measurements that are often used in physiological experiments on the cardiovascular function in ectotherm sauropsids. Apart from supporting knowledge on some already examined anatomical details, the present contribution adds information on so far unexamined species and structures, amongst them a description of the cardiac cartilage clasp in Crocodylus niloticus while the function of bypass in snakes remains unclear. The data from histology and bioimaging contributed to the paramaters chosen for the simulation model. The results of the simulation indicate that – apart from its known impact on supplying the lungs and digestive tract – pulmonary bypass reduces blood supply of the anterior and posterior body regions of crocodiles. Blood flow through Foramen Panizzae and aortic anastomosis is from the right into the left aorta without pulmonary bypass and inverts at its onset. As a consequence of vascular arrangement, decreasing blood flow into the pulmonary circulation during pulmonary bypass decreases left-ventricular return and consequently total cardiac stroke volume, with zero left-ventricular return at complete bypass. I conclude that pulmonary bypass must be incomplete or of limited duration. The simulation model shows that blood flows balance within a few heart beats after a change in pulmonary bypass, or Foramen Panizzae/ aortic anastomosis flow. The results support the idea that pulmonary bypass may improve digestive function by supporting gastric acid secretion and buffering the postprandial alkaline tide rather than being an adaptation to extended diving periods in crocodiles. The disadvantageous effects of pulmonary bypass on oxygen supply to the digestive tract may be eliminated by the so far undiscussed course of branches of the right aorta which additionally supply the digestive tract with oxygen-rich blood. Pulmonary bypass might also contribute to thermoregulation

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and treatment of cardiovascular diseases. For example, TAVI is an alternative to AVR for the treatment of severe aortic stenosis and TAFA is widely used for the treatment and cure of atrial fibrillation. In addition, catheter-based IVUS and OCT imaging of coronary arteries provides important information about the coronary lumen, wall and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial for the evaluation and treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation, motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods.We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence it is important to understand the application domain, clinical background, and imaging modality so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on background information of transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area

Automatic segmentation of the lumen of the carotid artery in ultrasound B-mode images

A new algorithm is proposed for the segmentation of the lumen and bifurcation boundaries of the carotid artery in B-mode ultrasound images. It uses the hipoechogenic characteristics of the lumen for the identification of the carotid boundaries and the echogenic characteristics for the identification of the bifurcation boundaries. The image to be segmented is processed with the application of an anisotropic diffusion filter for speckle removal and morphologic operators are employed in the detection of the artery. The obtained information is then used in the definition of two initial contours, one corresponding to the lumen and the other to the bifurcation boundaries, for the posterior application of the Chan-vese level set segmentation model. A set of longitudinal B-mode images of the common carotid artery (CCA) was acquired with a GE Healthcare Vivid-e ultrasound system (GE Healthcare, United Kingdom). All the acquired images include a part of the CCA and of the bifurcation that separates the CCA into the internal and external carotid arteries. In order to achieve the uppermost robustness in the imaging acquisition process, i.e., images with high contrast and low speckle noise, the scanner was adjusted differently for each acquisition and according to the medical exam. The obtained results prove that we were able to successfully apply a carotid segmentation technique based on cervical ultrasonography. The main advantage of the new segmentation method relies on the automatic identification of the carotid lumen, overcoming the limitations of the traditional methods

A study in the topology-aware reconstruction of thin tubular structures

This thesis is dedicated to the 3D reconstruction of thin tubular structures, such as cables or ropes, from a given image sequence. This is known to be a challenging task, mainly because of self-occlusions of the structure and its fine details. This new approach combines image processing tools with physics simulation to faithfully reconstruct jumbled and tangled cables in 3D. This method estimates the topology of the tubular object in the form of a single 1D path and also computes a topology-aware reconstruction of its geometry. This method is evaluated on both, synthetic and real datasets and demonstrate that this method favourably compares to state-of-the-art methods

Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video

Thin structures, such as wire-frame sculptures, fences, cables, power lines, and tree branches, are common in the real world. It is extremely challenging to acquire their 3D digital models using traditional image-based or depth-based reconstruction methods because thin structures often lack distinct point features and have severe self-occlusion. We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera. Specifically, we present a new curve-based approach to estimate accurate camera poses by establishing correspondences between featureless thin objects in the foreground in consecutive video frames, without requiring visual texture in the background scene to lock on. Enabled by this effective curve-based camera pose estimation strategy, we develop an iterative optimization method with tailored measures on geometry, topology as well as self-occlusion handling for reconstructing 3D thin structures. Extensive validations on a variety of thin structures show that our method achieves accurate camera pose estimation and faithful reconstruction of 3D thin structures with complex shape and topology at a level that has not been attained by other existing reconstruction methods.Comment: Accepted by SIGGRAPH 202

{Vid2Curve}: {S}imultaneous Camera Motion Estimation and Thin Structure Reconstruction from an {RGB} Video

Thin structures, such as wire-frame sculptures, fences, cables, power lines, and tree branches, are common in the real world. It is extremely challenging to acquire their 3D digital models using traditional image-based or depth-based reconstruction methods because thin structures often lack distinct point features and have severe self-occlusion. We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera. Specifically, we present a new curve-based approach to estimate accurate camera poses by establishing correspondences between featureless thin objects in the foreground in consecutive video frames, without requiring visual texture in the background scene to lock on. Enabled by this effective curve-based camera pose estimation strategy, we develop an iterative optimization method with tailored measures on geometry, topology as well as self-occlusion handling for reconstructing 3D thin structures. Extensive validations on a variety of thin structures show that our method achieves accurate camera pose estimation and faithful reconstruction of 3D thin structures with complex shape and topology at a level that has not been attained by other existing reconstruction methods