Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds

Shape reconstruction from sparse point clouds/images is a challenging and relevant task required for a va- riety of applications in computer vision and medical image analysis (e.g. surgical navigation, cardiac mo- tion analysis, augmented/virtual reality systems). A subset of such methods, viz. 3D shape reconstruction from 2D contours, is especially relevant for computer-aided diagnosis and intervention applications in- volving meshes derived from multiple 2D image slices, views or projections. We propose a deep learning architecture, coined Mesh Reconstruction Network (MR-Net), which tackles this problem. MR-Net enables accurate 3D mesh reconstruction in real-time despite missing data and with sparse annotations. Using 3D cardiac shape reconstruction from 2D contours defined on short-axis cardiac magnetic resonance image slices as an exemplar, we demonstrate that our approach consistently outperforms state-of-the-art tech- niques for shape reconstruction from unstructured point clouds. Our approach can reconstruct 3D cardiac meshes to within 2.5-mm point-to-point error, concerning the ground-truth data (the original image spa- tial resolution is ∼1 . 8 ×1 . 8 ×10 mm 3 ). We further evaluate the robustness of the proposed approach to incomplete data, and contours estimated using an automatic segmentation algorithm. MR-Net is generic and could reconstruct shapes of other organs, making it compelling as a tool for various applications in medical image analysi

Nonrigid reconstruction of 3D breast surfaces with a low-cost RGBD camera for surgical planning and aesthetic evaluation

Accounting for 26% of all new cancer cases worldwide, breast cancer remains the most common form of cancer in women. Although early breast cancer has a favourable long-term prognosis, roughly a third of patients suffer from a suboptimal aesthetic outcome despite breast conserving cancer treatment. Clinical-quality 3D modelling of the breast surface therefore assumes an increasingly important role in advancing treatment planning, prediction and evaluation of breast cosmesis. Yet, existing 3D torso scanners are expensive and either infrastructure-heavy or subject to motion artefacts. In this paper we employ a single consumer-grade RGBD camera with an ICP-based registration approach to jointly align all points from a sequence of depth images non-rigidly. Subtle body deformation due to postural sway and respiration is successfully mitigated leading to a higher geometric accuracy through regularised locally affine transformations. We present results from 6 clinical cases where our method compares well with the gold standard and outperforms a previous approach. We show that our method produces better reconstructions qualitatively by visual assessment and quantitatively by consistently obtaining lower landmark error scores and yielding more accurate breast volume estimates

High-pressure operando STM studies giving insight in CO oxidation and NO reduction over Pt(110)

AbstractTwo catalytic systems have been studied at high pressures on the Pt(110) surface on an atomic level. The first system was the oxidation of CO by O2 towards CO2. In the framework of the second reaction, namely NO reduction, the effect of room temperature exposure of the surface to NO and H2 was investigated. To study these reaction systems at relevant pressures, the ReactorSTM has been used. This is a unique system which consists of a compact STM in which a flow reactor is integrated. The combined reactor with STM is housed inside a conventional vacuum system to allow for traditional surface science preparation and analysis techniques. The STM images obtained with the ReactorSTM under reaction conditions show the lifting of the (1x2) missing row reconstruction by high-pressure CO exposure. The lifting is followed by the formation of the (1x1) metallic Pt(110) structure for high CO/O2 ratios and a (1x2) lifted-row type surface oxide structure for more O2-rich conditions. The room temperature exposure of Pt(110) to H2 results in the formation of a (1x4) missing-row structure and deeper, nested missing rows. The exposure to high-pressure NO removes these missing-row structures

Combining local-physical and global-statistical models for sequential deformable shape from motion

The final publication is available at link.springer.comIn this paper, we simultaneously estimate camera pose and non-rigid 3D shape from a monocular video, using a sequential solution that combines local and global representations. We model the object as an ensemble of particles, each ruled by the linear equation of the Newton's second law of motion. This dynamic model is incorporated into a bundle adjustment framework, in combination with simple regularization components that ensure temporal and spatial consistency. The resulting approach allows to sequentially estimate shape and camera poses, while progressively learning a global low-rank model of the shape that is fed back into the optimization scheme, introducing thus, global constraints. The overall combination of local (physical) and global (statistical) constraints yields a solution that is both efficient and robust to several artifacts such as noisy and missing data or sudden camera motions, without requiring any training data at all. Validation is done in a variety of real application domains, including articulated and non-rigid motion, both for continuous and discontinuous shapes. Our on-line methodology yields significantly more accurate reconstructions than competing sequential approaches, being even comparable to the more computationally demanding batch methods.Peer ReviewedPostprint (author's final draft

AUTOMATIC FAÇADE SEGMENTATION FOR THERMAL RETROFIT

Abstract. In this paper we present an automated method to derive highly detailed 3D vector models of modern building facades from terrestrial laser scanning data. The developed procedure can be divided into two main steps: firstly the main elements constituting the facade are identified by means of a segmentation process, then the 3D vector model is generated including some priors on architectural scenes. The identification of main facade elements is based on random sampling and detection of planar elements including topology information in the process to reduce under- and over-segmentation problems. Finally, the prevalence of straight lines and orthogonal intersections in the vector model generation phase is exploited to set additional constraints to enforce automated modeling. Contemporary a further classification is performed, enriching the data with semantics by means of a classification tree. The main application field for these vector models is the design of external insulation thermal retrofit. In particular, in this paper we present a possible application for energy efficiency evaluation of buildings by mean of Infrared Thermography data overlaid to the facade model