PlaNeRF: SVD Unsupervised 3D Plane Regularization for NeRF Large-Scale Scene Reconstruction

Neural Radiance Fields (NeRF) enable 3D scene reconstruction from 2D images and camera poses for Novel View Synthesis (NVS). Although NeRF can produce photorealistic results, it often suffers from overfitting to training views, leading to poor geometry reconstruction, especially in low-texture areas. This limitation restricts many important applications which require accurate geometry, such as extrapolated NVS, HD mapping and scene editing. To address this limitation, we propose a new method to improve NeRF's 3D structure using only RGB images and semantic maps. Our approach introduces a novel plane regularization based on Singular Value Decomposition (SVD), that does not rely on any geometric prior. In addition, we leverage the Structural Similarity Index Measure (SSIM) in our loss design to properly initialize the volumetric representation of NeRF. Quantitative and qualitative results show that our method outperforms popular regularization approaches in accurate geometry reconstruction for large-scale outdoor scenes and achieves SoTA rendering quality on the KITTI-360 NVS benchmark.Comment: 14 pages, 7 figure

Accuracy analysis of a mobile mapping system for close range photogrammetric projects

[EN] Image-based mapping solutions require accurate exterior orientation parameters independently of the cameras used for a survey. This paper analyses the inclusion of up to two stereo-based geometric constraints in the form of baseline distance and convergence angle between camera axes to boost the integrated sensor orientation performance on outdoor close-range projects. A terrestrial low-cost mobile mapping GNSS/IMU multi-camera system is used to test the performance of the stereo-based geometric constraint on a weak geometric network in a stop-and-go survey. The influence of the number of control points (CPs) is analysed to confirm the performance and usability of the geometric constraints in real live terrestrial projects where far from ideal setups can exist across the survey. Improvements in image residuals up to 9 times and deviation errors better than 1 cm are expected when at least three CPs are incorporated into the adjustmentThe authors gratefully acknowledge the support from the Spanish Ministerio de Economia y Competitividad to the project HAR2014-59873-R. Contributions on direct georeferencing from professors Dr. David Hernandez-Lopez, Dr. Luis Garcia-Asenjo and D. Pascual Garrigues are highly appreciated.Navarro Tarin, S.; Lerma García, JL. (2016). Accuracy analysis of a mobile mapping system for close range photogrammetric projects. Measurement. 93:148-156. https://doi.org/10.1016/j.measurement.2016.07.0301481569

BENDING THE DOMING EFFECT IN STRUCTURE FROM MOTION RECONSTRUCTIONS THROUGH BUNDLE ADJUSTMENT

Structure from Motion techniques provides low-cost and flexible methods that can be adopted in arial surveying to collect topographic data with accurate results. Nevertheless, the so-called "doming effect", due to unfortunate acquisition conditions or unreliable modeling of radial distortion, has been recognized as a critical issue that disrupts the quality of the attained 3D reconstruction. In this paper we propose a novel method, that works effectively in the presence of a nearly flat soil, to tackle a posteriori the doming effect: an automatic ground detection method is used to capture the doming deformation flawing the reconstruction, which in turn is wrapped to the correct geometry by iteratively enforcing a planarity constraint through a Bundle Adjustment framework. Experiments on real word datasets demonstrate promising results

A full photometric and geometric model for attached webcam/matte screen devices

International audienceWe present a thorough photometric and geometric study of the multimedia devices composed of both a matte screen and an attached camera, where it is shown that the light emitted by an image displayed on the monitor can be expressed in closed-form at any point facing the screen, and that the geometric calibration of the camera attached to the screen can be simplified by introducing simple geometric constraints. These theoretical contributions are experimentally validated in a photometric stereo application with extended sources, where a colored scene is reconstructed while watching a collection of graylevel images displayed on the screen, providing a cheap and entertaining way to acquire realistic 3D-representations for, e.g., augmented reality

Robust Temporally Coherent Laplacian Protrusion Segmentation of 3D Articulated Bodies

In motion analysis and understanding it is important to be able to fit a suitable model or structure to the temporal series of observed data, in order to describe motion patterns in a compact way, and to discriminate between them. In an unsupervised context, i.e., no prior model of the moving object(s) is available, such a structure has to be learned from the data in a bottom-up fashion. In recent times, volumetric approaches in which the motion is captured from a number of cameras and a voxel-set representation of the body is built from the camera views, have gained ground due to attractive features such as inherent view-invariance and robustness to occlusions. Automatic, unsupervised segmentation of moving bodies along entire sequences, in a temporally-coherent and robust way, has the potential to provide a means of constructing a bottom-up model of the moving body, and track motion cues that may be later exploited for motion classification. Spectral methods such as locally linear embedding (LLE) can be useful in this context, as they preserve "protrusions", i.e., high-curvature regions of the 3D volume, of articulated shapes, while improving their separation in a lower dimensional space, making them in this way easier to cluster. In this paper we therefore propose a spectral approach to unsupervised and temporally-coherent body-protrusion segmentation along time sequences. Volumetric shapes are clustered in an embedding space, clusters are propagated in time to ensure coherence, and merged or split to accommodate changes in the body's topology. Experiments on both synthetic and real sequences of dense voxel-set data are shown. This supports the ability of the proposed method to cluster body-parts consistently over time in a totally unsupervised fashion, its robustness to sampling density and shape quality, and its potential for bottom-up model constructionComment: 31 pages, 26 figure

Vision technology/algorithms for space robotics applications

The thrust of automation and robotics for space applications has been proposed for increased productivity, improved reliability, increased flexibility, higher safety, and for the performance of automating time-consuming tasks, increasing productivity/performance of crew-accomplished tasks, and performing tasks beyond the capability of the crew. This paper provides a review of efforts currently in progress in the area of robotic vision. Both systems and algorithms are discussed. The evolution of future vision/sensing is projected to include the fusion of multisensors ranging from microwave to optical with multimode capability to include position, attitude, recognition, and motion parameters. The key feature of the overall system design will be small size and weight, fast signal processing, robust algorithms, and accurate parameter determination. These aspects of vision/sensing are also discussed

FROM THE GENERAL DOCUMENTATION OF HADRIAN'S VILLA TO DESIGN ANALYSIS OF COMPLEX CUPOLAS: A PROCEDURAL APPROACH

Abstract. The paper illustrates the progress of Hadrian's Villa digital documentation with special emphasis on a series of modelling issues emerged while studying vaults and cupolas of the site. Together with the more general problem of giving scientific coherence to both active and passive sensor outputs – systematically gathered from 2013 – a methodological problem concerning data interpretation of complex opus caementicium vaults have become dramatically important for the interdisciplinary research team. A methodology for improving the understanding the original shapes of Hadrianic cupolas was designed to provide scholars and professionals operating at the Villa with reliable and easy to use outputs, for interpretation, restoration, maintenance practice. Sensors integration played a fundamental role since allowed researchers a global understanding of intrados and extrados surfaces using reverse modelling applications. Features and 2D primitives extracted from high-resolution models were analysed in order to create flexible procedural models of reconstruction hypothesis/completion of cupolas. Due to the very nature of these shapes (apparently irregular), but with a solid geometric conception, we applied the last achievements of Catmull-Clark bicubic surfaces in combination with Visual Programming Language (VPL).</p