Systematizing Virtual Reconstructionof Lost or Never Built Architectures

European Architectural Cultural Heritage is immense. Yet part of this Heritage is invisible: churches, synagogues, mosques that have either been destroyed or never been built. Now the digital world offers the possibility to bring these artefacts to a new life, through 3D reconstruction. This way of studying and representing the past has become increasingly important in the academic world and the domain of digital entertainment. These applications make use of the so-called ‘virtual 3D reconstructions’, which are 3D models based on figurative/textual sources or ruins of artefacts that no longer exist or have never been built.This paper aims to present ‘CoVHer’ (Computer-based Visualisation of Architectural Cultural Heritage), an Erasmus Plus Project that deals with this vast theme and involves five universities and two private companies from five European countries (Italy, Spain, Portugal, Poland and Germany). The main objective of CoVHer is to define applicable/practice guidelines and operational methodologies aimed at the study, implementation, visualization and critical evaluation of the 3D models. Some of the ongoing theoretical studies developed in the project will be presented. In particular, this paper will focus on the systematization of the reconstruction process. It defines and classifies different aspects of 3D digital modelling; and other aspects concerning visualization in the field of architectural hypothetical reconstruction

Differentiable Surface Triangulation

Triangle meshes remain the most popular data representation for surface geometry. This ubiquitous representation is essentially a hybrid one that decouples continuous vertex locations from the discrete topological triangulation. Unfortunately, the combinatorial nature of the triangulation prevents taking derivatives over the space of possible meshings of any given surface. As a result, to date, mesh processing and optimization techniques have been unable to truly take advantage of modular gradient descent components of modern optimization frameworks. In this work, we present a differentiable surface triangulation that enables optimization for any per-vertex or per-face differentiable objective function over the space of underlying surface triangulations. Our method builds on the result that any 2D triangulation can be achieved by a suitably perturbed weighted Delaunay triangulation. We translate this result into a computational algorithm by proposing a soft relaxation of the classical weighted Delaunay triangulation and optimizing over vertex weights and vertex locations. We extend the algorithm to 3D by decomposing shapes into developable sets and differentiably meshing each set with suitable boundary constraints. We demonstrate the efficacy of our method on various planar and surface meshes on a range of difficult-to-optimize objective functions. Our code can be found online: https://github.com/mrakotosaon/diff-surface-triangulation

Multisource Point Clouds, Point Simplification and Surface Reconstruction

As data acquisition technology continues to advance, the improvement and upgrade of the algorithms for surface reconstruction are required. In this paper, we utilized multiple terrestrial Light Detection And Ranging (Lidar) systems to acquire point clouds with different levels of complexity, namely dynamic and rigid targets for surface reconstruction. We propose a robust and effective method to obtain simplified and uniform resample points for surface reconstruction. The method was evaluated. A point reduction of up to 99.371% with a standard deviation of 0.2 cm was achieved. In addition, well-known surface reconstruction methods, i.e., Alpha shapes, Screened Poisson reconstruction (SPR), the Crust, and Algebraic point set surfaces (APSS Marching Cubes), were utilized for object reconstruction. We evaluated the benefits in exploiting simplified and uniform points, as well as different density points, for surface reconstruction. These reconstruction methods and their capacities in handling data imperfections were analyzed and discussed. The findings are that (i) the capacity of surface reconstruction in dealing with diverse objects needs to be improved; (ii) when the number of points reaches the level of millions (e.g., approximately five million points in our data), point simplification is necessary, as otherwise, the reconstruction methods might fail; (iii) for some reconstruction methods, the number of input points is proportional to the number of output meshes; but a few methods are in the opposite; (iv) all reconstruction methods are beneficial from the reduction of running time; and (v) a balance between the geometric details and the level of smoothing is needed. Some methods produce detailed and accurate geometry, but their capacity to deal with data imperfection is poor, while some other methods exhibit the opposite characteristics

Impacts of surface model generation approaches on raytracing-based solar potential estimation in urban areas

Raytracing-based methods are widely used for quantifying irradiation on building surfaces. Urban 3D surface models are necessary input for raytracing simulations, which can be generated from open-source point cloud data with the help of surface reconstruction algorithms. In research and engineering practice, various algorithms are being used for this purpose; each leading to different mesh topologies and corresponding performance. This paper compares the impacts of four different reconstruction algorithms by investigating their performance using DAYSIM raytracing simulations. The analysis is carried out for five configurations with various urban morphologies. Results show that the reconstructed models consistently underestimate the shading influence due to geometrical shrinkages that emerge from the various model generation procedures. The explicit algorithms, with Generic Delaunay a notable example, have better performance with less embedded error than the implicit algorithms in both daily and annual simulations. Results also show that diffuse irradiance is responsible for larger contributions to the overall error than direct components. This effect becomes more prominent when modeling reflected irradiation in urban environments. Additionally, the work shows that solar elevation and shading geometry types also affect the error magnitude. The paper concludes by providing reconstruction algorithm selection criteria for photovoltaic practitioners and urban energy planners

3D Reconstruction Using High Resolution Implicit Surface Representations and Memory Management Strategies

La disponibilité de capteurs de numérisation 3D rapides et précis a permis de capturer de très grands ensembles de points à la surface de différents objets qui véhiculent la géométrie des objets. La métrologie appliquée consiste en l'application de mesures dans différents domaines tels que le contrôle qualité, l'inspection, la conception de produits et la rétroingénierie. Une fois que le nuage de points 3D non organisés couvrant toute la surface de l'objet a été capturé, un modèle de la surface doit être construit si des mesures métrologiques doivent être effectuées sur l'objet. Dans la reconstruction 3D en temps réel, à l'aide de scanners 3D portables, une représentation de surface implicite très efficace est le cadre de champ vectoriel, qui suppose que la surface est approchée par un plan dans chaque voxel. Le champ vectoriel contient la normale à la surface et la matrice de covariance des points tombant à l'intérieur d'un voxel. L'approche globale proposée dans ce projet est basée sur le cadre Vector Field. Le principal problème abordé dans ce projet est la résolution de l'incrément de consommation de mémoire et la précision du modèle reconstruit dans le champ vectoriel. Ce tte approche effectue une sélection objective de la taille optimale des voxels dans le cadre de champ vectoriel pour maintenir la consommation de mémoire aussi faible que possible et toujours obtenir un modèle précis de la surface. De plus, un ajustement d e surface d'ordre élevé est utilisé pour augmenter la précision du modèle. Étant donné que notre approche ne nécessite aucune paramétrisation ni calcul complexe, et qu'au lieu de travailler avec chaque point, nous travaillons avec des voxels dans le champ vectoriel, cela réduit la complexité du calcul.The availability of fast and accurate 3D scanning sensors has made it possible to capture very large sets of points at the surface of different objects that convey the geometry of the objects. A pplied metrology consists in the application of measurements in different fields such as quality control, inspection, product design and reverse engineering. Once the cloud of unorganized 3D points covering the entire surface of the object has been capture d, a model of the surface must be built if metrologic measurements are to be performed on the object. In realtime 3D reconstruction, using handheld 3D scanners a very efficient implicit surface representation is the Vector Field framework, which assumes that the surface is approximated by a plane in each voxel. The vector field contains the normal to the surface and the covariance matrix of the points falling inside a voxel. The proposed global approach in this project is based on the Vector Field framew ork. The main problem addressed in this project is solving the memory consumption increment and the accuracy of the reconstructed model in the vector field. This approach performs an objective selection of the optimal voxels size in the vector field frame work to keep the memory consumption as low as possible and still achieve an accurate model of the surface. Moreover, a highorder surface fitting is used to increase the accuracy of the model. Since our approach do not require any parametrization and compl ex calculation, and instead of working with each point we are working with voxels in the vector field, then it reduces the computational complexity

How to build a 2d and 3d aerial multispectral map?—all steps deeply explained

UIDB/04111/2020 PCIF/SSI/0102/2017 IF/00325/2015 UIDB/00066/2020The increased development of camera resolution, processing power, and aerial platforms helped to create more cost-efficient approaches to capture and generate point clouds to assist in scientific fields. The continuous development of methods to produce three-dimensional models based on two-dimensional images such as Structure from Motion (SfM) and Multi-View Stereopsis (MVS) allowed to improve the resolution of the produced models by a significant amount. By taking inspiration from the free and accessible workflow made available by OpenDroneMap, a detailed analysis of the processes is displayed in this paper. As of the writing of this paper, no literature was found that described in detail the necessary steps and processes that would allow the creation of digital models in two or three dimensions based on aerial images. With this, and based on the workflow of OpenDroneMap, a detailed study was performed. The digital model reconstruction process takes the initial aerial images obtained from the field survey and passes them through a series of stages. From each stage, a product is acquired and used for the following stage, for example, at the end of the initial stage a sparse reconstruction is produced, obtained by extracting features of the images and matching them, which is used in the following step, to increase its resolution. Additionally, from the analysis of the workflow, adaptations were made to the standard workflow in order to increase the compatibility of the developed system to different types of image sets. Particularly, adaptations focused on thermal imagery were made. Due to the low presence of strong features and therefore difficulty to match features across thermal images, a modification was implemented, so thermal models could be produced alongside the already implemented processes for multispectral and RGB image sets.publishersversionpublishe

Digital Multispectral Map Reconstruction Using Aerial Imagery

Advances made in the computer vision field allowed for the establishment of faster and more accurate photogrammetry techniques. Structure from Motion(SfM) is a photogrammetric technique focused on the digital spatial reconstruction of objects based on a sequence of images. The benefit of Unmanned Aerial Vehicle (UAV) platforms allowed the ability to acquire high fidelity imagery intended for environmental mapping. This way, UAV platforms became a heavily adopted method of survey. The combination of SfM and the recent improvements of Unmanned Aerial Vehicle (UAV) platforms granted greater flexibility and applicability, opening a new path for a new remote sensing technique aimed to replace more traditional and laborious approaches often associated with high monetary costs. The continued development of digital reconstruction software and advances in the field of computer processing allowed for a more affordable and higher resolution solution when compared to the traditional methods. The present work proposed a digital reconstruction algorithm based on images taken by a UAV platform inspired by the work made available by the open-source project OpenDroneMap. The aerial images are inserted in the computer vision program and several operations are applied to them, including detection and matching of features, point cloud reconstruction, meshing, and texturing, which results in a final product that represents the surveyed site. Additionally, from the study, it was concluded that an implementation which addresses the processing of thermal images was not integrated in the works of OpenDroneMap. By this point, their work was altered to allow for the reconstruction of thermal maps without sacrificing the resolution of the final model. Standard methods to process thermal images required a larger image footprint (or area of ground capture in a frame), the reason for this is that these types of images lack the presence of invariable features and by increasing the image’s footprint, the number of features present in each frame also rises. However, this method of image capture results in a lower resolution of the final product. The algorithm was developed using open-source libraries. In order to validate the obtained results, this model was compared to data obtained from commercial products, like Pix4D. Furthermore, due to circumstances brought about by the current pandemic, it was not possible to conduct a field study for the comparison and assessment of our results, as such the validation of the models was performed by verifying if the geographic location of the model was performed correctly and by visually assessing the generated maps.Avanços no campo da visão computacional permitiu o desenvolvimento de algoritmos mais eficientes de fotogrametria. Structure from Motion (SfM) é uma técnica de fotogrametria que tem como objetivo a reconstrução digital de objectos no espaço derivados de uma sequência de imagens. A característica importante que os Veículos Aérios não-tripulados (UAV) conseguem fornecer, a nível de mapeamento, é a sua capacidade de obter um conjunto de imagens de alta resolução. Devido a isto, UAV tornaram-se num dos métodos adotados no estudo de topografia. A combinação entre SfM e recentes avanços nos UAV permitiram uma melhor flexibilidade e aplicabilidade, permitindo deste modo desenvolver um novo método de Remote Sensing. Este método pretende substituir técnicas tradicionais, as quais estão associadas a mão-de-obra intensiva e a custos monetários elevados. Avanços contínuos feitos em softwares de reconstrução digital e no poder de processamento resultou em modelos de maior resolução e menos dispendiosos comparando a métodos tradicionais. O presente estudo propõe um algoritmo de reconstrução digital baseado em imagens obtidas através de UAV inspiradas no estudo disponibilizado pela OpenDroneMap. Estas imagens são inseridas no programa de visão computacional, onde várias operações são realizadas, incluindo: deteção e correspondência de caracteristicas, geração da point cloud, meshing e texturação dos quais resulta o produto final que representa o local em estudo. De forma complementar, concluiu-se que o trabalho da OpenDroneMap não incluia um processo de tratamento de imagens térmicas. Desta forma, alterações foram efetuadas que permitissem a criação de mapas térmicos sem sacrificar resolução do produto final, pois métodos típicos para processamento de imagens térmicas requerem uma área de captura maior, devido à falta de características invariantes neste tipo de imagens, o que leva a uma redução de resolução. Desta forma, o programa proposto foi desenvolvido através de bibliotecas open-source e os resultados foram comparados com modelos gerados através de software comerciais. Além do mais, devido à situação pandémica atual, não foi possível efetuar um estudo de campo para validar os modelos obtidos, como tal esta verificação foi feita através da correta localização geográfica do modelo, bem como avaliação visual dos modelos criados

Surface Reconstruction from Distributed Angle Measurements

This thesis presents an innovative solution to the shape measurement of large structures for space applications. The current state-of-the-art heavily relies on optical solutions such as cameras or lasers to recover the shape of a surface. Because of the impracticality of placing a system in front of a large structure flying in space, new solutions need to be developed. The proposed solution is to embed angular sensors (such as sun sensors) directly on the surface. The sensors provide a collection of distributed measurements that form a discrete map of the angular orientation of the structure. An integration scheme can then estimate the 3D shape of the surface. A mathematical model to perform the integration from angle measurements to the shape of a 3D surface is presented first. This model is purely geometric and serves as a basis for similar concepts. The surface is known in a reference configuration and is assumed to have deformed inextensibly to its current shape. Inextensibility conditions are enforced through a discretization of the metric tensor generating a finite number of constraints. This model parameterizes the shape of the surface using a small number of unknowns, and thus requires a small number of sensors. We study the singularities of the equations and derive necessary conditions for the problem to be well-posed. The limitations of the algorithm are highlighted. Simulations are performed on developable surfaces to analyze the performance of the method and to show the influence of the parameters used in the algorithm. Optimal schemes which lower the RMS error between the reconstructed shape and the actual one are presented. An experimental validation of the proposed solution and algorithm is performed on a 1.3 x 0.25 m structure with 14 embedded sun sensors. The sensors measure the two local angles of the surface from a light source placed in front of the surface. A small, lightweight and expandable design of the sensors is shown in this thesis. A calibration procedure accurately correlates the output of the sensor with a 0.5° precision. The procedure also highlights the limitations of the design. The structure was deformed in bending and torsion with amplitudes of a few centimeters, and its shape was reconstructed to an accuracy on the order of a millimeter. The accuracy of the initial algorithm is found to be limited by local shape deformations caused by the mechanical response of the structure. A new algorithm, replacing the discrete inextensibility conditions with the equilibrium equations derived from a finite-element model, is shown. This new algorithm is tested on the experimental structure and the accuracy of the reconstruction is increased by a factor of 2. The RMS error is under a millimeter on average over the different applied shapes and goes as low as 0.3 mm. To understand how this solution can apply to large space structures, simulations are performed on a model of a large planar spacecraft. A 25 x 25 m structure representing the current concept for the Caltech Space Solar Power Project satellite is used as an example. Sensors with similar noise properties as the ones built for the experiment are placed on the spacecraft. A finite-element model combining the vibration of the spacecraft with large rigid body rotations is presented. This model is used in a Kalman filter that estimates the shape of the structure by iterative prediction from the dynamic finite-element model and correction from the angle measurements. Simulations are performed around the thruster actuation applied at the corner of the structure to follow a specific guidance scheme that is optimal for space solar power satellites. The actuation creates both vibrations of the structure with amplitudes of few centimeters and large rotations of the spacecraft. The designed Kalman filter can accurately estimate both effects and it is shown that millimeter accuracy is achievable. The relationship between the number of sensors, the reconstructed shape error, as well as potential stiffness deviations in the FE model is studied. The results provide first order estimates of the performance of this measurement system, in order to enable the design of future space missions.</p