The rapid proliferation of three-dimensional (3D) data acquisition techniques based either on image-based approaches or on range-based approaches is changing the way cultural heritage is recorded. Independently of the technique selected, low-cost structure-from-motion (SfM) photogrammetric computer vision, stereo-plotting, surveying either with measuring tape, total station or global navigation satellite system (GNSS), up to laser scanning/LiDAR/RADAR, among others, there is a need to fuse efficiently different datasets. This research aims to test the fusion performance of the new 3DVEM - Register GEO software for the recording of a complex sculpture. The fusion of data will consider several point clouds acquired with a terrestrial laser scanner and one high-resolution point cloud generated with SfM software. The registration process is performed to merge all the data in a final homogeneous framework (reference system). The results obtained will be used to create a high-resolution 3D model of the sculpture from the consolidated data

Lerma, Jose Luis

Munumer, Estibaliz

English

UCL Discovery

FUSION OF 3D DATA FROM DIFFERENT IMAGE-BASED AND RANGE-BASED SOURCES FOR EFFICIENT HERITAGE RECORDING Estibaliz Muñumer, José Luis Lerma Universitat Politècnica de València Valencia, Spain esmuuher@topo.upv.es; jllerma@cgf.upv.es   Abstract—The rapid proliferation of three-dimensional (3D) data acquisition techniques based either on image-based approaches or on range-based approaches is changing the way cultural heritage is recorded. Independently of the technique selected, low-cost structure-from-motion (SfM) photogrammetric computer vision, stereo-plotting, surveying either with measuring tape, total station or global navigation satellite system (GNSS), up to laser scanning/LiDAR/RADAR, among others, there is a need to fuse efficiently different datasets. This research aims to test the fusion performance of the new 3DVEM – Register GEO software for the recording of a complex sculpture. The fusion of data will consider several point clouds acquired with a terrestrial laser scanner and one high-resolution point cloud generated with SfM software. The registration process is performed to merge all the data in a final homogeneous framework (reference system). The results obtained will be used to create a high-resolution 3D model of the sculpture from the consolidated data.  Index Terms—sculpture; heritage; 3D modeling; data merging; registration; alignment; point clouds; laser scanning; SfM I.  INTRODUCTION Recording cultural heritage objects and sites requires effective solutions to satisfy the most stringent requirements of the survey, not only regarding user’s needs, but also accuracy, cost, level of detail, temporal constraints and contextual factors.  Many methods can be followed for the acquisition and processing of 3D data [1]. An extensive review on the image-based performance can be found in [2]. The possibility to carry out the 3D digitization of objects is also possible with range-based solutions [3].  The main feature of the latter is that they are able to measure accurately featureless objects. The fusion of the different range images is usually undertaken with the Iterative Closest Point (ICP) method or any of its variants [4].  For georeferencing, total stations and GNSS receivers are the most common direct techniques used to deliver accurate control point measurements. The more complex and larger the site, monument or object, more demanding will be the need to provide control points across the surveying area. The existence of control points is not only beneficial for image-based solutions but also for projects undertaken with terrestrial laser scanners (TLS). II. CASE OF STUDY: ‘CADASCÚ AL SEU LLOC’ The complex sculpture selected for the recording and documentation at the Campus of the Polytechnic University of Valencia (‘Universitat Politècnica de València’, UPV) was created by Evarist Navarro, a well-known artist in the Valencian Community due to his popular work in clay. In his early beginning he always focused in sculpting small objects such as jugs and glasses. Afterwards, he moved into larger sized creations. In 1991 Navarro became Professor at the Sculpture Department, Faculty of the Fine Arts at the UPV. In 1992, he was invited to take part in the sculpture program of the UPV Campus, as well as other remarkable artists, placing his piece of art called ‘Cadascú Al Seu Lloc’, composed of three blocks [5]: one large with six tentacles; another midsize ‘H shape’ (Fig. 1), and finally an arched smaller block. In this spirit, the sculpture was built in clay.   Fig. 1. ‘Cadascú Al Seu Lloc’ at the UPV Campus: close up view of the ‘H shape’. The geometry is quite linear and its surface keeps the roughness of the clay. The midsize block was the selected piece of art for this study. Its dimension can be fitted into a rectangular parallelepiped of 304 (L) cm x 60 (W) cm x 86 (H) cm (foreground block in Fig. 1). This paper focuses on the indirect registration undertaken between point clouds from distinct sources and the subsequent process of mesh generation. III. DATA AQUISITION AND REGISTRATION A. Terrestrial Laser Scanner (TLS) The Leica ScanStation 2 was selected for the terrestrial laser scanning survey. It is a pulse-based panoramic TLS measuring up to 50000 points/s. Its nominal accuracy (one sigma) for single measurement is 6 mm at 50 m range. Despite its large dimensions, weight and slow scanning speed compared to the new models launched in the last recent years, this laser scanning system is still valid to record many of the existing heritage objects and sites. Moreover, when the TLS is properly calibrated geometrically [12].  A total of five scans distributed throughout the surrounding gardens of the Campus were acquired to record most of the relevant sculptures of the eastern area. The sampling resolution varied between 4.5-7 mm. Each scan was acquired from a different scan position and orientation. Therefore, the resulting point clouds were in different local Cartesian coordinate systems (Fig. 2a). Paddle targets were placed across the surveying area in order to undertake the indirect registration to set all the point clouds in a single common reference system. This step yielded new exterior orientation parameters (rotation and translation) for each scan station.  The targets can be measured and surveyed with conventional surveying devices, e.g. total station or GNSS, in order to achieve coordinates in a global reference system [6]. A total of 6172433 raw points were saved to PTS files from Cyclone software. Table 1 shows the number of points acquired per scan station (SW).  TABLE I.  NUMBER OF POINTS PER SCAN POSITIONS Scan Position Number of Points Acquired SW1 1112728 SW3 1550163 SW4 1835999 SW5 758161 SW6 915382  The indirect registration was successfully undertaken with 3DVEM – Register. A root mean square error of 1.5 mm was achieved, with a maximum residual error of 3.5 mm. The precision of the exterior orientation parameters was smaller than 0.01º for the rotation angles and up to 1 mm for the positions. After registration (Fig. 2b), those five point clouds are merged onto one single coordinate system. 3DVEM – Register equals the exterior orientation parameters of the registration compared to commercial Leica Cyclone software and outperforms FARO Scene [13]. In addition, it is very quick, straightforward and simple. Furthermore, the extensive statistical report provided by 3DVEM – Register can be used to confirm the quality of the registration, beyond conventional reports.  Fig. 2. Terrestrial laser scanning point clouds: a) before registration; b) after registration B. Digital camera The major disadvantage of a TLS nowadays is still the economic viewpoint: not everybody can buy or even rent either phase-based or pulse-based TLS as they are still too expensive for many of the small projects dealing with cultural heritage (namely in archaeology and architecture). However, everybody is carrying a digital camera in any of the existing mobile tablets, smartphones, or can buy either a digital single-lens reflex (SLR) camera, a semi-professional camera or just a consumer camera: a digital camera is more affordable to any user and eventually much easier to use. The following section shows the chance of getting great 3D reconstruction results from images taken with a purchasable imaging device in any of the forms, DSLR, compact camera or even with a mobile phone camera [7]. A Canon EOS-1Ds Mark III body and a Canon EF 24 mm f/2.8 lens was used to shoot Navarro’s sculptural block. The weather conditions determine the quality of the images taken. Therefore, it is recommended to avoid strong sunlight, hard shadows and high temperatures. In addition, in order to obtain right stereoscopic vision, an 80% of overlapping portion was attempted to keep between sequential images, decreasing until 60% in the worst scenario. The images were covering the whole sculpture. Two main paths were defined: first, one camera path travelling the block in a parallel sequence; second, another camera path travelling the block in a continuous tilted sequence. Figure 3 shows a display of the bundle of images covering the sculpture.  Fig. 3. Camera positions and attitudes surrounding the scuplture A total of 172 images were selected for the following image-based processing steps. Blur as well as over/under exposure images were removed and not considered for the survey. IV. POINT CLOUD  GENERATION FROM IMAGES The set of images can be used to reconstruct the geometry and color of the present condition of the sculpture. In fact, from the imagery, a dense point cloud is foreseen. VisualSFM software (GUI application for 3D reconstruction using SfM [8]) was used for this survey. VisualSFM runs fast by exploiting multicore parallelism for feature detection, feature matching, and bundle adjustment. It was installed in a Windows PC together with the CMVS binaries (cmvs.exe, pmvs2.exe, genOption.exe, pthreadVC2.dll) [9, 10]. The dense 3D point cloud was obtained after four steps: 1) Adding all the images. 2) Image matching: feature detection and description. 3) Sparse reconstruction. 4) Dense reconstruction. A total of 14706 image matches were processed. After 8.5 h, a dense reconstruction (point cloud) of 4637088 points was obtained (Fig. 4b).  V. FUSION OF IMAGE-BASED AND RANGE-BASED POINT CLOUDS The fusion of the diverse point clouds coming from image-based SfM algorithms and TLS datasets (Fig. 4) comes next. The fusion will be undertaken with the new GEO module of 3DVEM software, called 3DVEM – Register GEO. It is an evolution of the registration software 3DVEM – Register [9] targeting the registration/alignment not only of laser scanning point clouds (with six degrees of freedom, three rotations and three translations) but whatever surveying dataset in the form of point coordinates, point clouds and/or 3D models. The new powerful and low-cost 3DVEM – Register GEO software is able to fuse both point clouds and 3D models, based on three sets of geometric constraints (scale, translations and rotations). There exists also the change to set interactively the scale based on distance measurements, as well as the alignment of the new Cartesian axes. Furthermore, its absolute transformation functionality allows users to carry out a global least squares transformation either in local, global or UTM coordinates as far as control points are available.  Fig. 4. Point clouds derived from TLS (left) and VisualSFM (right) The registration between TLS datasets was fully automatic due to the existence of paddle target measurements. However, the imagery and the laser scanning survey were acquired in different days. Therefore, for the registration of the point clouds coming from the image-based SfM approach and the registered TLS dataset some control points were interactively measured.  In fact, a total of four natural tie points were measured using the 3DVEM – 3D Viewer, Editor & Meter software [11] in both point clouds. The registration of the two point clouds was executed in three simple steps: 1) Register module: import the SfM point cloud as well as the corresponding control points.  2) GEO module: import the corresponding TLS control points. 3) GEO module: compute the 3D transformation (Fig. 5).   Fig. 5. GEO Module – Adjustment of the point clouds 3DVEM – Register GEO software was used to undertake the adjustment; the results obtained were far better than expected. Check the resulting parameters achieved after the transformation (Fig. 5). The resulting SfM point cloud was merged and easily georeferenced to the other dataset (Fig. 6).   Fig. 6. Fusion of both point clouds The software provides additional tools for scaling, rotating and translating into any absolute coordinate system (Fig. 7). Furthermore, it is possible to georeferenced straight-ahead in UTM coordinates unlike most of the laser scanning software.   Fig. 7. Global transformation parameters that can be applied to any point cloud/3D model VI. NOISE REDUCTION AND MESH CREATION The merged point cloud was extremely dense. Therefore, all the points outside the sculpture were cleaned. It is assumed that measurements follow a Gaussian distribution. MeshLab software was used to remove noise: 45% of the total number of vertexes (Fig. 8).   Fig. 8. Cleaned and noise-free merged point cloud Once the point cloud was simplified, the surface normals were computed prior to surface generation. In particular, the Poisson surface reconstruction was run to obtain the mesh. After transferring the RGB values from the point cloud, a final texturized 3D model was obtained (Fig. 9).  Fig. 9. Texturized 3D model of the ‘H shape’ sculpture. VII. CONCLUSION This paper has presented the fusion of 3D data acquired from different range-based and/or image-based sources. The existence of many measuring devices and techniques requires the user’s selection of cost-efficient tools to yield comprehensive 3D (and more and more 4D) analysis and understanding of our cultural heritage. The two 3DVEM software modules presented herein: Register and GEO, altogether with the Viewer, Editor & Meter, allow users to achieve maximum fusion satisfaction with minimum resources, either in software or in hardware.  The point cloud achieved with SfM is noteworthy as far as exists right image network geometry, enough object texture and even illumination. The results even outperform or at least wisely complement laser scanning.  A new era of low-cost and high quality 3D models is foreseen with the software presented herein. ACKNOWLEDGMENT The authors would like to thank the Photogrammetry & Laser Scanning Research Group (GIFLE) teamwork for the feedback provided during the research. REFERENCES [1] S. El-Hakim, J-A. Beraldin, M. Picard and G. Godin, “Detailed 3D Reconstruction of Large-Scale Heritage Sites with Integrated Techniques” in IEEE Computer Graphics and Application, May/June 2014, pp. 21-29. [2] F. Remondino and S. El-Hakim, “Image-based 3D modelling: A review” in The Photogrammetric Record 21(115), September 2006, pp. 269-291. [3] J-A. Beraldin, L. Cournoyer, M. Rioux, F. Blais, S.F. El-Hakim and G. Godin, “Object model creation from multiple range images: Acquisition, Calibration, Model Building and Verification” in Proc. 1st Int’l Conf. on 3-D Digital maging and Modeling. In: IEEE, 1997, pp. 326–333.  [4] S. Rusinkiewicz and M. Levoy, “Efficient Variants of the ICP Algortithm” in Third Internation Conference on o3-D Digital Imaging and Modelling, 2001, pp. 145-152. [5] UPV Radiotelevisió, “Campus Escultòric: Cadascú al seu lloc”, https://www.youtube.com/watch?v=-ekEoFvy5Z0 (Accessed on May 27, 2015). [6] W. Boehler, G. Heinz, A. Marbsand and M. Siebold, “3D Scanning software: an introduction” in CIPA Heritage Documentation, International Workshop on Scanning for Cultural Heritage Recording, 2002, pp. 47-56. [7] J. L. Lerma and C. Muir, “Evaluating the 3D documentation of an early Christian upright stone with carvings from Scotland with multiple images” in Journal of Archaeological Science, vol. 46, June 2014, pp. 311-318. [8] C. Wu, “VisualSFM: A Visual Structure from Motion System",            http://ccwu.me/vsfm/ (Accessed on May 27, 2015). [9] C. Wu, “Towards Linear-time Incremental Structure From Motion” in International Conference on 3D Vision, 2013.  [10] Y. Furukawa “Clustering Views for Multi-view Stereo (CMVS)”, http://www.di.ens.fr/cmvs/ (Accessed on March 12, 2015). [11] S. Fabado, A. E. Seguí, M. Cabrelles, S. Navarro, D. García-De-San-Miguel and J. L. Lerma, “3DVEM Software modules for efficient management of point clouds and photorealistic 3D models” in ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XXIV International CIPA Symposium, 2013, pp. 255-260.  [12] J. L. Lerma and D. García-San-Miguel, “Self-calibration of terrestrial laser scanners: selection of the best geometric additional parameters” in ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. II-5, pp. 219-226. [13] S. Artese, C. Altomare, J. L. Lerma and R. Zinno, “Terrestrial laser scanning registration analysis and its effects on a masterpiece dome” in 3D/4D Documentation in Cultural Heritage, International Conference On Cultural Heritage EuroMed, 2014, pp. 89-99. 

Fusion of 3D data from different image-based and range-based sources for efficient heritage recording

https://discovery.ucl.ac.uk/10144857/1/201509_Digital_Heritage.pdf

Fusion of 3D data from different image-based and range-based sources for efficient heritage recording

Abstract

Similar works

Full text

Available Versions

UCL Discovery