Generating Absolute-Scale Point Cloud Data of Built Infrastructure Scenes Using a Monocular Camera Setting

The global scale of Point Cloud Data (PCD) generated through monocular photo/videogrammetry is unknown, and can be calculated using at least one known dimension of the scene. Measuring one or more dimensions for this purpose induces a manual step in the 3D reconstruction process; this increases the effort and reduces the speed of reconstructing scenes, and induces substantial human error in the process due to the high level of measurement accuracy needed. Other ways of measuring such dimensions are based on acquiring additional information by either using extra sensors or specific classes of objects existing in the scene; we found that these solutions are not simple, cost effective or general enough to be considered practical for reconstructing both indoor and outdoor built infrastructure scenes. To address the issue, in this paper, we propose a novel method for automatically calculating the absolute scale of built infrastructure PCD. We use a pre-measured cube for outdoor scenes and a sheet of paper for indoor environments as the calibration patterns. Assuming that the dimensions of these objects are known, the proposed method extracts the objects’ corner points in 2D video frames using a novel algorithm. The extracted corner points are then matched between the consecutive frames. Finally, the corresponding corner points are reconstructed along with other features of the scenes to determine the real world scale. To evaluate the performance of the method, ten indoor and ten outdoor cases were selected and the absolute-scale PCD for each case was computed. Results illustrated the proposed algorithm is able to reconstruct the predefined objects with a high success rate while the generated absolute scale PCD is sufficiently accurate.This is the accepted manuscript. The final version is available from ASCE at http://dx.doi.org/10.1061/(ASCE)CP.1943-5487.000041

Multistep Explicit Stereo Camera Calibration Approach to Improve Euclidean Accuracy of Large-Scale 3D Reconstruction

The spatial accuracy of point clouds generated by stereo image-based 3D reconstruction algorithms is very sensitive to the intrinsic and extrinsic camera parameters determined during camera calibration. The existing camera calibration algorithms induce a significant amount of error due to poor estimation accuracies in camera parameters when they are used for large-scale scenes such as mapping civil infrastructure. This leads to higher uncertainties in the location of 3D points, and may result in the failure of the whole reconstruction process. This paper proposes a novel procedure to address this problem. It hypothesizes that a set of multiple calibrations created by videotaping a moving calibration pattern along a specific path can increase overall calibration accuracy. This is achieved by using conventional camera calibration algorithms to perform separate estimations for some predefined distance values. The result, which includes multiple sets of camera parameters, is then used in the Structure from Motion process to improve the Euclidean accuracy of the reconstruction. The proposed method has been tested on infrastructure scenes and experimental analyses indicate more than 25% improvement in the spatial accuracy of 3D points.This is the accepted manuscript. The final version is available from ASCE at http://dx.doi.org/10.1061/(ASCE)CP.1943-5487.000045

Development of 3D city model using videogrammetry technique

3D city model is a representation of urban area in digital format that contains building and other information. The current approaches are using photogrammetry and laser scanning to develop 3D city model. However, these techniques are time consuming and quite costly. Besides that, laser scanning and photogrammetry need professional skills and expertise to handle hardware and tools. In this study, videogrammetry is proposed as a technique to develop 3D city model. This technique uses video frame sequences to generate point cloud. Videos are processed using EyesCloud3D by eCapture. EyesCloud3D allows user to upload raw data of video format to generate point clouds. There are five main phases in this study to generate 3D city model which are calibration, video recording, point cloud extraction, 3D modeling and 3D city model representation. In this study, 3D city model with Level of Detail 2 is produced. Simple query is performed from the database to retrieve the attributes of the 3D city model

Evaluasi Metode Aerial Videogrametri untuk Rekonstruksi 3D Bangunan (Studi Kasus: Candi Singasari, Jawa Timur)

Candi Singasari merupakan salah satu situs warisan dunia dimana upaya untuk konservasi, inventarisasi, dan dokumentasi cagar budaya tersebut diperlukan. Sebagai langkah awal yang dapat dilakukan adalah membentuk model rekosntruksi 3D Candi Singasari. Adapun teknologi yang ditawarkan dalam pemodelan rekonstruksi 3D bangunan begitu bervariasi dengan kelebihan dan kelemahan masing-masing seperti teknologi LiDAR, laser scanner, dan fotogrametri jarak dekat. Dewasa ini, perkembangan dalam dunia fotogrametri untuk kegiatan pemodelan 3D begitu pesat hingga adanya metode yang berkembang saat ini yaitu metode videogrametri. Penelitian ini menggunakan teknologi videogramteri dan konsep SfM untuk membentuk rekonstruksi 3D Candi Singasari. Teknologi videogrametri memiliki kelebihan seperti akuisisi data yang sederhana, harga ekonomis, minim perubahan fotometrik dalam gambar, dan daya overlap tinggi dari data yang dihasilkan. Kriteria yang digunakan dalam rekonstruksi 3D Candi Singasari menggunakan klasifikasi Level of Detail (LoD). Pada penelitian ini difokuskan pada dua aspek yaitu aspek evaluasi visual rekonstruksi 3D dan perhitungan nilai ketelitian geometrik struktur Candi Singasari. Hasil evaluasi visual rekonstruksi 3D diperoleh bentuk point cloud sejumlah 308.248 titik, dense cloud sejumlah 1.126.457 titik, dan textured yang meliputi 225.291 titik (permukaan/face) dan 113.595 vertex dari 161 frame video dengan rata-rata deteksi fitur tiap frame video sejumlah 2.242 titik yang diolah menggunakan konsep structure from motion (SfM). Sedangkan evaluasi untuk validasi nilai geometrik struktur Candi Singasari diperoleh dari hasil korelasi dan uji standar kesalahan. Hasil korelasi antara variabel ICP ukuran model rekonstruksi 3D terhadap ukuran di lapangan menunjukkan nilai kolerasi untuk koordinat easting 0.998, koordinat northing 0.997, dan koordinat Z 0.998. Adapun untuk nilai uji standar kesalahan untuk koordinat easting diterima 83%, koordinat northing diterima 91.7%, dan koordinat Z diterima 91.7% juga nilai RMSE koordinat easting 0.177 meter, northing 0.194 meter, dan koordinat Z 0.168 meter

Point Cloud Data Cleaning and Refining for 3D As-Built Modeling of Built Infrastructure

Spatial sensing of built infrastructure is now a common practice within the AEC industry and results are commonly encapsulated in the form of dense point cloud data (PCD). PCD of built infrastructure might consist of millions of spatial points and it is well known that processing all these points is neither necessary nor computationally feasible. In addition, due to several reasons including hardware and/or software deficiencies, there might be several outliers that need to be removed from the PCD before further processing. As the result, cleaning and refining PCD is a paramount step in the process of spatial sensing and object-oriented modeling of built infrastructure scenes. This research work entails two parts: The first part provides an in-depth literature review on current states of practice and research on the concept of PCD cleaning. The second part presents the authors’ suggested framework for cleaning and refining PCD of built infrastructure. This prototype mainly consists of three major components: (1) removing outliers; (2) filling holes and gaps on surfaces of PCD; and (3) balancing the density of different areas of PCD based on a plane recognition approach. Several case studies are presented to demonstrate the efficiency of the proposed framework.This is the author accepted manuscript. The final version is available from ASCE via https://doi.org/10.1061/9780784479827.093#sthash.SfsxrNpd.dpu

Evaluasi Metode Aerial Videogrametri Untuk Rekonstruksi 3D Bangunan (Studi Kasus: Candi Singasari, Jawa Timur) - Evaluation Of 3D Building Reconstruction Using Aerial Videogrammetry Technique (Case Study: Singasari Temple, East Java)

Candi Singasari merupakan salah satu situs warisan dunia dimana upaya untuk konservasi, inventarisasi, dan dokumentasi cagar budaya tersebut diperlukan. Sebagai langkah awal yang dapat dilakukan dalam upaya dokumentasi cagar budaya adalah membentuk model rekosntruksi 3D Candi Singasari. Adapun teknologi yang ditawarkan dalam pemodelan rekonstruksi 3D bangunan begitu bervariasi dengan kelebihan dan kelemahan masing-masing seperti teknologi LiDAR, laser scanner, dan fotogrametri jarak dekat. Dewasa ini, perkembangan dalam dunia fotogrametri untuk kegiatan pemodelan 3D begitu pesat hingga adanya metode yang berkembang saat ini yaitu metode videogrametri. Penelitian ini menggunakan teknologi videogramteri dan metode SfM untuk membentuk rekonstruksi 3D Candi Singasari. Teknologi videogrametri memiliki kelebihan seperti akuisisi data yang sederhana, harga ekonomis, minim perubahan fotometrik dalam gambar, dan daya overlap tinggi dari data yang dihasilkan. Kriteria yang digunakan dalam rekonstruksi 3D Candi Singasari menggunakan klasifikasi Level of Detail (LoD). Pada penelitian ini difokuskan pada dua aspek yaitu aspek evaluasi visual rekonstruksi 3D dan perhitungan nilai ketelitian geometrik struktur Candi Singasari. Hasil evaluasi visual rekonstruksi 3D diperoleh bentuk point cloud sejumlah 308.248 titik, dense cloud sejumlah 1.126.457 titik, dan textured yang meliputi 225.291 titik (permukaan/face) dan 113.595 vertex dari 161 frame video dengan rata-rata deteksi fitur tiap frame video sejumlah 2.242 titik yang diolah menggunakan metode structure from motion (SfM). Sedangkan evaluasi untuk validasi nilai geometrik struktur Candi Singasari diperoleh dari hasil korelasi dan uji standar kesalahan. Hasil korelasi antara variabel ICP ukuran model rekonstruksi 3D terhadap ukuran di lapangan menunjukkan nilai kolerasi untuk koordinat easting sebesar 0.998, koordinat northing sebesar 0.997, dan koordinat Z 0.998. Adapun untuk nilai uji standar kesalahan untuk koordinat easting diterima sebesar 83%, koordinat northing diterima sebesar 91.7%, dan koordinat Z diterima sebesar 91.7% juga nilai RMSE koordinat easting sebesar 0.177 meter, northing sebesar 0.194 meter, dan Z sebesar 0.168 meter. ======================================================================================================================== Singasari Temple is one of the world heritage site where efforts for conservation, inventory, and documentation of cultural heritage needed. As a first step to do is to form a 3D model of reconstruction for the Singasari Temple. The technology offered in the modelling of a 3D building reconstruction is so varied with the advantage and disadvantage of each technology such as LiDAR, Laser Scanner, and close range photogrammetry. Now, photogrammetry technique developments for 3D modelling activity so fast in the world that their methods were developed at this time such a videogrammetry method. This studyresearch uses videogrammetry method and SfM (Structure from Motion) for 3D building reconstruction of the Singasari Temple. Videogrammetry method has advantages such as simply of data acquisition, economical price, minimum photometric change in the image, and the generated power of high overlap data. Criteria used for Singasari 3D reconstruction use level of detail (LOD) classification. In this study focused on two aspects, visual evaluation of 3D reconstruction and value calculation of the geometric accuracy of Singasari Temple structure. The result of visual evaluation of 3D reconstruction is obtained the point cloud a number of 308,248 points, dense point cloud number 1,126,457 points, and the textured cover 225,291 points (surface), and 113,595,161 vertex of video frames with the average detection feature each video frame number 2,242 points whichs is processed using method of structure from motion (SfM). While the evaluation for validation of the geometric value structure of Singasari Temple is obtained from the correlation between two variables and standard test error. The correlation between the variable ICP 3D reconstruction model size to the size of the field shows the correlation value for 0.998 amounted to easting coordinate, northing coordinate of 0.997, and Z coordinate of 0.998. as for the value of standard test for easting coordinate error accepted by 83%, northing coordinate 91.7%, and Z coordinates 91.7% and RMSE value for easting coordinate 0.177 meters, northing coordinate 0.194 meter, and Z coordinate 0.168 meters

EFFECT OF KEYFRAMES EXTRACTION FROM THERMAL INFRARED VIDEO STREAM TO GENERATE DENSE POINT CLOUD OF THE BUILDING'S FACADE

Keyframes extraction is required and effective for the 3D reconstruction of objects from a thermal video sequence to increase geometric accuracy, reduce the volume of aerial triangulation calculations, and generate the dense point cloud. The primary goal and focus of this paper are to assess the effect of keyframes extraction from the thermal infrared video sequence on the geometric accuracy of the dense point cloud generated. The method of keyframes extraction of thermal infrared video presented in this paper consists of three basic steps. (A) The ability to identify and remove blur frames from non-blur frames in a sequence of recorded frames. (B) The ability to apply the standard baseline condition between sequence frames to establish the overlap condition and prevent the creation of degeneracy conditions. (C) Evaluating degeneracy conditions and keyframes extraction using Geometric Robust Information Criteria (GRIC). The performance evaluation criteria for keyframes extraction in the generation of the thermal infrared dense point cloud in this paper are to assess the increase in density of the generated three-dimensional point cloud and reduce reprojection error. Based on the results and assessments presented in this paper, using keyframes increases the density of the thermal infrared dense point cloud by about 0.03% to 0.10% of points per square meter. It reduces the reprojection error by about 0.005% of pixels (2 times)

Image-Based 3D Reconstruction of Utah Roadway Assets

Understanding the condition of roadway assets is important for transportation agencies to plan for future improvements and asset management purposes quantitatively. Since these assets are distributed across the country, a manual data collection system falls short of the automated methods due to time and cost issues. Some pioneer departments of transportation in the United States use mobile Light Detection and Ranging (LiDAR) to monitor highway assets and pavement condition data. However, LiDAR is expensive and not affordable for every maintenance agency. Additionally, special technical knowledge is required to perform this method, which may not be accessible to the maintenance agency staff. Recently, image-based 3D reconstruction has been shown to be a cheaper and simpler technology than LiDAR. In this report, we assess the alternative method (image-based) for reconstructing 3D models (virtual 3D point clouds) of transportation agencies. The analysis of the data quality and associated costs holds the promise for conducting a feasible roadway asset inventory

An empirical assessment of real-time progressive stereo reconstruction

3D reconstruction from images, the problem of reconstructing depth from images, is one of the most well-studied problems within computer vision. In part because it is academically interesting, but also because of the significant growth in the use of 3D models. This growth can be attributed to the development of augmented reality, 3D printing and indoor mapping. Progressive stereo reconstruction is the sequential application of stereo reconstructions to reconstruct a scene. To achieve a reliable progressive stereo reconstruction a combination of best practice algorithms needs to be used. The purpose of this research is to determine the combinat ion of best practice algorithms that lead to the most accurate and efficient progressive stereo reconstruction i.e the best practice combination. In order to obtain a similarity reconstruction the in t rinsic parameters of the camera need to be known. If they are not known they are determined by capturing ten images of a checkerboard with a known calibration pattern from different angles and using the moving plane algori thm. Thereafter in order to perform a near real-time reconstruction frames are acquired and reconstructed simultaneously. For the first pair of frames keypoints are detected and matched using a best practice keypoint detection and matching algorithm. The motion of the camera between the frames is then determined by decomposing the essential matrix which is determined from the fundamental matrix, which is determined using a best practice ego-motion estimation algorithm. Finally the keypoints are reconstructed using a best practice reconstruction algorithm. For sequential frames each frame is paired with t he previous frame and keypoints are therefore only detected in the sequential frame. They are detected , matched and reconstructed in the same fashion as the first pair of frames, however to ensure that the reconstructed points are in the same scale as the points reconstructed from the first pair of frames the motion of the camera between t he frames is estimated from 3D-2D correspondences using a best practice algorithm. If the purpose of progressive reconstruction is for visualization the best practice combination algorithm for keypoint detection was found to be Speeded Up Robust Features (SURF) as it results in more reconstructed points than Scale-Invariant Feature Transform (SIFT). SIFT is however more computationally efficient and thus better suited if the number of reconstructed points does not matter, for example if the purpose of progressive reconstruction is for camera tracking. For all purposes the best practice combination algorithm for matching was found to be optical flow as it is the most efficient and for ego-motion estimation the best practice combination algorithm was found to be the 5-point algorithm as it is robust to points located on planes. This research is significant as the effects of the key steps of progressive reconstruction and the choices made at each step on the accuracy and efficiency of the reconstruction as a whole have never been studied. As a result progressive stereo reconstruction can now be performed in near real-time on a mobile device without compromising the accuracy of reconstruction