Automatic Registration of Multiple Texel Images to Form a 3-Dimensional Texel Image

Three-dimensional (3D) imagery has gained a lot of importance in today\u27s world, be it in the field of entertainment, documentation, or defense. Multiple methods for creating 3D images have been proposed in the past. A few famous methods used for 3D image matching are those that include usage of 2D images as stereo pairs or computing 3D rigid body transformations based on range information of points. The Iterative Closest Point algorithm (ICP) and its variants are well known for registration of point clouds, which can be used to create 3D surfaces. This thesis provides an algorithm, which is a continuation of the work done previously at Utah State University, to create accurate 3D images based on texel images obtained from the handheld texel camera built at USU. The first part of the thesis briefly reviews the structure and working of the handheld texel camera and the technique of creating texel images using the device and calibrating the images to mitigate the effect of lens distortions. A method is then suggested to reduce the errors in the range information in the image caused by walk error and wiggling error and also to compensate for the timing error induced in the individual pixels of the lidar sensor. A way to add a correcting factor to the range information to compensate for any oset in the origin assumed by the sensor and the actual center of perspective (COP) of the sensor is suggested in the later part of the thesis, thus correcting the images for the inaccuracies caused by the oset. The second half of the thesis brie y goes over the work previously done on 3D image matching and registration to produce 3D images. A few changes are suggested in some parts of the existing method, which use concepts of epipolar geometry in the RANSAC algorithm and use planar interpolation to accurately obtain the 3D co-ordinates of points from 2D coordinates. An iterative solution is proposed to correct erroneously chosen correspondences or reject bad correspondences to improve the rigid body transformation. The transformation thus obtained is used to compute more point matches, which are in turn used to estimate a more accurate least squares solution for the rigid body transformation. Results show that the calibration techniques and the changes implemented in the point cloud matching algorithm, suggested in this thesis, improve the accuracy of the images and produce 3D images with correct matching

Generation Of An Accurate, Metric Spatial Database Of A Large Multi Storied Building

This thesis presents the development of a novel method to generate an accurate, metric spatial database of a large multi storied building during construction. The algorithm uses the 3D CAD model of the building and the video of the structure captured by an Unmanned Aircraft System (UAS). The spatial database is then used to perform several inspection procedures such as, metric data analysis, spatial query for images, visualization through 3D textured model. The video is processed using a simultaneous localization and mapping (SLAM) system. SLAM generates a sparse 3D map of the environment. Our algorithm registers the 3D map with the 3D CAD model to generate the accurate metric spatial database. The user can click on the desired part of the CAD model for inspection and the image of that part will be shown by using the spatial indexing between the CAD model and the spatially distributed images. The image returned by the spatial query can be used to extract metric information. The spatial database is also used to generate a 3D textured model which provides a visual as-built documentation. The metric data calculation and textured model reconstruction methods have been compared to the state of the art Pix4D software (Latest Release (Version 3.1)). The proposed method has a mean squared error (MSE) of 31.9 cm2 and standard deviation of 4.28 cm where Pix4D had a higher MSE of 45.6 cm2 and standard deviation of 4.91 cm. Using statistical t-test and ANOVA tests we have shown that we are statistically 99% confident that the proposed algorithm has performed better than Pix4D

TwinTex: Geometry-aware Texture Generation for Abstracted 3D Architectural Models

Coarse architectural models are often generated at scales ranging from individual buildings to scenes for downstream applications such as Digital Twin City, Metaverse, LODs, etc. Such piece-wise planar models can be abstracted as twins from 3D dense reconstructions. However, these models typically lack realistic texture relative to the real building or scene, making them unsuitable for vivid display or direct reference. In this paper, we present TwinTex, the first automatic texture mapping framework to generate a photo-realistic texture for a piece-wise planar proxy. Our method addresses most challenges occurring in such twin texture generation. Specifically, for each primitive plane, we first select a small set of photos with greedy heuristics considering photometric quality, perspective quality and facade texture completeness. Then, different levels of line features (LoLs) are extracted from the set of selected photos to generate guidance for later steps. With LoLs, we employ optimization algorithms to align texture with geometry from local to global. Finally, we fine-tune a diffusion model with a multi-mask initialization component and a new dataset to inpaint the missing region. Experimental results on many buildings, indoor scenes and man-made objects of varying complexity demonstrate the generalization ability of our algorithm. Our approach surpasses state-of-the-art texture mapping methods in terms of high-fidelity quality and reaches a human-expert production level with much less effort. Project page: https://vcc.tech/research/2023/TwinTex.Comment: Accepted to SIGGRAPH ASIA 202

Structure-aware content creation : detection, retargeting and deformation

Nowadays, access to digital information has become ubiquitous, while three-dimensional visual representation is becoming indispensable to knowledge understanding and information retrieval. Three-dimensional digitization plays a natural role in bridging connections between the real and virtual world, which prompt the huge demand for massive three-dimensional digital content. But reducing the effort required for three-dimensional modeling has been a practical problem, and long standing challenge in compute graphics and related fields. In this thesis, we propose several techniques for lightening up the content creation process, which have the common theme of being structure-aware, ie maintaining global relations among the parts of shape. We are especially interested in formulating our algorithms such that they make use of symmetry structures, because of their concise yet highly abstract principles are universally applicable to most regular patterns. We introduce our work from three different aspects in this thesis. First, we characterized spaces of symmetry preserving deformations, and developed a method to explore this space in real-time, which significantly simplified the generation of symmetry preserving shape variants. Second, we empirically studied three-dimensional offset statistics, and developed a fully automatic retargeting application, which is based on verified sparsity. Finally, we made step forward in solving the approximate three-dimensional partial symmetry detection problem, using a novel co-occurrence analysis method, which could serve as the foundation to high-level applications.Jetzt hat die Zugang zu digitalen Informationen allgegenwärtig geworden. Dreidimensionale visuelle Darstellung wird immer zum Einsichtsverständnis und Informationswiedergewinnung unverzichtbar. Dreidimensionale Digitalisierung verbindet die reale und virtuelle Welt auf natürliche Weise, die prompt die große Nachfrage nach massiven dreidimensionale digitale Inhalte. Es ist immer noch ein praktisches Problem und langjährige Herausforderung in Computergrafik und verwandten Bereichen, die den Aufwand für die dreidimensionale Modellierung reduzieren. In dieser Dissertation schlagen wir verschiedene Techniken zur Aufhellung der Erstellung von Inhalten auf, im Rahmen der gemeinsamen Thema der struktur-bewusst zu sein, d.h. globalen Beziehungen zwischen den Teilen der Gestalt beibehalten wird. Besonders interessiert sind wir bei der Formulierung unserer Algorithmen, so dass sie den Einsatz von Symmetrische Strukturen machen, wegen ihrer knappen, aber sehr abstrakten Prinzipien für die meisten regelmäßigen Mustern universell einsetzbar sind. Wir stellen unsere Arbei aus drei verschiedenen Aspekte in dieser Dissertation. Erstens befinden wir Räume der Verformungen, die Symmetrien zu erhalten, und entwickelten wir eine Methode, diesen Raum in Echtzeit zu erkunden, die deutlich die Erzeugung von Gestalten vereinfacht, die Symmetrien zu bewahren. Zweitens haben wir empirisch untersucht dreidimensionale Offset Statistiken und entwickelten eine vollautomatische Applikation für Retargeting, die auf den verifizierte Seltenheit basiert. Schließlich treten wir uns auf die ungefähre dreidimensionalen Teilsymmetrie Erkennungsproblem zu lösen, auf der Grundlage unserer neuen Kookkurrenz Analyseverfahren, die viele hochrangige Anwendungen dienen verwendet werden könnten

Damage detection and monitoring for tunnel inspection based on computer vision

The deterioration of the underground infrastructure of the major cities around the world, due to ageing, has become a topic of great concern among engineers. Visual inspection, as part of the routine maintenance procedures, is a common practice used in the condition assessment of infrastructure to ensure its safety and serviceability. This practice, however, is labour-intensive, costly and inaccurate and, therefore, a new system based on computer vision technology is presented in this thesis, aiming to tackle these inadequacies. This thesis proposes a novel mosaicing system for inspection reporting, which can create an almost distortion-free mosaic of tunnels, thus allowing a large area of tunnels to be visualised. The system relies on Structure from Motion (SFM), which enables the system to cope with images with a general camera motion, in contrast to standard mosaicing software that can cope only with a strict camera motion. The system involves the automatic robust estimation of a 3D cylindrical surface using a Support Vector Machine to classify 3D points to improve the accuracy of the estimation. It is shown that some curvatures are observed in the mosaics when an inaccurate surface is used for mosaicing, while the mosaics from a surface estimated using the proposed method are almost distortion-free. New feature matching algorithms aiming to improve the performance of SFM systems are proposed. These algorithms apply a spatial consistency constraint to match features with a similar topography, in contrast to other matching algorithms that rely on matching based on the similar appearance of local image patches. The Shape Context and Random Forest algorithms are combined in the proposed algorithm, revealing promising results. The final contribution is a new change detection system for monitoring cracks in multi-temporal images. The system can cope with images with a general camera motion achieved by geometrical registration using SFM, unlike other systems that assume fixed or controlled cameras. The system performs photometric normalisation to cope with illumination variation in the images, and also a motion-invariant change detection algorithm is applied to handle deformable objects. It is shown that the results from the proposed change detection system are still impractical for use with tunnel images from a real environment, and further study is required

Rank3DGAN: Semantic Mesh Generation Using Relative Attributes

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 66599