Method for 3D modelling based on structure from motion processing of sparse 2D images

A method based on Structure from Motion for processing a plurality of sparse images acquired by one or more acquisition devices to generate a sparse 3D points cloud and of a plurality of internal and external parameters of the acquisition devices includes the steps of collecting the images; extracting keypoints therefrom and generating keypoint descriptors; organizing the images in a proximity graph; pairwise image matching and generating keypoints connecting tracks according maximum proximity between keypoints; performing an autocalibration between image clusters to extract internal and external parameters of the acquisition devices, wherein calibration groups are defined that contain a plurality of image clusters and wherein a clustering algorithm iteratively merges the clusters in a model expressed in a common local reference system starting from clusters belonging to the same calibration group; and performing a Euclidean reconstruction of the object as a sparse 3D point cloud based on the extracted parameters

Rough Estimation of Interior Dimensions Using Structure from Motion Techniques

Tänapäeva uusimaid võtteid kasutades on võimalik liikumisest piltide peal eraldada struktuure, mille täpsust võib võrrelda laser skännerite täpsusega. Kahjuks vajavad need algoritmid siiski huvipunkte ning tekstuurseid pindasid, et anda parimaid tulemusi. Enamus algoritmidel on halb jõudlus kui nad peavad töötlema monotoonsete värvidega või tekstuurivaeseid pindu. Lisaks on tihti algoritmide tulemustes auke, mida algses stseenis ei esine. Selline projektsioon ei ole väga kasulik ega kasutatav juhul kui on stseenist oodatakse järjepidavust ja täielikkust, mitte detailide täpsust. Käesolevas töös proovib autor kasutada liikumisest struktuuri eraldamise võtteid koos uudsete ideedega, selleks et luua projektsioon siseruumist fookusega toa olemusele (õiged pikkuste suhted, õige põranda plaan), mitte objektide detailsusele. Töö eesmärgiks on välja pakkuda algoritm, mis suudab taasluua stseeni, mida oleks võimalik kasutada rakendustes, kus rõhk on stseeni täielikkusel, mitte detailsusel (nt. õige tee leidmise probleemid).Nowadays structure from motion algorithms have become accurate enough to compete with laser scanner accuracy, however most of the algorithms require points of interest and textured surfaces in order to give better results. Most algorithms will have poor performance when it comes to monotonically coloured or textureless surfaces. Furthermore, the output of the algorithms will have gaps in the projection of the structure it is trying to recreate. This kind of projection would be useless in a case where consistency and completeness of surfaces is more important than the level of detail. In this thesis the author will try to use structure from motion techniques and new ideas to create a projection of an interior room which focuses on the essence of the room (I.e aspect ratio, correct floor plan) rather than on the level of detail of objects in the room. The goal of this thesis will be to create an algorithm which can generate a projection out of a sparse point cloud (result of SfM) that is consistent enough to allow it to be used for applications that require a more complete model rather than a detailed one (I.e robot pathfinding, indoor people tracking)

Constrained Bundle Adjustment for Structure From Motion Using Uncalibrated Multi-Camera Systems

Structure from motion using uncalibrated multi-camera systems is a challenging task. This paper proposes a bundle adjustment solution that implements a baseline constraint respecting that these cameras are static to each other. We assume these cameras are mounted on a mobile platform, uncalibrated, and coarsely synchronized. To this end, we propose the baseline constraint that is formulated for the scenario in which the cameras have overlapping views. The constraint is incorporated in the bundle adjustment solution to keep the relative motion of different cameras static. Experiments were conducted using video frames of two collocated GoPro cameras mounted on a vehicle with no system calibration. These two cameras were placed capturing overlapping contents. We performed our bundle adjustment using the proposed constraint and then produced 3D dense point clouds. Evaluations were performed by comparing these dense point clouds against LiDAR reference data. We showed that, as compared to traditional bundle adjustment, our proposed method achieved an improvement of 29.38%.Comment: to be published in ISPRS Congress 202

Three dimensional information estimation and tracking for moving objects detection using two cameras framework

Calibration, matching and tracking are major concerns to obtain 3D information consisting of depth, direction and velocity. In finding depth, camera parameters and matched points are two necessary inputs. Depth, direction and matched points can be achieved accurately if cameras are well calibrated using manual traditional calibration. However, most of the manual traditional calibration methods are inconvenient to use because markers or real size of an object in the real world must be provided or known. Self-calibration can solve the traditional calibration limitation, but not on depth and matched points. Other approaches attempted to match corresponding object using 2D visual information without calibration, but they suffer low matching accuracy under huge perspective distortion. This research focuses on achieving 3D information using self-calibrated tracking system. In this system, matching and tracking are done under self-calibrated condition. There are three contributions introduced in this research to achieve the objectives. Firstly, orientation correction is introduced to obtain better relationship matrices for matching purpose during tracking. Secondly, after having relationship matrices another post-processing method, which is status based matching, is introduced for improving object matching result. This proposed matching algorithm is able to achieve almost 90% of matching rate. Depth is estimated after the status based matching. Thirdly, tracking is done based on x-y coordinates and the estimated depth under self-calibrated condition. Results show that the proposed self-calibrated tracking system successfully differentiates the location of objects even under occlusion in the field of view, and is able to determine the direction and the velocity of multiple moving objects

3D Reconstruction of Historic Landmarks from Flickr Pictures

Tato práce popisuje problematiku návrhu a vývoje aplikace pro rekonstrukci 3D modelů z 2D obrazových dat, označované jako bundle adjustment. Práce analyzuje proces 3D rekonstrukce a důkladně popisuje jednotlivé kroky. Prvním z kroků je automatizované získání obrazové sady z internetu. Je představena sada skriptů pro hromadné stahování obrázků ze služeb Flickr a Google Images a shrnuty požadavky na tyto obrázky pro co nejlepší 3D rekonstrukci. Práce dále popisuje různé detektory, extraktory a párovací algoritmy klíčových bodů v obraze s cílem najít nejvhodnější kombinaci pro rekonstrukci budov. Poté je vysvětlen proces rekonstrukce 3D struktury, její optimalizace a jak je tato problematika realizovaná v našem programu. Závěr práce testuje výsledky získané z implementovaného programu pro několik různých datových sad a porovnává je s výsledky ostatních podobných programů, představených v úvodu práce.This thesis describes challenges in design and development of an application which reconstructs 3D model given set of 2D images. This technique is called bundle adjustment. The thesi discusses the 3D reconstruction pipeline and elaborates on each step. The first step covers dataset acquisition from the internet. The scripts used to download such data from Flickr and Google Images are described and image characteristics necessary for a good reconstruction are identified. Hereafter the paper compares different feature detectors, extractors and matchers to find best suited combination for reconstruction of historic landmarks. This is followed by description the reconstruction and optimization steps and their implementation. At the end of the thesis the implemented solution is examined on several datasets and compared with other existing solutions presented at the very beginning of the thesis.