Quality Enhancement of 3D Models Reconstructed By RGB-D Camera Systems

Low-cost RGB-D cameras like Microsoft\u27s Kinect capture RGB data for each vertex while reconstructing 3D models from objects with obvious drawbacks of poor mesh and texture qualities due to their hardware limitations. In this thesis we propose a combined method that enhances geometrically and chromatically 3D models reconstructed by RGB-D camera systems. Our approach utilizes Butterfly Subdivision and Surface Fitting techniques to generate smoother triangle surface meshes, where sharp features can be well preserved or minimized by different Surface Fitting algorithms. Additionally the global contrast of mesh textures is enhanced by using a modified Histogram Equalization algorithm, in which the new intensity of each vertex is obtained by applying cumulative distribution function and calculating the accumulated normalized histogram of the texture. A number of experimental results and comparisons demonstrate that our method efficiently and effectively improves the geometric and chromatic quality of 3D models reconstructed from RGB-D cameras

Semantically Guided Depth Upsampling

We present a novel method for accurate and efficient up- sampling of sparse depth data, guided by high-resolution imagery. Our approach goes beyond the use of intensity cues only and additionally exploits object boundary cues through structured edge detection and semantic scene labeling for guidance. Both cues are combined within a geodesic distance measure that allows for boundary-preserving depth in- terpolation while utilizing local context. We model the observed scene structure by locally planar elements and formulate the upsampling task as a global energy minimization problem. Our method determines glob- ally consistent solutions and preserves fine details and sharp depth bound- aries. In our experiments on several public datasets at different levels of application, we demonstrate superior performance of our approach over the state-of-the-art, even for very sparse measurements.Comment: German Conference on Pattern Recognition 2016 (Oral

Virtuaalse proovikabiini 3D kehakujude ja roboti juhtimisalgoritmide uurimine

Väitekirja elektrooniline versioon ei sisalda publikatsiooneVirtuaalne riiete proovimine on üks põhilistest teenustest, mille pakkumine võib suurendada rõivapoodide edukust, sest tänu sellele lahendusele väheneb füüsilise töö vajadus proovimise faasis ning riiete proovimine muutub kasutaja jaoks mugavamaks. Samas pole enamikel varem välja pakutud masinnägemise ja graafika meetoditel õnnestunud inimkeha realistlik modelleerimine, eriti terve keha 3D modelleerimine, mis vajab suurt kogust andmeid ja palju arvutuslikku ressurssi. Varasemad katsed on ebaõnnestunud põhiliselt seetõttu, et ei ole suudetud korralikult arvesse võtta samaaegseid muutusi keha pinnal. Lisaks pole varasemad meetodid enamasti suutnud kujutiste liikumisi realistlikult reaalajas visualiseerida. Käesolev projekt kavatseb kõrvaldada eelmainitud puudused nii, et rahuldada virtuaalse proovikabiini vajadusi. Välja pakutud meetod seisneb nii kasutaja keha kui ka riiete skaneerimises, analüüsimises, modelleerimises, mõõtmete arvutamises, orientiiride paigutamises, mannekeenidelt võetud 3D visuaalsete andmete segmenteerimises ning riiete mudeli paigutamises ja visualiseerimises kasutaja kehal. Selle projekti käigus koguti visuaalseid andmeid kasutades 3D laserskannerit ja Kinecti optilist kaamerat ning koostati nendest andmebaas. Neid andmeid kasutati välja töötatud algoritmide testimiseks, mis peamiselt tegelevad riiete realistliku visuaalse kujutamisega inimkehal ja suuruse pakkumise süsteemi täiendamisega virtuaalse proovikabiini kontekstis.Virtual fitting constitutes a fundamental element of the developments expected to rise the commercial prosperity of online garment retailers to a new level, as it is expected to reduce the load of the manual labor and physical efforts required. Nevertheless, most of the previously proposed computer vision and graphics methods have failed to accurately and realistically model the human body, especially, when it comes to the 3D modeling of the whole human body. The failure is largely related to the huge data and calculations required, which in reality is caused mainly by inability to properly account for the simultaneous variations in the body surface. In addition, most of the foregoing techniques cannot render realistic movement representations in real-time. This project intends to overcome the aforementioned shortcomings so as to satisfy the requirements of a virtual fitting room. The proposed methodology consists in scanning and performing some specific analyses of both the user's body and the prospective garment to be virtually fitted, modeling, extracting measurements and assigning reference points on them, and segmenting the 3D visual data imported from the mannequins. Finally, superimposing, adopting and depicting the resulting garment model on the user's body. The project is intended to gather sufficient amounts of visual data using a 3D laser scanner and the Kinect optical camera, to manage it in form of a usable database, in order to experimentally implement the algorithms devised. The latter will provide a realistic visual representation of the garment on the body, and enhance the size-advisor system in the context of the virtual fitting room under study

SceneFlowFields: Dense Interpolation of Sparse Scene Flow Correspondences

While most scene flow methods use either variational optimization or a strong rigid motion assumption, we show for the first time that scene flow can also be estimated by dense interpolation of sparse matches. To this end, we find sparse matches across two stereo image pairs that are detected without any prior regularization and perform dense interpolation preserving geometric and motion boundaries by using edge information. A few iterations of variational energy minimization are performed to refine our results, which are thoroughly evaluated on the KITTI benchmark and additionally compared to state-of-the-art on MPI Sintel. For application in an automotive context, we further show that an optional ego-motion model helps to boost performance and blends smoothly into our approach to produce a segmentation of the scene into static and dynamic parts.Comment: IEEE Winter Conference on Applications of Computer Vision (WACV), 201

Acquisition of Surface Light Fields from Videos

La tesi presenta un nuovo approccio per la stima di Surface Light Field di oggetti reali, a partire da sequenze video acquisite in condizioni di illuminazione fisse e non controllate. Il metodo proposto si basa sulla separazione delle due componenti principali dell'apparenza superficiale dell'oggetto: la componente diffusiva, modellata come colore RGB, e la componente speculare, approssimata mediante un modello parametrico funzione della posizione dell'osservatore. L'apparenza superficiale ricostruita permette una visualizzazione fotorealistica e in real-time dell'oggetto al variare della posizione dell'osservatore, consentendo una navigazione 3D interattiva

Machine Learning Approach to Retrieving Physical Variables from Remotely Sensed Data

Scientists from all over the world make use of remotely sensed data from hundreds of satellites to better understand the Earth. However, physical measurements from an instrument is sometimes missing either because the instrument hasn\u27t been launched yet or the design of the instrument omitted a particular spectral band. Measurements received from the instrument may also be corrupt due to malfunction in the detectors on the instrument. Fortunately, there are machine learning techniques to estimate the missing or corrupt data. Using these techniques we can make use of the available data to its full potential. We present work on four different problems where the use of machine learning techniques helps to extract more information from available data. We demonstrate how missing or corrupt spectral measurements from a sensor can be accurately interpolated from existing spectral observations. Sometimes this requires data fusion from multiple sensors at different spatial and spectral resolution. The reconstructed measurements can then be used to develop products useful to scientists, such as cloud-top pressure, or produce true color imagery for visualization. Additionally, segmentation and image processing techniques can help solve classification problems important for ocean studies, such as the detection of clear-sky over ocean for a sea surface temperature product. In each case, we provide detailed analysis of the problem and empirical evidence that these problems can be solved effectively using machine learning techniques

Application of TLS intensity data for detection of brick walls defects

Terrestrial Laser Scanning (TLS) is a well-established technique for remote acquisition of geometrical data of a tested object. For the past two decades it has been commonly used in geodesy, surveying and related areas for acquiring data about spacing of civil engineering structures and buildings. An average TLS apparatus, apart from 3D coordinates registers radiometric information of laser beam reflectance. This radiometric information of the laser beam reflectance is usually called intensity and has no meaning for solely geometric measurements. Nevertheless, the value of intensity depends mainly on physicochemical properties of scanned material such as roughness, colour and saturation. Keeping these facts in mind, authors suggest using changes in value of intensity to locate various imperfections on a brick wall. So far, authors have conducted a thorough and successful research programme dedicated to detection of saturation and saturation movement in brick walls. Based on this experience a new research programme was conducted focused on various aspects of detection of brick wall defects. The main aim of the paper is to present the possibility of using the intensity value in for the diagnostics of the technical condition of a brick walls. Advantages and limitations of harnessing TLS for detection of surface defects of brick walls are presented and discussed in the paper.Peer ReviewedPostprint (published version

P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior

Monocular depth estimation is vital for scene understanding and downstream tasks. We focus on the supervised setup, in which ground-truth depth is available only at training time. Based on knowledge about the high regularity of real 3D scenes, we propose a method that learns to selectively leverage information from coplanar pixels to improve the predicted depth. In particular, we introduce a piecewise planarity prior which states that for each pixel, there is a seed pixel which shares the same planar 3D surface with the former. Motivated by this prior, we design a network with two heads. The first head outputs pixel-level plane coefficients, while the second one outputs a dense offset vector field that identifies the positions of seed pixels. The plane coefficients of seed pixels are then used to predict depth at each position. The resulting prediction is adaptively fused with the initial prediction from the first head via a learned confidence to account for potential deviations from precise local planarity. The entire architecture is trained end-to-end thanks to the differentiability of the proposed modules and it learns to predict regular depth maps, with sharp edges at occlusion boundaries. An extensive evaluation of our method shows that we set the new state of the art in supervised monocular depth estimation, surpassing prior methods on NYU Depth-v2 and on the Garg split of KITTI. Our method delivers depth maps that yield plausible 3D reconstructions of the input scenes. Code is available at: https://github.com/SysCV/P3DepthComment: Accepted at CVPR 202