Line Based Multi-Range Asymmetric Conditional Random Field For Terrestrial Laser Scanning Data Classification

Terrestrial Laser Scanning (TLS) is a ground-based, active imaging method that rapidly acquires accurate, highly dense three-dimensional point cloud of object surfaces by laser range finding. For fully utilizing its benefits, developing a robust method to classify many objects of interests from huge amounts of laser point clouds is urgently required. However, classifying massive TLS data faces many challenges, such as complex urban scene, partial data acquisition from occlusion. To make an automatic, accurate and robust TLS data classification, we present a line-based multi-range asymmetric Conditional Random Field algorithm. The first contribution is to propose a line-base TLS data classification method. In this thesis, we are interested in seven classes: building, roof, pedestrian road (PR), tree, low man-made object (LMO), vehicle road (VR), and low vegetation (LV). The line-based classification is implemented in each scan profile, which follows the line profiling nature of laser scanning mechanism.Ten conventional local classifiers are tested, including popular generative and discriminative classifiers, and experimental results validate that the line-based method can achieve satisfying classification performance. However, local classifiers implement labeling task on individual line independently of its neighborhood, the inference of which often suffers from similar local appearance across different object classes. The second contribution is to propose a multi-range asymmetric Conditional Random Field (maCRF) model, which uses object context as post-classification to improve the performance of a local generative classifier. The maCRF incorporates appearance, local smoothness constraint, and global scene layout regularity together into a probabilistic graphical model. The local smoothness enforces that lines in a local area to have the same class label, while scene layout favours an asymmetric regularity of spatial arrangement between different object classes within long-range, which is considered both in vertical (above-bellow relation) and horizontal (front-behind) directions. The asymmetric regularity allows capturing directional spatial arrangement between pairwise objects (e.g. it allows ground is lower than building, not vice-versa). The third contribution is to extend the maCRF model by adding across scan profile context, which is called Across scan profile Multi-range Asymmetric Conditional Random Field (amaCRF) model. Due to the sweeping nature of laser scanning, the sequentially acquired TLS data has strong spatial dependency, and the across scan profile context can provide more contextual information. The final contribution is to propose a sequential classification strategy. Along the sweeping direction of laser scanning, amaCRF models were sequentially constructed. By dynamically updating posterior probability of common scan profiles, contextual information propagates through adjacent scan profiles

Automated segmentation, detection and fitting of piping elements from terrestrial LIDAR data

Since the invention of light detection and ranging (LIDAR) in the early 1960s, it has been adopted for use in numerous applications, from topographical mapping with airborne LIDAR platforms to surveying of urban sites with terrestrial LIDAR systems. Static terrestrial LIDAR has become an especially effective tool for surveying, in some cases replacing traditional techniques such as electronic total stations and GPS methods. Current state-of-the-art LIDAR scanners have very fine spatial resolution, generating precise 3D point cloud data with millimeter accuracy. Therefore, LIDAR data can provide 3D details of a scene with an unprecedented level of details. However, automated exploitation of LIDAR data is challenging, due to the non-uniform spatial sampling of the point clouds as well as to the massive volumes of data, which may range from a few million points to hundreds of millions of points depending on the size and complexity of the scene being scanned. ^ This dissertation focuses on addressing these challenges to automatically exploit large LIDAR point clouds of piping systems in industrial sites, such as chemical plants, oil refineries, and steel mills. A complete processing chain is proposed in this work, using raw LIDAR point clouds as input and generating cylinder parameter estimates for pipe segments as the output, which could then be used to produce computer aided design (CAD) models of pipes. The processing chain consists of three stages: (1) segmentation of LIDAR point clouds, (2) detection and identification of piping elements, and (3) cylinder fitting and parameter estimation. The final output of the cylinder fitting stage gives the estimated orientation, position, and radius of each detected pipe element. ^ A robust octree-based split and merge segmentation algorithm is proposed in this dissertation that can efficiently process LIDAR data. Following octree decomposition of the point cloud, graph theory analysis is used during the splitting process to separate points within each octant into components based on spatial connectivity. A series of connectivity criteria (proximity, orientation, and curvature) are developed for the merging process, which exploits contextual information to effectively merge cylindrical segments into complete pipes and planar segments into complete walls. Furthermore, by conducting surface fitting of segments and analyzing their principal curvatures, the proposed segmentation approach is capable of detecting and identifying the piping segments. ^ A novel cylinder fitting technique is proposed to accurately estimate the cylinder parameters for each detected piping segment from the terrestrial LIDAR point cloud. Specifically, the orientation, radius, and position of each piping element must be robustly estimated in the presence of noise. An original formulation has been developed to estimate the cylinder axis orientation using gradient descent optimization of an angular distance cost function. The cost function is based on the concept that surface normals of points in a cylinder point cloud are perpendicular to the cylinder axis. The key contribution of this algorithm is its capability to accurately estimate the cylinder orientation in the presence of noise without requiring a good initial starting point. After estimation of the cylinder\u27s axis orientation, the radius and position are then estimated in the 2D space formed from the projection of the 3D cylinder point cloud onto the plane perpendicular to the cylinder\u27s axis. With these high quality approximations, a least squares estimation in 3D is made for the final cylinder parameters. ^ Following cylinder fitting, the estimated parameters of each detected piping segment are used to generate a CAD model of the piping system. The algorithms and techniques in this dissertation form a complete processing chain that can automatically exploit large LIDAR point cloud of piping systems and generate CAD models

Walk2Map: Extracting Floor Plans from Indoor Walk Trajectories

Recent years have seen a proliferation of new digital products for the efficient management of indoor spaces, with important applications like emergency management, virtual property showcasing and interior design. These products rely on accurate 3D models of the environments considered, including information on both architectural and non-permanent elements. These models must be created from measured data such as RGB-D images or 3D point clouds, whose capture and consolidation involves lengthy data workflows. This strongly limits the rate at which 3D models can be produced, preventing the adoption of many digital services for indoor space management. We provide an alternative to such data-intensive procedures by presenting Walk2Map, a data-driven approach to generate floor plans only from trajectories of a person walking inside the rooms. Thanks to recent advances in data-driven inertial odometry, such minimalistic input data can be acquired from the IMU readings of consumer-level smartphones, which allows for an effortless and scalable mapping of real-world indoor spaces. Our work is based on learning the latent relation between an indoor walk trajectory and the information represented in a floor plan: interior space footprint, portals, and furniture. We distinguish between recovering area-related (interior footprint, furniture) and wall-related (doors) information and use two different neural architectures for the two tasks: an image-based Encoder-Decoder and a Graph Convolutional Network, respectively. We train our networks using scanned 3D indoor models and apply them in a cascaded fashion on an indoor walk trajectory at inference time. We perform a qualitative and quantitative evaluation using both simulated and measured, real-world trajectories, and compare against a baseline method for image-to-image translation. The experiments confirm the feasibility of our approach.Comment: To be published in Computer Graphics Forum (Proc. Eurographics 2021

A FLEXIBLE METHODOLOGY FOR OUTDOOR/INDOOR BUILDING RECONSTRUCTION FROM OCCLUDED POINT CLOUDS

Terrestrial Laser Scanning data are increasingly used in building survey not only in cultural heritage domain but also for as-built modelling of large and medium size civil structures. However, raw point clouds derived from laser scanning generally not directly ready for the generation of such models. A time-consuming manual modelling phase has to be taken into account. In addition the large presence of occlusion and clutter may turn out in low-quality building models when state-of-the-art automatic modelling procedures are applied. This paper presents an automated procedure to convert raw point clouds into semantically-enriched building models. The developed method mainly focuses on a geometrical complexity typical of modern buildings with clear prevalence of planar features A characteristic of this methodology is the possibility to work with outdoor and indoor building environments. In order to operate under severe occlusions and clutter a couple of completion algorithms were designed to generate a plausible and reliable model. Finally, some examples of the developed modelling procedure are presented and discussed

Walk2Map: Extracting Floor Plans from Indoor Walk Trajectories

Recent years have seen a proliferation of new digital products for the efficient management of indoor spaces, with important applications like emergency management, virtual property showcasing and interior design. While highly innovative and effective, these products rely on accurate 3D models of the environments considered, including information on both architectural and non-permanent elements. These models must be created from measured data such as RGB-D images or 3D point clouds, whose capture and consolidation involves lengthy data workflows. This strongly limits the rate at which 3D models can be produced, preventing the adoption of many digital services for indoor space management. We provide a radical alternative to such data-intensive procedures by presentingWalk2Map, a data-driven approach to generate floor plans only from trajectories of a person walking inside the rooms. Thanks to recent advances in data-driven inertial odometry, such minimalistic input data can be acquired from the IMU readings of consumer-level smartphones, which allows for an effortless and scalable mapping of real-world indoor spaces. Our work is based on learning the latent relation between an indoor walk trajectory and the information represented in a floor plan: interior space footprint, portals, and furniture. We distinguish between recovering area-related (interior footprint, furniture) and wall-related (doors) information and use two different neural architectures for the two tasks: an image-based Encoder-Decoder and a Graph Convolutional Network, respectively. We train our networks using scanned 3D indoor models and apply them in a cascaded fashion on an indoor walk trajectory at inference time. We perform a qualitative and quantitative evaluation using both trajectories simulated from scanned models of interiors and measured, real-world trajectories, and compare against a baseline method for image-to-image translation. The experiments confirm that our technique is viable and allows recovering reliable floor plans from minimal walk trajectory data

Walk2Map: Extracting Floor Plans from Indoor Walk Trajectories

Recent years have seen a proliferation of new digital products for the efficient management of indoor spaces, with important applications like emergency management, virtual property showcasing and interior design. While highly innovative and effective, these products rely on accurate 3D models of the environments considered, including information on both architectural and non-permanent elements. These models must be created from measured data such as RGB-D images or 3D point clouds, whose capture and consolidation involves lengthy data workflows. This strongly limits the rate at which 3D models can be produced, preventing the adoption of many digital services for indoor space management. We provide a radical alternative to such data-intensive procedures by presenting Walk2Map, a data-driven approach to generate floor plans only from trajectories of a person walking inside the rooms. Thanks to recent advances in data-driven inertial odometry, such minimalistic input data can be acquired from the IMU readings of consumer-level smartphones, which allows for an effortless and scalable mapping of real-world indoor spaces. Our work is based on learning the latent relation between an indoor walk trajectory and the information represented in a floor plan: interior space footprint, portals, and furniture. We distinguish between recovering area-related (interior footprint, furniture) and wall-related (doors) information and use two different neural architectures for the two tasks: an image-based Encoder-Decoder and a Graph Convolutional Network, respectively. We train our networks using scanned 3D indoor models and apply them in a cascaded fashion on an indoor walk trajectory at inference time. We perform a qualitative and quantitative evaluation using both trajectories simulated from scanned models of interiors and measured, real-world trajectories, and compare against a baseline method for image-to-image translation. The experiments confirm that our technique is viable and allows recovering reliable floor plans from minimal walk trajectory data

Hierarchical structure-and-motion recovery from uncalibrated images

This paper addresses the structure-and-motion problem, that requires to find camera motion and 3D struc- ture from point matches. A new pipeline, dubbed Samantha, is presented, that departs from the prevailing sequential paradigm and embraces instead a hierarchical approach. This method has several advantages, like a provably lower computational complexity, which is necessary to achieve true scalability, and better error containment, leading to more stability and less drift. Moreover, a practical autocalibration procedure allows to process images without ancillary information. Experiments with real data assess the accuracy and the computational efficiency of the method.Comment: Accepted for publication in CVI

Istraživanje i modeliranje nepoznatog poligonalnog prostora zasnovano na nesigurnim podacima udaljenosti

We consider problem of exploration and mapping of unknown indoor environments using laser range finder. We assume a setup with a resolved localization problem and known uncertainty sensor models. Most exploration algorithms are based on detection of a boundary between explored and unexplored regions. They are, however, not efficient in practice due to uncertainties in measurement, localization and map building. The exploration and mapping algorithm is proposed that extends Ekman’s exploration algorithm by removing rigid constraints on the range sensor and robot localization. The proposed algorithm includes line extraction algorithm developed by Pfister, which incorporates noise models of the range sensor and robot’s pose uncertainty. A line representation of the range data is used for creating polygon that represents explored region from each measurement pose. The polygon edges that do not correspond to real environmental features are candidates for a new measurement pose. A general polygon clipping algorithm is used to obtain the total explored region as the union of polygons from different measurement poses. The proposed algorithm is tested and compared to the Ekman’s algorithm by simulations and experimentally on a Pioneer 3DX mobile robot equipped with SICK LMS-200 laser range finder.Razmatramo problem istraživanja i izgradnje karte nepoznatog unutarnjeg prostora koristeći laserski sensor udaljenosti. Pretpostavljamo riješenu lokalizaciju robota i poznati model nesigurnosti senzora. Većna se algoritama istraživanja zasniva na otkrivanju granica istraženog i neistraženog područja. Međutim, u praksi nisu učinkoviti zbog nesigurnosti mjerenja, lokalizacije i izgradnje karte. Razvijen je algoritam istraživanja i izgradnje karte koji proširuje Ekmanov algoritam uklanjanjem strogih ograničenja na senzor udaljenosti i lokalizaciju robota. Razvijeni algoritam uključuje algoritam izdvajanja linijskih segmenata prema Pfisteru, koji uzima u obzir utjecaje zašumljenosti senzora i nesigurnosti položaja mobilnog robota. Linijska reprezentacija podataka udaljenosti koristi se za stvaranje poligona koji predstavlja istraženo područje iz svakog mjernog položaja. Bridovi poligona koji se ne podudaraju sa stvarnim značajkama prostora su kandidati za novi mjerni položaj. Algoritam općenitog isijecanja poligona korišten je za dobivanje ukupnog istraženog područja kao unija poligona iz različitih mjernih položaja. Razvijeni algoritam testiran je i uspoređen s izvornim Ekmanovim algoritmom simulacijski i eksperimentalno na mobilnom robotu Pioneer 3DX opremljenim laserskim senzorom udaljenosti SICK LMS-200

Machine Learning Methods for Image Analysis in Medical Applications, from Alzheimer\u27s Disease, Brain Tumors, to Assisted Living

Healthcare has progressed greatly nowadays owing to technological advances, where machine learning plays an important role in processing and analyzing a large amount of medical data. This thesis investigates four healthcare-related issues (Alzheimer\u27s disease detection, glioma classification, human fall detection, and obstacle avoidance in prosthetic vision), where the underlying methodologies are associated with machine learning and computer vision. For Alzheimer’s disease (AD) diagnosis, apart from symptoms of patients, Magnetic Resonance Images (MRIs) also play an important role. Inspired by the success of deep learning, a new multi-stream multi-scale Convolutional Neural Network (CNN) architecture is proposed for AD detection from MRIs, where AD features are characterized in both the tissue level and the scale level for improved feature learning. Good classification performance is obtained for AD/NC (normal control) classification with test accuracy 94.74%. In glioma subtype classification, biopsies are usually needed for determining different molecular-based glioma subtypes. We investigate non-invasive glioma subtype prediction from MRIs by using deep learning. A 2D multi-stream CNN architecture is used to learn the features of gliomas from multi-modal MRIs, where the training dataset is enlarged with synthetic brain MRIs generated by pairwise Generative Adversarial Networks (GANs). Test accuracy 88.82% has been achieved for IDH mutation (a molecular-based subtype) prediction. A new deep semi-supervised learning method is also proposed to tackle the problem of missing molecular-related labels in training datasets for improving the performance of glioma classification. In other two applications, we also address video-based human fall detection by using co-saliency-enhanced Recurrent Convolutional Networks (RCNs), as well as obstacle avoidance in prosthetic vision by characterizing obstacle-related video features using a Spiking Neural Network (SNN). These investigations can benefit future research, where artificial intelligence/deep learning may open a new way for real medical applications