A review of laser scanning for geological and geotechnical applications in underground mining

Laser scanning can provide timely assessments of mine sites despite adverse challenges in the operational environment. Although there are several published articles on laser scanning, there is a need to review them in the context of underground mining applications. To this end, a holistic review of laser scanning is presented including progress in 3D scanning systems, data capture/processing techniques and primary applications in underground mines. Laser scanning technology has advanced significantly in terms of mobility and mapping, but there are constraints in coherent and consistent data collection at certain mines due to feature deficiency, dynamics, and environmental influences such as dust and water. Studies suggest that laser scanning has matured over the years for change detection, clearance measurements and structure mapping applications. However, there is scope for improvements in lithology identification, surface parameter measurements, logistic tracking and autonomous navigation. Laser scanning has the potential to provide real-time solutions but the lack of infrastructure in underground mines for data transfer, geodetic networking and processing capacity remain limiting factors. Nevertheless, laser scanners are becoming an integral part of mine automation thanks to their affordability, accuracy and mobility, which should support their widespread usage in years to come

Optimal scan planning for surveying large sites with static and mobile mapping systems

Since the last two decades, the use of laser scanners for generating accurate and dense 3D models has been rapidly growing in multiple disciplines. The reliance on human-expertise to perform an efficient scanning in terms of completeness and quality encouraged the researchers to develop strategies for carrying out an optimized and automated scan planning. Nevertheless, due to the predominant use of static terrestrial laser scanners (TLS), the most of developed methods have been focused on scan optimization by fixing standpoints on basis of static scanning. The increasing use of portable mobile laser scanning systems (MLS) enables faster non-stop acquisition which demands the planning of optimal scan trajectories. Therefore, a novel method addressing the absence of dynamic scan planning is proposed considering specific MLS constraints such as maximum acquisition time or closed-loops requirement. First, an initial analysis is carried out to determinate key-positions to reach during data acquisition. From these positions a navigable graph is generated to compute routes satisfying specific MLS constraints by a three-step process. This starts by estimating the number of routes necessary to subsequently carry out a coarse graph partition based on Kmedoids clustering. Next, a balancing algorithm was implemented to compute a balanced graph partition by node exchanging. Finally, partitions are extended by adding key nodes from their adjacent ones in order to provide a desirable overlapping between scans. The method was tested by simulating three laser scanner configurations in four indoor and outdoor real case studies. The acquisition quality of the computed scan planning was evaluated in terms of 3D completeness and point cloud density with the simulator Helios++

optimización da planificación de adquisición de datos LIDAR cara ó modelado 3D de interiores

The main objective of this doctoral thesis is the design, validation and implementation of methodologies that allow the geometric and topological modelling of navigable spaces, whether inside buildings or urban environments, to be integrated into three-dimensional geographic information systems (GIS-3D). The input data of this work will consist mainly of point clouds (which can be classified) acquired by LiDAR systems both indoors and outdoors. In addition, the use of BIM infrastructure models and cadastral maps is proposed depending on their availability. Point clouds provide a large amount of environmental information with high accuracy compared to data offered by other acquisition technologies. However, the lack of data structure and volume requires a great deal of processing effort. For this reason, the first step is to structure the data by dividing the input cloud into simpler entities that facilitate subsequent processes. For this first division, the physical elements present in the cloud will be considered, since they can be walls in the case of interior environments or kerbs in the case of exteriors. In order to generate navigation routes adapted to different mobile agents, the next objective will try to establish a semantic subdivision of space according to the functionalities of space. In the case of internal environments, it is possible to use BIM models to evaluate the results and the use of cadastral maps that support the division of the urban environment. Once the navigable space is divided, the design of topologically coherent navigation networks will be parameterized both geometrically and topologically. For this purpose, several spatial discretization techniques, such as 3D tessellations, will be studied to facilitate the establishment of topological relationships, adjacency, connectivity and inclusion between subspaces. Based on the geometric characterization and the topological relations established in the previous phase, the creation of three-dimensional navigation networks with multimodal support will be addressed and different levels of detail will be considered according to the mobility specifications of each agent and its purpose. Finally, the possibility of integrating the networks generated in a GIS-3D visualization system will be considered. For the correct visualization, the level of detail can be adjusted according to geometry and semantics. Aspects such as the type of user or transport, mobility, rights of access to spaces, etc. They must be considered at all times.El objetivo principal de esta tesis doctoral es el diseño, la validación y la implementación de metodologías que permitan el modelado geométrico y topológico de espacios navegables, ya sea de interiores de edificios o entornos urbanos, para integrarse en sistemas de información geográfica tridimensional (SIG). -3D). Los datos de partida de este trabajo consistirán principalmente en nubes de puntos (que pueden estar clasificados) adquiridas por sistemas LiDAR tanto en interiores como en exteriores. Además, se propone el uso de modelos BIM de infraestructuras y mapas catastrales en función de su disponibilidad. Las nubes de puntos proporcionan una gran cantidad de información del entorno con gran precisión con respecto a los datos ofrecidos por otras tecnologías de adquisición. Sin embargo, la falta de estructura de datos y su volumen requiere un gran esfuerzo de procesamiento. Por este motivo, el primer paso que se debe realizar consiste en estructurar los datos dividiendo la nube de entrada en entidades más simples que facilitan los procesos posteriores. Para esta primera división se considerarán los elementos físicos presentes en la nube, ya que pueden ser paredes en el caso de entornos interiores o bordillos en el caso de los exteriores. Con el propósito de generar rutas de navegación adaptadas a diferentes agentes móviles, el próximo objetivo intentará establecer una subdivisión semántica del espacio de acuerdo con las funcionalidades del espacio. En el caso de entornos internos, es posible utilizar modelos BIM para evaluar los resultados y el uso de mapas catastrales que sirven de apoyo en la división del entorno urbano. Una vez que se divide el espacio navegable, se parametrizará tanto geométrica como topológicamente al diseño de redes de navegación topológicamente coherentes. Para este propósito, se estudiarán varias técnicas de discretización espacial, como las teselaciones 3D, para facilitar el establecimiento de relaciones topológicas, la adyacencia, la conectividad y la inclusión entre subespacios. A partir de la caracterización geométrica y las relaciones topológicas establecidas en la fase anterior, se abordará la creación de redes de navegación tridimensionales con soporte multimodal y se considerarán diversos niveles de detalle según las especificaciones de movilidad de cada agente y su propósito. Finalmente, se contemplará la posibilidad de integrar las redes generadas en un sistema de visualización tridimensional 3D SIG 3D. Para la correcta visualización, el nivel de detalle se puede ajustar en función de la geometría y la semántica. Aspectos como el tipo de usuario o transporte, movilidad, derechos de acceso a espacios, etc. Deben ser considerados en todo momento.O obxectivo principal desta tese doutoral é o deseño, validación e implementación de metodoloxías que permitan o modelado xeométrico e topolóxico de espazos navegables, ben sexa de interiores de edificios ou de entornos urbanos, ca fin de seren integrados en Sistemas de Información Xeográfica tridimensionais (SIX-3D). Os datos de partida deste traballo constarán principalmente de nubes de puntos (que poden estar clasificadas) adquiridas por sistemas LiDAR tanto en interiores como en exteriores. Ademáis plantease o uso de modelos BIM de infraestruturas e mapas catastrais dependendo da súa dispoñibilidade. As nubes de puntos proporcionan unha gran cantidade de información do entorno cunha gran precisión respecto os datos que ofrecen outras tecnoloxías de adquisición. Sen embargo, a falta de estrutura dos datos e a seu volume esixe un amplo esforzo de procesado. Por este motivo o primeiro paso a levar a cabo consiste nunha estruturación dos datos mediante a división da nube de entrada en entidades máis sinxelas que faciliten os procesos posteriores. Para esta primeira división consideraranse elementos físicos presentes na nube como poden ser paredes no caso de entornos interiores ou bordillos no caso de exteriores. Coa finalidade de xerar rutas de navegación adaptadas a distintos axentes móbiles, o seguinte obxectivo tratará de establecer unha subdivisión semántica do espazo de acordo as funcionalidades do espazo. No caso de entornos interiores plantease a posibilidade de empregar modelos BIM para avaliar os resultados e o uso de mapas catastrais que sirvan de apoio na división do entorno urbano. Unha vez divido o espazo navigable parametrizarase tanto xeométricamente como topolóxicamene de cara ao deseño de redes de navegación topolóxicamente coherentes. Para este fin estudaranse varias técnicas de discretización de espazos como como son as teselacións 3D co obxectivo de facilitar establecer relacións topolóxicas, de adxacencia, conectividade e inclusión entre subespazos. A partir da caracterización xeométrica e das relación topolóxicas establecidas na fase previa abordarase a creación de redes de navegación tridimensionais con soporte multi-modal e considerando varios niveis de detalle de acordo as especificacións de mobilidade de cada axente e a súa finalidade. Finalmente comtemplarase a posibilidade de integrar as redes xeradas nun sistema SIX 3D visualización tridimensional. Para a correcta visualización o nivel de detalle poderá axustarse en base a xeometría e a semántica. Aspectos como o tipo de usuario ou transporte, mobilidade, dereitos de acceso a espazos, etc. deberán ser considerados en todo momento

The Burning Bush: Linking LiDAR-derived Shrub Architecture to Flammability

Light detection and ranging (LiDAR) and terrestrial laser scanning (TLS) sensors are powerful tools for characterizing vegetation structure and for constructing three-dimensional (3D) models of trees, also known as quantitative structural models (QSM). 3D models and structural traits derived from them provide valuable information for biodiversity conservation, forest management, and fire behavior modeling. However, vegetation studies and 3D modeling methodologies often only focus on the forest canopy, with little attention given to understory vegetation. In particular, 3D structural information of shrubs is limited or not included in fire behavior models. Yet, understory vegetation is an important component of forested ecosystems, and has an essential role in determining fire behavior. In this dissertation, I explored the use of TLS data and quantitative structure models to model shrub architecture in three related studies. In the first study, I present a semi-automated methodology for reconstructing architecturally different shrubs from TLS LiDAR. By investigating shrubs with different architectures and point cloud densities, I showed that occlusion, shrub complexity, and shape greatly affect the accuracy of shrub models. In my second study, I assessed the 3D architectural drivers of understory flammability by evaluating the use of architectural metrics derived from the TLS point cloud and 3D reconstructions of the shrubs. I focused on eight species common in the understory of the fire-prone longleaf pine forest ecosystem of the state of Florida, USA. I found a general tendency for each species to be associated with a unique combination of flammability and architectural traits. Novel shrub architectural traits were found to be complementary to the direct use of TLS data and improved flammability predictions. The inherent complexity of shrub architecture and uncertainty in the TLS point cloud make scaling up from an individual shrub to a plot level a challenging task. Therefore, in my third study, I explored the effects of lidar uncertainty on vegetation parameter prediction accuracy. I developed a practical workflow to create synthetic forest stands with varying densities, which were subsequently scanned with simulated terrestrial lidar. This provided data sets quantitatively similar to those created by real-world LiDAR measurements, but with the advantage of exact knowledge of the forest plot parameters, The results showed that the lidar scan location had a large effect on prediction accuracy. Furthermore, occlusion is strongly related to the sampling density and plot complexity. The results of this study illustrate the potential of non-destructive lidar approaches for quantifying shrub architectural traits. TLS, empirical quantitative structural models, and synthetic models provide valuable insights into shrub structure and fire behavior

3D model for indoor spaces using depth sensor

In recent years, 3D model for indoor spaces have become highly demanded in the development of technology. Many approaches to 3D visualisation and modelling especially for indoor environment was developed such as laser scanner, photogrammetry, computer vision, image and many more. However, most of the technique relies on the experience of the operator to get the best result. Besides that, the equipment is quite expensive and time-consuming in terms of processing. This paper focuses on the data acquisition and visualisation of a 3D model for an indoor space by using a depth sensor. In this study, EyesMap3D Pro by Ecapture is used to collect 3D data of the indoor spaces. The EyesMap3D Pro depth sensor is able to generate 3D point clouds in high speed and high mobility due to the portability and light weight of the device. However, more attention must be paid on data acquisition, data processing, visualizing, and evaluation of the depth sensor data. Hence, this paper will discuss the data processing from extracting features from 3D point clouds to 3D indoor models. Afterwards, the evaluation on the 3D models is made to ensure the suitability in indoor model and indoor mapping application. In this study, the 3D model was exported to 3D GIS-ready format for displaying and storing more information of the indoor spaces

Point Cloud-based Deep Learning and UAV Path Planning for Surface Defect Detection of Concrete Bridges

Over the past decades, several bridges have collapsed, causing many losses due to the lack of proper monitoring and inspection. Although several new techniques have been developed to detect bridge defects, annual visual inspection remains the main approach. Visual inspection, using naked eyes, is time-consuming and subjective because of human errors. Light Detection and Ranging (LiDAR) scanning is a new technology to collect 3D point clouds. The main strength of point clouds over 2D images is collecting the third dimension of the scanned objects. Deep Learning (DL)-based methods have attracted the researchers’ attention for concrete surface defect detection. However, no point cloud-based DL method is currently available for semantic segmentation of bridge surface defects without converting the raw point cloud dataset into other representations, which results in increasing the size of the dataset and leads to some challenges regarding storage capacity, cost, and training time. Some promising point cloud-based semantic segmentation methods (i.e., PointNet and PointNet++) have been applied in segmenting bridge components (i.e., slabs, piers), but not for segmenting surface defects (i.e., cracks, spalls). Moreover, most of the current point cloud-based concrete surface defect detection methods focus on only one type of defects. On the other hand, in DL, a dataset plays a key role in terms of variety, diversity, accuracy, and size. The lack of publicly available point cloud datasets for bridge surface defects is one of the reasons of the lack of studies in the area of point cloud-based methods. Furthermore, compared with terrestrial LiDAR scanning, LiDAR-equipped Unmanned Aerial Vehicle (UAV) is capable of scanning the inaccessible surfaces of the bridges at a closer distance with higher safety. Although the UAV flying path can be controlled using remote controllers, automating and optimizing UAV path planning is preferable for being able to trace a collision-free path with minimum flight time. To increase the efficiency and accuracy of this approach, it is crucial to scan all parts of the bridge with a near perpendicular view. However, in the case of obstacle existence (e.g., bridge piers), achieving full coverage with near perpendicular view may not be possible. To provide more accurate results, using overlapping views is recommended. However, this method could result in increasing the inspection cost and time. Therefore, overlapping views should be considered only for surface areas where defects are expected. Addressing the above issues, this research aims to: (1) create a publicly available point cloud dataset for concrete bridge surface defect semantic segmentation, (2) develop a point cloud-based semantic segmentation DL method to detect different types of concrete surface defects, and (3) propose a novel near-optimal path planning method for LiDAR-equipped UAV with respect to the minimum path length and maximum coverage considering the potential locations of defects. On this premise, a point cloud-based DL method for semantic segmentation of concrete bridge surface defects (i.e., cracks and spalls), called SNEPointNet++, is developed. To have a network with high-performance, SNEPointNet++ focuses on two main characteristics related to surface defects (i.e., normal vector and depth) and takes into account the issues related to the point cloud dataset (i.e., small size and imbalanced dataset). Sensitivity analysis is applied to capture the best combination of hyperparameters and investigate their effects on network performance. The dataset, which was collected from four concrete bridges, was annotated, augmented, and classified into three classes: cracks, spalls, and no defect. This dataset is made available for other researchers. The model was trained and evaluated using 60% and 20% of the dataset, respectively. Testing on the remaining part of the dataset resulted in 93% recall (69% IoU) and 92% recall (82.5% IoU) for cracks and spalls, respectively. Moreover, the results show that the spalls of the segments deeper than 7 cm (severe spalls) can be detected with 99% recall. On the other hand, this research proposes a 3D path planning method for using a UAV equipped with a LiDAR for bridge inspection to have efficient data collection. The method integrates a Genetic Algorithm (GA) and A* algorithm to solve the Traveling Salesman Problem (TSP), considering the potential locations of bridge surface defects such as cracks. The objective is to minimize the time of flight while achieving maximum visibility. The method provides the potential locations of surface defects to efficiently achieve perpendicular and overlapping views for sampling the viewpoints. Calculating the visibility with respect to the level of criticality leads to giving the priority to covering the areas with higher risk levels. Applying the proposed method on a 3-span bridge in Alberta, the results reveal that considering overlapping views based on the level of criticality of the zones and perpendicular views for all viewpoints leads to accurate and time-efficient data collection

Engineering Support Systems for Industrial Machines and Plants

In the business of industrial machines and plants, rapid and detailed estimates for planning installation, replacement of equipment, or maintenance work are key requirements for meeting the demands for greater reliability, lower costs and for maintaining safe and secure operation. These demands have been addressed by developing technology driven by IT. When replacing equipment at complex building or plants with high equipment density, the existing state of the installation locations and transportation routes for old and new equipment need to be properly measured. We have met this need by developing parts recognition technology based on 3D measurement, and by developing high-speed calculation technology of optimal routes for installation parts. This chapter provides an overview of these development projects with some real business application results

Planning for scanning in construction : optimizing 3D laser scanning operations using building information modelling and a novel specification on surface scanning completeness.

Application of Terrestrial Laser Scanning (TLS) technology in the Architectural Engineering and Construction (AEC) industry is gaining popularity because the technology uniquely offers the means to create as-built three-dimensional (3D) models of existing facilities, and conduct construction project progress and dimensional quality measurements. An open challenge with regard to the use TLS for such applications is to efficiently generate effective scanning plans that satisfy pre-defined point cloud quality specifications. Two such specifications are currently commonly used: Level of Accuracy (LOA) that focuses on individual point precision, and Level of Detail (LOD) that focuses on point density. Given such specifications, current practice sees professionals manually prepare scanning plans using existing 2D CAD drawings, some ad-hoc rules (of thumb), and their experience. Yet, it is difficult to manually generate and analyse laser scanning plans to ensure they satisfy scanning quality specifications such as those above. Manually-defined plans may easily lead to over-scanning, or worse under-scanning with incomplete data (which may require the team to go back on site to acquire complementary data). To minimize the risk of producing inadequate scanning plans, some semi-automated and automated methods have been proposed by researchers that use the 3D (BIM) model generated during the design stage. These methods take consideration for LOA and LOD. However, these are point-based specifications that do not guarantee that a sufficient amount of the surface of each object is covered by the acquired data, despite this aspect being important to many of the applications for which TLS is employed (e.g. modelling existing facilities). Therefore, this research uniquely proposes a novel planning for scanning quality specification, called Level of Surface Completeness (LOC) that assesses point cloud quality in terms of surface completeness. In addition, an approach is proposed for automatic planning for scanning in the AEC industry that takes both LOA and LOC specifications into account. The approach is ‘generic’ in the sense that it can be employed for any type of project. It is designed to generate automatic laser scanning plans using as input: (1) the facility’s 3D BIM model; (2) the scanner’s characteristics; and (3) the LOA and LOC specifications. The output is the smallest set of scanning locations necessary to achieve those requirements. The optimal solution is found by formulating the problem as a binary integer programming optimization problem, which is easily solved using a branch-and-cut algorithm. To assess the performance of the approach, experiments are conducted using a simple concrete structural model, a more complex structural model, and a section of the latter extended with Mechanical Electrical and Plumbing (MEP) components. The overall performance of the proposed approach for automatic planning for scanning is encouraging, showing that it is possible to take surface-based specifications into account in automated planning-for-scanning algorithms. However, the experimental results also highlight a significant weakness of the approach presented here which is that it does not take into account the overlapping of surfaces covered from different scanning locations and thus may inaccurately assess covered surfaces