Development of Mining Sector Applications for Emerging Remote Sensing and Deep Learning Technologies

This thesis uses neural networks and deep learning to address practical, real-world problems in the mining sector. The main focus is on developing novel applications in the area of object detection from remotely sensed data. This area has many potential mining applications and is an important part of moving towards data driven strategic decision making across the mining sector. The scientific contributions of this research are twofold; firstly, each of the three case studies demonstrate new applications which couple remote sensing and neural network based technologies for improved data driven decision making. Secondly, the thesis presents a framework to guide implementation of these technologies in the mining sector, providing a guide for researchers and professionals undertaking further studies of this type. The first case study builds a fully connected neural network method to locate supporting rock bolts from 3D laser scan data. This method combines input features from the remote sensing and mobile robotics research communities, generating accuracy scores up to 22% higher than those found using either feature set in isolation. The neural network approach also is compared to the widely used random forest classifier and is shown to outperform this classifier on the test datasets. Additionally, the algorithms’ performance is enhanced by adding a confusion class to the training data and by grouping the output predictions using density based spatial clustering. The method is tested on two datasets, gathered using different laser scanners, in different types of underground mines which have different rock bolting patterns. In both cases the method is found to be highly capable of detecting the rock bolts with recall scores of 0.87-0.96. The second case study investigates modern deep learning for LiDAR data. Here, multiple transfer learning strategies and LiDAR data representations are examined for the task of identifying historic mining remains. A transfer learning approach based on a Lunar crater detection model is used, due to the task similarities between both the underlying data structures and the geometries of the objects to be detected. The relationship between dataset resolution and detection accuracy is also examined, with the results showing that the approach is capable of detecting pits and shafts to a high degree of accuracy with precision and recall scores between 0.80-0.92, provided the input data is of sufficient quality and resolution. Alongside resolution, different LiDAR data representations are explored, showing that the precision-recall balance varies depending on the input LiDAR data representation. The third case study creates a deep convolutional neural network model to detect artisanal scale mining from multispectral satellite data. This model is trained from initialisation without transfer learning and demonstrates that accurate multispectral models can be built from a smaller training dataset when appropriate design and data augmentation strategies are adopted. Alongside the deep learning model, novel mosaicing algorithms are developed both to improve cloud cover penetration and to decrease noise in the final prediction maps. When applied to the study area, the results from this model provide valuable information about the expansion, migration and forest encroachment of artisanal scale mining in southwestern Ghana over the last four years. Finally, this thesis presents an implementation framework for these neural network based object detection models, to generalise the findings from this research to new mining sector deep learning tasks. This framework can be used to identify applications which would benefit from neural network approaches; to build the models; and to apply these algorithms in a real world environment. The case study chapters confirm that the neural network models are capable of interpreting remotely sensed data to a high degree of accuracy on real world mining problems, while the framework guides the development of new models to solve a wide range of related challenges

Advances in Condition Monitoring, Optimization and Control for Complex Industrial Processes

The book documents 25 papers collected from the Special Issue “Advances in Condition Monitoring, Optimization and Control for Complex Industrial Processes”, highlighting recent research trends in complex industrial processes. The book aims to stimulate the research field and be of benefit to readers from both academic institutes and industrial sectors

A review of laser scanning for geological and geotechnical applications in underground mining

Laser scanning can provide timely assessments of mine sites despite adverse challenges in the operational environment. Although there are several published articles on laser scanning, there is a need to review them in the context of underground mining applications. To this end, a holistic review of laser scanning is presented including progress in 3D scanning systems, data capture/processing techniques and primary applications in underground mines. Laser scanning technology has advanced significantly in terms of mobility and mapping, but there are constraints in coherent and consistent data collection at certain mines due to feature deficiency, dynamics, and environmental influences such as dust and water. Studies suggest that laser scanning has matured over the years for change detection, clearance measurements and structure mapping applications. However, there is scope for improvements in lithology identification, surface parameter measurements, logistic tracking and autonomous navigation. Laser scanning has the potential to provide real-time solutions but the lack of infrastructure in underground mines for data transfer, geodetic networking and processing capacity remain limiting factors. Nevertheless, laser scanners are becoming an integral part of mine automation thanks to their affordability, accuracy and mobility, which should support their widespread usage in years to come

A machine learning approach for the detection of supporting rock bolts from laser scan data in an underground mine

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this recordRock bolts are a crucial part of underground infrastructure support; however, current methods to locate and record their positions are manual, time consuming and generally incomplete. This paper describes an effective method to automatically locate supporting rock bolts from a 3D laser scanned point cloud. The proposed method utilises a machine learning classifier combined with point descriptors based on neighbourhood properties to classify all data points as either ‘bolt’ or ‘not-bolt’ before using the Density Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm to divide the results into candidate bolt objects. The centroids of these objects are then computed and output as simple georeferenced 3D coordinates to be used by surveyors, mine managers and automated machines. Two classifiers were tested, a random forest and a shallow neural network, with the neural network providing the more accurate results. Alongside the different classifiers, different input feature types were also examined, including the eigenvalue based geometric features popular in the remote sensing community and the point histogram based features more common in the mobile robotics community. It was found that a combination of both feature sets provided the strongest results. The obtained precision and recall scores were 0.59 and 0.70 for the individual laser points and 0.93 and 0.86 for the bolt objects. This demonstrates that the model is robust to noise and misclassifications, as the bolt is still detected even if edge points are misclassified, provided that there are enough correct points to form a cluster. In some cases, the model can detect bolts which are not visible to the human interpreter.University of Exete

Displacement back analysis for a high slope of the dagangshan hydroelectric power station based on BP neural network and particle swarm optimization

The right bank high slope of the Dagangshan Hydroelectric Power Station is located in complicated geological conditions with deep fractures and unloading cracks. How to obtain the mechanical parameters and then evaluate the safety of the slope are the key problems. This paper presented a displacement back analysis for the slope using an artificial neural network model (ANN) and particle swarm optimization model (PSO). A numerical model was established to simulate the displacement increment results, acquiring training data for the artificial neural network model. The backpropagation ANN model was used to establish a mapping function between the mechanical parameters and the monitoring displacements. The PSO model was applied to initialize the weights and thresholds of the backpropagation (BP) network model and determine suitable values of the mechanical parameters. Then the elastic moduli of the rock masses were obtained according to the monitoring displacement data at different excavation stages, and the BP neural network model was proved to be valid by comparing the measured displacements, the displacements predicted by the BP neural network model, and the numerical simulation using the back-analyzed parameters. The proposed model is useful for rock mechanical parameters determination and instability investigation of rock slopes

Optimising mobile laser scanning for underground mines

Despite several technological advancements, underground mines are still largely relied on visual inspections or discretely placed direct-contact measurement sensors for routine monitoring. Such approaches are manual and often yield inconclusive, unreliable and unscalable results besides exposing mine personnel to field hazards. Mobile laser scanning (MLS) promises an automated approach that can generate comprehensive information by accurately capturing large-scale 3D data. Currently, the application of MLS has relatively remained limited in mining due to challenges in the post-registration of scans and the unavailability of suitable processing algorithms to provide a fully automated mapping solution. Additionally, constraints such as the absence of a spatial positioning network and the deficiency of distinguishable features in underground mining spaces pose challenges in mobile mapping. This thesis aims to address these challenges in mine inspections by optimising different aspects of MLS: (1) collection of large-scale registered point cloud scans of underground environments, (2) geological mapping of structural discontinuities, and (3) inspection of structural support features. Firstly, a spatial positioning network was designed using novel three-dimensional unique identifiers (3DUID) tags and a 3D registration workflow (3DReG), to accurately obtain georeferenced and coregistered point cloud scans, enabling multi-temporal mapping. Secondly, two fully automated methods were developed for mapping structural discontinuities from point cloud scans – clustering on local point descriptors (CLPD) and amplitude and phase decomposition (APD). These methods were tested on both surface and underground rock mass for discontinuity characterisation and kinematic analysis of the failure types. The developed algorithms significantly outperformed existing approaches, including the conventional method of compass and tape measurements. Finally, different machine learning approaches were used to automate the recognition of structural support features, i.e. roof bolts from point clouds, in a computationally efficient manner. Roof bolts being mapped from a scanned point cloud provided an insight into their installation pattern, which underpinned the applicability of laser scanning to inspect roof supports rapidly. Overall, the outcomes of this study lead to reduced human involvement in field assessments of underground mines using MLS, demonstrating its potential for routine multi-temporal monitoring

Systematic review of georisk in underground hard rock mines

Deep mining, driven by the increasing need of the sustainable use of mineral resources yields a possibility to fully extract the untapped deposits. Nevertheless, large depths remain challenging and complex environment. Rockbursts and induced seismicity, considered as georisks, are one of the most relevant risks identified in the field, which threat both safety and economics. Risk management tools and guidelines are essential to maintain safe and economically feasible extraction, but they still need improvements. One such opportunity identified here is the development of on-line georisk management systems, and going even further, a creation of a risk management concept covering the entire mine. This master’s thesis is a part of the On-line Risk Management in Deep Mines (ORMID) project, funded by the Academy of Finland, running under the Mineral Resources and Material Substitution programme. Systematic review of the literature was conducted to enable addressing the thesis goals: identification of the gaps in research concerning on-line georisk management in underground hard rock mines, establishment of the state-of-the-art of the developments in that topic, and providing recommendations for future research. Three databases were used for the search: Scopus, ScienceDirect, and IEEEXplore. To search the databases 12 keywords and phrases were formulated. The search was conducted in three phases: out of 13 767 studies identified, 98 were taken into manual investigation, and 50 of them were finally included in this master’s thesis. The gap identified in this study is the lack of research that examines the methods of rockmass stress calculation and forecasting based on the strain measurements. Only two examples of them were obtained in the study. Another issue that comes out is a need of deeper understanding of rockburst phenomenon. Moreover, a very low number of systems capable of on-line georisk management was identified. Dynamic Intelligent Ground Monitoring (Digmine), Mine Seismicity Risk Analysis Program (MS-RAP) and one standard architecture (AziSA) of the on-line georisk management were recognized as the state-of-the-art. To manage the georisks the state-of-the-art method represents an immediate rockburst warning method based on microseismicity analysis, already utilised in Digmine. BurstSupport software aiming to assist the geotechnical engineers in evaluating different rockburst support options in a burst-prone ground was considered as the state-of-the-art georisk mitigation method identified in the study. Identification of these developments resulted from rigorous inclusion and exclusion criteria, selected keywords and databases. Different choice of these would yield dissimilar results, what indicates that not all of the research about the topic of interest was identified. Recommendations and a roadmap addressing upcoming assignments in the ORMID project are presented in this thesis

Identification of cement manufacturing raw materials using machine vision

In the mining and manufacturing industry, there is a need for a non-extractive system to identify raw materials on conveying systems. Such a system would allow identification of raw materials on conveying systems preventing cross-contamination when the materials arrive at the final storage location. This project used machine vision techniques to identify cement manufacturing raw materials (clinker, gypsum and, limestone). Firstly, a representative sample (25 x 10kg samples of each material) was collected using a stratified random sampling procedure. This stratified random sampling procedure ensured the sample accurately represented the raw material in the stockpile. A dual purpose test bed and controlled lighting camera enclosure (for static model development and future dynamic system implementation) were constructed to minimise the effect of varying ambient light. This test bed and camera enclosure allowed the CMOS global shutter industrial camera to take twenty, 24bit colour images (8bit for each colour) of each sample. These images were catalogued and stored in a database for further model training and verification purposes. These images were pre-processed by a median filter which allowed any over saturated pixels (due to raw material surface moisture reflection) to have their intensity level reduced by replacing its value by the median value of its local neighbours. From the filtered image the individual red, green and blue (RGB) components were passed to a Histogram function which binned (255 bins for 8-bit colour) the various pixel intensities. The statistical features (weighted mean, skewness and kurtosis) of each colour's histogram were then stored in an array which then passed to the image feature database. A varying amount of feature arrays were used to train and verify the success of a probabilistic neural network (PNN) model. Initial optimisation of the PNN model was conducted using a local search algorithm which changed the smoothing parameter which achieved 94.83% accuracy. This model was then improved by implementing a Supervised Learning Probabilistic Neural Network (SLPNN). This model added data weight which changed the height of the Gaussian distribution function and input variable vector weight which changes the width of Gaussian distribution function. The implementation of the Supervised Learning Probabilistic Neural Network improved the models accuracy to 99.57%. Further model field testing will be required to verify the system in an operational environment where the camera enclosure will be subjected to dust, noise, varying temperatures and moisture. The Supervised Learning Probabilistic Neural Network outperforms the standard Probabilistic Neural Network which has been proven by this work. This work supports the claim that Machine Vision can be successfully be used to identify cement manufacturing raw materials with a high success rate. It also contributes to the literature by classifying clinker, gypsum and limestone in one body of work

Drill monitor with strata strength classification in near-real time

"The process of drilling and bolting the roof is currently one of the most dangerous jobs in underground mining, resulting in about 1,000 accidents with injuries each year in the United States. Researchers from the Spokane Research Laboratory of the National Institute for Occupational Safety and Health are studying the use of a drill monitoring system to estimate the strength of successive layers of rock and assess the integrity of a mine roof so that roof drill operators can be warned when a weak layer is being drilled. Measurements taken during the drilling can be converted to suitably scaled features so that a neural network can classify mine roof strata in terms of relative strength. The feasibility of this concept has been demonstrated in the laboratory. The research project was undertaken in order to increase the safety of underground miners, especially those involved in roof bolting., The system should be applicable to the mobile drills used in underground mining and would likely find wider application as well." - NIOSHTIC-2Includes bibliographical references (p. 14