Fusion of Real Time Thermal Image and 1D/2D/3D Depth Laser Readings for Remote Thermal Sensing in Industrial Plants by Means of UAVs and/or Robots

This paper presents fast procedures for thermal infrared remote sensing in dark, GPS-denied environments, such as those found in industrial plants such as in High-Voltage Direct Current (HVDC) converter stations. These procedures are based on the combination of the depth estimation obtained from either a 1-Dimensional LIDAR laser or a 2-Dimensional Hokuyo laser or a 3D MultiSense SLB laser sensor and the visible and thermal cameras from a FLIR Duo R dual-sensor thermal camera. The combination of these sensors/cameras is suitable to be mounted on Unmanned Aerial Vehicles (UAVs) and/or robots in order to provide reliable information about the potential malfunctions, which can be found within the hazardous environment. For example, the capabilities of the developed software and hardware system corresponding to the combination of the 1-D LIDAR sensor and the FLIR Duo R dual-sensor thermal camera is assessed from the point of the accuracy of results and the required computational times: the obtained computational times are under 10 ms, with a maximum localization error of 8 mm and an average standard deviation for the measured temperatures of 1.11 degree Celsius, which results are obtained for a number of test cases. The paper is structured as follows: the description of the system used for identification and localization of hotspots in industrial plants is presented in section II. In section III, the method for faults identification and localization in plants by using a 1-Dimensional LIDAR laser sensor and thermal images is described together with results. In section IV the real time thermal image processing is presented. Fusion of the 2-Dimensional depth laser Hokuyo and the thermal images is described in section V. In section VI the combination of the 3D MultiSense SLB laser and thermal images is described. In section VII a discussion and several conclusions are drawn

Secure Encoded Instruction Graphs for End-to-End Data Validation in Autonomous Robots

As autonomous robots become increasingly ubiquitous, more attention is being paid to the security of robotic operation. Autonomous robots can be seen as cyber-physical systems that transverse the virtual realm and operate in the human dimension. As a consequence, securing the operation of autonomous robots goes beyond securing data, from sensor input to mission instructions, towards securing the interaction with their environment. There is a lack of research towards methods that would allow a robot to ensure that both its sensors and actuators are operating correctly without external feedback. This paper introduces a robotic mission encoding method that serves as an end-to-end validation framework for autonomous robots. In particular, we put our framework into practice with a proof of concept describing a novel map encoding method that allows robots to navigate an objective environment with almost-zero a priori knowledge of it, and to validate operational instructions. We also demonstrate the applicability of our framework through experiments with real robots for two different map encoding methods. The encoded maps inherit all the advantages of traditional landmark-based navigation, with the addition of cryptographic hashes that enable end-to-end information validation. This end-to-end validation can be applied to virtually any aspect of robotic operation where there is a predefined set of operations or instructions given to the robot

Experimental indoors localization using distance measurements obtained from visual data

In recent years, the emergence of artificial intelligence increases the demand of automatic and robust localization outdoors and indoors. While GPS provides enough accuracy in most outdoor cases, there is still a lack of robust and efficient indoor localization systems available on the market. In this report, an experimental framework for indoor localization is developed and tested. To operate an automatic robot owned by LCAV laboratory, two different operation modes have been successfully implemented, including controlling a robot in real-time or with extra input containing a list of commands. Also, a new visual fiducial system have been developed, and is able to capture the locations of Apriltags inside a room accurately. Multidimensional scaling (MDS) and Squared range least square (SRLS) algorithms containing distance information will also be introduced and the final localization result is within 2% error of tolerance compared with the ground truth results measured by a laser meter

A review of laser scanning for geological and geotechnical applications in underground mining

Laser scanning can provide timely assessments of mine sites despite adverse challenges in the operational environment. Although there are several published articles on laser scanning, there is a need to review them in the context of underground mining applications. To this end, a holistic review of laser scanning is presented including progress in 3D scanning systems, data capture/processing techniques and primary applications in underground mines. Laser scanning technology has advanced significantly in terms of mobility and mapping, but there are constraints in coherent and consistent data collection at certain mines due to feature deficiency, dynamics, and environmental influences such as dust and water. Studies suggest that laser scanning has matured over the years for change detection, clearance measurements and structure mapping applications. However, there is scope for improvements in lithology identification, surface parameter measurements, logistic tracking and autonomous navigation. Laser scanning has the potential to provide real-time solutions but the lack of infrastructure in underground mines for data transfer, geodetic networking and processing capacity remain limiting factors. Nevertheless, laser scanners are becoming an integral part of mine automation thanks to their affordability, accuracy and mobility, which should support their widespread usage in years to come

Localization and navigation using QR code for mobile robot in indoor environment

Localization and navigation are fundamental issues to mobile robotics. An approach for localization and navigation for a mobile robot in indoor environment is proposed in this paper. In this approach, QR codes are used as landmarks to provide global pose references. Label and location information is stored in QR codes which are strategically placed in the operating environment. The mobile robot is equipped with an industrial camera pointing to the ceiling to read QR codes at high speed. The pose of the robot is estimated according to the positional relationship between QR codes and the robot. For the purpose of collision-free navigation, a laser range finder (LRF) is applied to build a map in unknown environment and detect obstacles. Dijkstra algorithm and Dynamic Window Approach (DWA) are applied in path planning based on a 2D grid map. Experimental results show that this approach has good feasibility and effectiveness

Fusion of low-cost and light-weight sensor system for mobile flexible manipulator

There is a need for non-industrial robots such as in homecare and eldercare. Light-weight mobile robots preferred as compared to conventional fixed based robots as the former is safe, portable, convenient and economical to implement. Sensor system for light-weight mobile flexible manipulator is studied in this research. A mobile flexible link manipulator (MFLM) contributes to high amount of vibrations at the tip, giving rise to inaccurate position estimations. In a control system, there inevitably exists a lag between the sensor feedback and the controller. Consequently, it contributed to instable control of the MFLM. Hence, there it is a need to predict the tip trajectory of the MFLM. Fusion of low cost sensors is studied to enhance prediction accuracy at the MFLM’s tip. A digital camera and an accelerometer are used predict tip of the MFLM. The main disadvantage of camera is the delayed feedback due to the slow data rate and long processing time, while accelerometer composes cumulative errors. Wheel encoder and webcam are used for position estimation of the mobile platform. The strengths and limitations of each sensor were compared. To solve the above problem, model based predictive sensor systems have been investigated for used on the mobile flexible link manipulator using the selected sensors. Mathematical models were being developed for modeling the reaction of the mobile platform and flexible manipulator when subjected to a series of input voltages and loads. The model-based Kalman filter fusion prediction algorithm was developed, which gave reasonability good predictions of the vibrations of the tip of flexible manipulator on the mobile platform. To facilitate evaluation of the novel predictive system, a mobile platform was fabricated, where the flexible manipulator and the sensors are mounted onto the platform. Straight path motions were performed for the experimental tests. The results showed that predictive algorithm with modelled input to the Extended Kalman filter have best prediction to the tip vibration of the MFLM

Optimising mobile laser scanning for underground mines

Despite several technological advancements, underground mines are still largely relied on visual inspections or discretely placed direct-contact measurement sensors for routine monitoring. Such approaches are manual and often yield inconclusive, unreliable and unscalable results besides exposing mine personnel to field hazards. Mobile laser scanning (MLS) promises an automated approach that can generate comprehensive information by accurately capturing large-scale 3D data. Currently, the application of MLS has relatively remained limited in mining due to challenges in the post-registration of scans and the unavailability of suitable processing algorithms to provide a fully automated mapping solution. Additionally, constraints such as the absence of a spatial positioning network and the deficiency of distinguishable features in underground mining spaces pose challenges in mobile mapping. This thesis aims to address these challenges in mine inspections by optimising different aspects of MLS: (1) collection of large-scale registered point cloud scans of underground environments, (2) geological mapping of structural discontinuities, and (3) inspection of structural support features. Firstly, a spatial positioning network was designed using novel three-dimensional unique identifiers (3DUID) tags and a 3D registration workflow (3DReG), to accurately obtain georeferenced and coregistered point cloud scans, enabling multi-temporal mapping. Secondly, two fully automated methods were developed for mapping structural discontinuities from point cloud scans – clustering on local point descriptors (CLPD) and amplitude and phase decomposition (APD). These methods were tested on both surface and underground rock mass for discontinuity characterisation and kinematic analysis of the failure types. The developed algorithms significantly outperformed existing approaches, including the conventional method of compass and tape measurements. Finally, different machine learning approaches were used to automate the recognition of structural support features, i.e. roof bolts from point clouds, in a computationally efficient manner. Roof bolts being mapped from a scanned point cloud provided an insight into their installation pattern, which underpinned the applicability of laser scanning to inspect roof supports rapidly. Overall, the outcomes of this study lead to reduced human involvement in field assessments of underground mines using MLS, demonstrating its potential for routine multi-temporal monitoring