Laser beams-based localization methods for Boom-type roadheader using underground camera non-uniform blur model

The efficiency of automatic underground tunneling is significantly depends on the localization accuracy and reliable for the Boom-type roadheader. In comparison with other underground equipment positioning methods, vision-based measurement has gained attention for its advantages of noncontact and no accumulated error. However, the harsh underground environment, especially the geometric errors brought by the vibration of the machine body to the underground camera model, has a certain influence on the accuracy and stability for the vision-based underground localization. In this paper, a laser beams-based localization methods for the machine body of Boom-type roadheader is presented, which can tackle the dense-dust, low illumination environment with the stray lights interference. Taking mining vibration into consideration, an underground camera non-uniform blur model that incorporate the two-layer glasses refraction effect was established to eliminate vibration errors. The blur model explicitly reveals the change of imaging optical path under the influence of vibration and double layer explosion-proof glass. On the basis of this, the underground laser beams extraction and positioning are presents, which is with well environmental adaptability, and the improved 2P3L (two-points-three-lines) localization model from line correspondences are developed. Experimental evaluation are designed to verify the performance of the proposed method, and the deblurring algorithm is investigated and evaluated. The results show that the proposed methods is effective to restore the blurred laser beams image that caused by the vibration, and can meet the precision need of roadheader body localization for roadway construction in coal mine

Precisely positioning method for roadheaders and robotic roadheader system

To overcome the problems faced in underground positioning,the robotic roadheader mine Internet of things (IoT) was designed which connected multi sensors,surveying devices and industrial computers,and a novel positioning method was developed,based on which a robotic roadheader system was built.The robotic roadheader system was employed for real-world tunnel cutting.The experimental results verifies the high accuracy of the positioning method,which achieves an RMSE error less than 5 cm,and the robust running of the robotic roadheader indicates that the robotic roadheader system can precisely perceive the surrounding environment and make precise interaction with the working environment,which yield a safety,high efficiency,and precisely underground tunnel building and coal mining and an unmanned working surface

Position and orientation measurement technology for bolter miner body based on dual-screen visual target

Aiming at the problem that it is difficult to achieve the real-time and accurate measurement of the bolter miner’s position and orientation during the excavation process in coal mines, which leads to the difficulty in achieving directional excavation, a guidance method for bolter miner based on dual-screen visual target is proposed. Using two vertically installed light-sensitive imaging screens to form the dual-screen visual target surfaces and the indication laser emitted by the laser instrument presents light spots on the front and rear target surfaces. Combining with the visual measurement, high-precision raster calibration and other technology are applied to establish the mapping relationship of the spot centroid between 2D-3D coordinates, which is used to form the point cloud data of the coordinates. Based on the principle of grid indexing, coordinate transformation and Euler angle solving, combining with the biaxial inclinometer at the bottom of target to obtain the bolter miner body’s real-time position and orientation, the key points’ horizontal/vertical deviations relative to the roadway axis are calculated, which can provide data support for deviation correction during the excavation process. The off-target problem of the system is analyzed by constructing a mathematical model. Meanwhile, the effectiveness of the guidance method is verified by building an experimental platform. The experimental results indicate that this method can achieve a precision measurement of six-degrees-of-freedom spatial pose for the machine body. When the measurement distance is 9 m, the repeatability measurement precision of the yaw angle is better than 0.01º and the error of absolute measurement is less than 0.05º. Within the measurement range of 15−40 m, which uses the total station and mining laser to set the planning line, the measurement errors of key points’ horizontal/vertical deviations are less than 5 mm and 15 mm, respectively. The guiding system developed based on this method has also been successfully applied to the underground roadway excavation in coal mine, which fully meets the requirements of underground roadway excavation and the positioning of the machine body’s key points. The error characteristic of the guiding method is independent of the test distance. Also, all optical measurement functions involved in the method are realized inside the target, which can effectively shield the influence of the underground complex environment for the measurement function, and greatly improve the capacity of anti-dust interference in field application

Safe Path Planning Method Based on Collision Prediction for Robotic Roadheader in Narrow Tunnels

Safe path planning is essential for the autonomous operation of robotic roadheader in narrow underground tunnels, where limited perception and the robot’s geometric constraints present significant challenges. Traditional path planning methods often fail to address these issues. This paper proposes a collision prediction-integrated path planning method tailored for robotic roadheader in confined environments. The method comprises two components: collision prediction and path planning. A collision prediction model based on artificial potential fields is developed, considering the non-convex shape of the roadheader and enhancing scalability. By utilizing tunnel design information, a composite potential field model is created for both obstacles and the roadheader, enabling real-time collision forecasting. The A* algorithm is modified to incorporate the robot’s motion constraints, using a segmented weighted heuristic function based on collision predictions. Path smoothness is achieved through Bézier curve smoothing. Experimental results in both obstacle-free and obstacle-laden scenarios show that the proposed method outperforms traditional approaches in terms of computational efficiency, path length, and smoothness, ensuring safe, efficient navigation in narrow tunnels

On the academic ideology of “Digging is modelling”

To realize safe, efficient, and intelligent excavation of coal mine roadways, the academic concept of “Digging is modelling” is proposed,which defines thecontent and architectural framework of the concept, as well as extracts the key technical issues related to it. Specifically, these include multiplexmining model construction technology integrating multi-source information, the intelligent cutting technology based on mining model, the intelligent temporary support technology based on mining model, the intelligent permanent support technology based on mining model, the intelligent navigation technology based on mining model, and the mechanical equipments intelligence parallel cooperative control technology based on mining model. The problem of mining model construction is addressed by proposing a method that integrates multi-source data such as geological exploration, mine design, and advance detection. This method provides a unified basis for the model construction of various subsystem of the excavating system. Furthermore, to address the issue of intelligent cutting based on mining model, a coupling submodel of mining model and cutting subsystem model is established. Intelligent cutting trajectory planning and cutting parameter optimization methods are proposed, an intelligent cutting strategy for mining is formulated, and adaptive planning of the cutting subsystem is realized. In order to address the issue of intelligent temporary support based on mining model, a sub-model for temporary support is established and coupled with the cuttingmining model and temporary support subsystem. Additionally, an adaptive adjustment method for temporary support posture and support force is proposed to ensure the safe and reliable operation of the temporary support subsystem, improve the stability of the surrounding rock, and lay a spatio-temporal foundation for parallel and cooperative digging and anchoring operations. To address the issue of permanent support based on mining model, we have established a permanent support subsystem coupled with temporary support mining model. Additionally, we propose a collaborative control method for each drilling and anchoring equipment in the permanent support subsystem under limited time and space. This approach aims to achieve efficient collaborative control of the permanent support subsystem. Aiming to address the challenge of intelligent navigation based on mining models, a sub-model integrating mining model and navigation subsystem is established. Furthermore, an accurate navigation method for intelligent excavating system, combining inertial navigation with total station technology, is proposed to enhance the precision of roadway driving and formation quality. In order to address the issue of intelligent parallel cooperative control in a cluster based on the tunnel model, we have established a parallel cooperative control sub-model that is integrated with the tunnel model and the cluster cooperative control subsystem. Additionally, we have developed a multi-machine parallel cooperative control strategy and proposed a cooperative control method for multi-task and multi-system intelligent excavating systems to achieve safe and efficient driving. The shield mine excavation robot system developed by team based on the academic concept of “Digging is modelling”. This system has been successfully utilized by Shaanxi Coal and Chemical Industry Group Shaanxi Xiao Bao Dang Mining Co., Ltd., effectively addressing challenges in mine roadway excavation under complex geological conditions such as thick dirt, high hardness, and serious sheet wall. As a result, it has significantly enhanced the safety, efficiency, and intelligence level of tunnel excavation

Bacterial Programming Based Kinematic Chain Estimation of Construction Vehicle

Construction vehicle automation for high accuracy applications require information about the state of the machine, resulting in a fully sensitized system with precise kinematic parameters. Since the measurement of these parameters contains uncertainties, accurate measurement of them is an expensive task. Automatic calibration of link parameters makes the task of kinematic parameter determination easier. This study reports a method for forward kinematic chain estimation of an excavator by bacterial programming (BP) based on randomly placed inertial navigation systems (INS) per segments with microelectromechanical sensors (MEMS) within. MEMS INS with fusion techniques provide increasing accuracy with outstanding resilience against harsh environment in a rigid housing. With known robot kinematic the tool orientation estimation can be made more accurate also the path can be planned. The unknown model structure and parameters are established and identified by BP without any a priori or given information about the device according to Denavit-Hartenberg (DH) transformation conventions. Fundamentals of this approach are described in detail and shown on simulated measurement results

Lidar based map construction fusion method for underground coal mine shaft bottom

Intellectualization of coal mine is the technical support for high-quality development of coal industry, and robot replacement of key posts is the development trend of realizing efficient mining of coal with few people and no people. Simultaneous localization and mapping (SLAM) is one of the key technologies for autonomous movement and navigation of coal mine robots. The environment of underground coal mine is a typical unstructured environment, with narrow space, complex and changeable structure and uneven lighting, posing a severe challenge to the realization of SLAM in the underground coal mine. The research status of the map construction of the underground coal mine is summarized. In view of the shortcomings of the loopback detection of the LeGO-LOAM algorithm, the SegMatch algorithm is used to improve the loopback detection module of the LeGO-LOAM, the ICP algorithm is used to optimize the global map, and an improved algorithm integrating LeGO-LOAM and SegMatch is proposed, and the principle and implementation of the algorithm are discussed. The underground simulation scene experiments of coal mine were carried out, the mapping effect and accuracy of SLAM algorithm before and after the improvement were compared and analyzed, and the results showed that the map loopback effect constructed by the improved algorithm was better, and the estimated trajectory was smoother and more accurate. The construction method of two-dimensional occupied grid map is studied aiming at the navigation requirements, and the accuracy of the grid map constructed by this method is verified through experiments. The results show that the grid map after effectively filtering outliers such as dynamic obstacles has a mapping accuracy of 0.01 m, and the required storage space is 3 orders of magnitude lower than that of the point cloud map. The research results are helpful to the realization of SLAM and real-time positioning and autonomous navigation of the coal mine robot under the unstructured environment of the underground coal mine

Research on coal mine XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots

With the development of coal mine intelligence and the application of coal mine robots, an efficient collaboration between coal mine operators and coal mine robots plays a crucial role in the execution of complex underground tasks. To optimize the complex operational relationship of multiple coal mine operators and multiple robots, based on the concept of digital twin and extensive experience in the XR field, the research is conducted on the design and key technologies of XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots in coal mines. Firstly, for a typical scenario of collaboration between two types of coal mine operators (i.e central control operators and field control operators) and two types of coal mine robots (i.e. detection robots and operating robots) in complex tasks, the overall system architecture is designed. The system is divided into three parts: the physical subsystem, VR operation and maintenance subsystem, and AR operation and maintenance subsystem. The content, functions, and collaborative operation relationships among these three parts are introduced. Then, an analysis of key technologies related to the VR operation and maintenance subsystem, AR operation and maintenance subsystem, and communication networking is carried out. The solutions corresponding to each key technology are discussed, and the integration and operation of the two types of coal mine operators, two types of coal mine robots, and VR/AR operation and maintenance subsystem are implemented. Finally, a laboratory environment simulating complex underground conditions is set up to create a test site, where the task points and specific tasks are defined. The feasibility and effectiveness of the system are tested and verified. The experimental results show that the coal mine XR intelligent operation and maintenance system is able to function in collaborative situations between multiple humans and multiple robots corresponding to different complex tasks. Through the collaborative operation of the VR operation and maintenance subsystem and the AR operation and maintenance subsystem, the collaborative perception, decision-making, and control between virtual space and physical space can be achieved. This allows for the iterative optimization and verification of complex tasks in a physical space from a virtual space, forming an intelligent operational mode of human-human, human-robot, and robot-robot interactive collaboration

Advances in Condition Monitoring, Optimization and Control for Complex Industrial Processes

The book documents 25 papers collected from the Special Issue “Advances in Condition Monitoring, Optimization and Control for Complex Industrial Processes”, highlighting recent research trends in complex industrial processes. The book aims to stimulate the research field and be of benefit to readers from both academic institutes and industrial sectors

Affecting Fundamental Transformation in Future Construction Work Through Replication of the Master-Apprentice Learning Model in Human-Robot Worker Teams

Construction robots continue to be increasingly deployed on construction sites to assist human workers in various tasks to improve safety, efficiency, and productivity. Due to the recent and ongoing growth in robot capabilities and functionalities, humans and robots are now able to work side-by-side and share workspaces. However, due to inherent safety and trust-related concerns, human-robot collaboration is subject to strict safety standards that require robot motion and forces to be sensitive to proximate human workers. In addition, construction robots are required to perform construction tasks in unstructured and cluttered environments. The tasks are quasi-repetitive, and robots need to handle unexpected circumstances arising from loose tolerances and discrepancies between as-designed and as-built work. It is therefore impractical to pre-program construction robots or apply optimization methods to determine robot motion trajectories for the performance of typical construction work. This research first proposes a new taxonomy for human-robot collaboration on construction sites, which includes five levels: Pre-Programming, Adaptive Manipulation, Imitation Learning, Improvisatory Control, and Full Autonomy, and identifies the gaps existing in knowledge transfer between humans and assisting robots. In an attempt to address the identified gaps, this research focuses on three key studies: enabling construction robots to estimate their pose ubiquitously within the workspace (Pose Estimation), robots learning to perform construction tasks from human workers (Learning from Demonstration), and robots synchronizing their work plans with human collaborators in real-time (Digital Twin). First, this dissertation investigates the use of cameras as a novel sensor system for estimating the pose of large-scale robotic manipulators relative to the job sites. A deep convolutional network human pose estimation algorithm was adapted and fused with sensor-based poses to provide real-time uninterrupted 6-DOF pose estimates of the manipulator’s components. The network was trained with image datasets collected from a robotic excavator in the laboratory and conventional excavators on construction sites. The proposed system yielded an uninterrupted and centimeter-level accuracy pose estimation system for articulated construction robots. Second, this dissertation investigated Robot Learning from Demonstration (LfD) methods to teach robots how to perform quasi-repetitive construction tasks, such as the ceiling tile installation process. LfD methods have the potential to be used in teaching robots specific tasks through human demonstration, such that the robots can then perform the same tasks under different conditions. A visual LfD and a trajectory LfD methods are developed to incorporate the context translation model, Reinforcement Learning method, and generalized cylinders with orientation approach to generate the control policy for the robot to perform the subsequent tasks. The evaluated results in the Gazebo robotics simulator confirm the promise and applicability of the LfD method in teaching robot apprentices to perform quasi-repetitive tasks on construction sites. Third, this dissertation explores a safe working environment for human workers and robots. Robot simulations in online Digital Twins can be used to extend designed construction models, such as BIM (Building Information Models), to the construction phase for real-time monitoring of robot motion planning and control. A bi-directional communication system was developed to bridge robot simulations and physical robots in construction and digital fabrication. Through empirical studies, the high accuracy of the pose synchronization between physical and virtual robots demonstrated the potential for ensuring safety during proximate human-robot co-work.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169666/1/cjliang_1.pd