Deep learning in automated ultrasonic NDE -- developments, axioms and opportunities

The analysis of ultrasonic NDE data has traditionally been addressed by a trained operator manually interpreting data with the support of rudimentary automation tools. Recently, many demonstrations of deep learning (DL) techniques that address individual NDE tasks (data pre-processing, defect detection, defect characterisation, and property measurement) have started to emerge in the research community. These methods have the potential to offer high flexibility, efficiency, and accuracy subject to the availability of sufficient training data. Moreover, they enable the automation of complex processes that span one or more NDE steps (e.g. detection, characterisation, and sizing). There is, however, a lack of consensus on the direction and requirements that these new methods should follow. These elements are critical to help achieve automation of ultrasonic NDE driven by artificial intelligence such that the research community, industry, and regulatory bodies embrace it. This paper reviews the state-of-the-art of autonomous ultrasonic NDE enabled by DL methodologies. The review is organised by the NDE tasks that are addressed by means of DL approaches. Key remaining challenges for each task are noted. Basic axiomatic principles for DL methods in NDE are identified based on the literature review, relevant international regulations, and current industrial needs. By placing DL methods in the context of general NDE automation levels, this paper aims to provide a roadmap for future research and development in the area.Comment: Accepted version to be published in NDT & E Internationa

Using Assembled 2D LiDAR Data for Single Plant Detection

A 2D laser scanner was mounted on the front of the small 4-wheel autonomous robot with differential steering, in an angle of 30 degrees pointing downwards. The machine was able to drive between maize rows and collect timestamped data simultaneously. The position of the vehicle was tracked by a highly precise total station. The data of the total station and the laser scanner was fused to generate a 3D point cloud. This 3D representation was used to search for single plant positions, what could later be used for additional applications like single plant treatment and precision weeding. First all points belonging to the ground plane were removed. Afterwards outliers were filtered. For separating the resulting points, a k-d tree clustering was used. Of each single point cloud cluster the 3D centroid was evaluated and assumed as the resulting plant position. This was done on three different growth stages of the plants. Results showed good detection rates up to 70.7 % with a root mean square error of 3.6 cm, precise enough to allow single plant treatment

Development of an automated ultrasonic inspection cell for detecting subsurface discontinuities in cast iron

Manufacturing of value added products by companies in todays global marketplace requires the vendors to meet certain minimum standards, SAE, ASTM, ISO, etc. In a global economy, as a manufacturer for the year 2000 and beyond, preferred vendors will become ISO 9000 certified to maintain a market share of produced goods. An automated inspection cell was developed for the inspection of cast iron ports to detect subsurface discontinuities. The cell consists of an ultrasonic flaw detector (UFD), transducer, robot, immersion tank, computer, and software. Normal beam pulse-echo ultrasonic nondestructive testing is performed on each rough casting. Using test blocks and castings supplied by an industrial partner and working with a skilled ultrasonic inspector; ultrasonic transducer selection, initial inspection criteria, and UFD setup parameters were developed the gray iron castings used in this study. The skilled ultrasonic inspector\u27s operation of the UFD was noted for development of the cell software. The ultrasonic inspection cell control software (UICCS) was designed and developed to perform the necessary functions for control of the robot and UFD in real-time. The UICCS performed two main tasks; emulating the manual operation of the UFD through the communication link with the unit, and evaluation of the ultrasonic signatures for detection of subsurface discontinuities. The next phase of the cell development involved the testing of a lot of 105 castings. These casting were processed through the inspection cell. The castings which passed the inspection criteria were returned to the manufacturer for machining into finished parts where they were visibly inspected for defects after machining. The castings that had ultrasonic signatures consistent with subsurface discontinues were manually inspected by the skilled ultrasonic inspector, with the manual inspection time recorded for comparison to the automated cycle time. The castings then were inspected using destructive testing techniques for detecting subsurface material voids. The developed automated inspection cell correctly classified the inspection locations 99.8% of the time. Compared to manual inspection (as measured in the study), the automated cell\u27s cycle time was 30 times more efficient

Development of a real-time ultrasonic sensing system for automated and robotic welding

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The implementation of robotic technology into welding processes is made difficult by the inherent process variables of part location, fit up, orientation and repeatability. Considering these aspects, to ensure weld reproducibility consistency and quality, advanced adaptive control techniques are essential. These involve not only the development of adequate sensors for seam tracking and joint recognition but also developments of overall machines with a level of artificial intelligence sufficient for automated welding. The development of such a prototype system which utilizes a manipulator arm, ultrasonic sensors and a transistorised welding power source is outlined. This system incorporates three essential aspects. It locates and tracks the welding seam ensuring correct positioning of the welding head relatively to the joint preparation. Additionally, it monitors the joint profile of the molten weld pool and modifies the relevant heat input parameters ensuring consistent penetration, joint filling and acceptable weld bead shape. Finally, it makes use of both the above information to reconstruct three-dimensional images of the weld pool silhouettes providing in-process inspection capabilities of the welded joints. Welding process control strategies have been incorporated into the system based on quantitative relationships between input parameters and weld bead shape configuration allowing real-time decisions to be made during the process of welding, without the need for operation intervention.British Technology Group (BTG

Multi-scale metrology for automated non-destructive testing systems

This thesis was previously held under moratorium from 5/05/2020 to 5/05/2022The use of lightweight composite structures in the aerospace industry is now commonplace. Unlike conventional materials, these parts can be moulded into complex aerodynamic shapes, which are diffcult to inspect rapidly using conventional Non-Destructive Testing (NDT) techniques. Industrial robots provide a means of automating the inspection process due to their high dexterity and improved path planning methods. This thesis concerns using industrial robots as a method for assessing the quality of components with complex geometries. The focus of the investigations in this thesis is on improving the overall system performance through the use of concepts from the field of metrology, specifically calibration and traceability. The use of computer vision is investigated as a way to increase automation levels by identifying a component's type and approximate position through comparison with CAD models. The challenges identified through this research include developing novel calibration techniques for optimising sensor integration, verifying system performance using laser trackers, and improving automation levels through optical sensing. The developed calibration techniques are evaluated experimentally using standard reference samples. A 70% increase in absolute accuracy was achieved in comparison to manual calibration techniques. Inspections were improved as verified by a 30% improvement in ultrasonic signal response. A new approach to automatically identify and estimate the pose of a component was developed specifically for automated NDT applications. The method uses 2D and 3D camera measurements along with CAD models to extract and match shape information. It was found that optical large volume measurements could provide suffciently high accuracy measurements to allow ultrasonic alignment methods to work, establishing a multi-scale metrology approach to increasing automation levels. A classification framework based on shape outlines extracted from images was shown to provide over 88% accuracy on a limited number of samples.The use of lightweight composite structures in the aerospace industry is now commonplace. Unlike conventional materials, these parts can be moulded into complex aerodynamic shapes, which are diffcult to inspect rapidly using conventional Non-Destructive Testing (NDT) techniques. Industrial robots provide a means of automating the inspection process due to their high dexterity and improved path planning methods. This thesis concerns using industrial robots as a method for assessing the quality of components with complex geometries. The focus of the investigations in this thesis is on improving the overall system performance through the use of concepts from the field of metrology, specifically calibration and traceability. The use of computer vision is investigated as a way to increase automation levels by identifying a component's type and approximate position through comparison with CAD models. The challenges identified through this research include developing novel calibration techniques for optimising sensor integration, verifying system performance using laser trackers, and improving automation levels through optical sensing. The developed calibration techniques are evaluated experimentally using standard reference samples. A 70% increase in absolute accuracy was achieved in comparison to manual calibration techniques. Inspections were improved as verified by a 30% improvement in ultrasonic signal response. A new approach to automatically identify and estimate the pose of a component was developed specifically for automated NDT applications. The method uses 2D and 3D camera measurements along with CAD models to extract and match shape information. It was found that optical large volume measurements could provide suffciently high accuracy measurements to allow ultrasonic alignment methods to work, establishing a multi-scale metrology approach to increasing automation levels. A classification framework based on shape outlines extracted from images was shown to provide over 88% accuracy on a limited number of samples

Improving the safety and efficiency of rail yard operations using robotics

textSignificant efforts have been expended by the railroad industry to make operations safer and more efficient through the intelligent use of sensor data. This work proposes to take the technology one step further to use this data for the control of physical systems designed to automate hazardous railroad operations, particularly those that require humans to interact with moving trains. To accomplish this, application specific requirements must be established to design self-contained machine vision and robotic solutions to eliminate the risks associated with existing manual operations. Present-day rail yard operations have been identified as good candidates to begin development. Manual uncoupling, in particular, of rolling stock in classification yards has been investigated. To automate this process, an intelligent robotic system must be able to detect, track, approach, contact, and manipulate constrained objects on equipment in motion. This work presents multiple prototypes capable of autonomously uncoupling full-scale freight cars using feedback from its surrounding environment. Geometric image processing algorithms and machine learning techniques were implemented to accurately identify cylindrical objects in point clouds generated in real-vi time. Unique methods fusing velocity and vision data were developed to synchronize a pair of moving rigid bodies in real-time. Multiple custom end-effectors with in-built compliance and fault tolerance were designed, fabricated, and tested for grasping and manipulating cylindrical objects. Finally, an event-driven robotic control application was developed to safely and reliably uncouple freight cars using data from 3D cameras, velocity sensors, force/torque transducers, and intelligent end-effector tooling. Experimental results in a lab setting confirm that modern robotic and sensing hardware can be used to reliably separate pairs of rolling stock up to two miles per hour. Additionally, subcomponents of the autonomous pin-pulling system (APPS) were designed to be modular to the point where they could be used to automate other hazardous, labor-intensive tasks found in U.S. classification yards. Overall, this work supports the deployment of autonomous robotic systems in semi-unstructured yard environments to increase the safety and efficiency of rail operations.Mechanical Engineerin

Mobile robot transportation in laboratory automation

In this dissertation a new mobile robot transportation system is developed for the modern laboratory automation to connect the distributed automated systems and workbenches. In the system, a series of scientific and technical robot indoor issues are presented and solved, including the multiple robot control strategy, the indoor transportation path planning, the hybrid robot indoor localization, the recharging optimization, the robot-automated door interface, the robot blind arm grasping & placing, etc. The experiments show the proposed system and methods are effective and efficient

Localization, Mapping and SLAM in Marine and Underwater Environments

The use of robots in marine and underwater applications is growing rapidly. These applications share the common requirement of modeling the environment and estimating the robots’ pose. Although there are several mapping, SLAM, target detection and localization methods, marine and underwater environments have several challenging characteristics, such as poor visibility, water currents, communication issues, sonar inaccuracies or unstructured environments, that have to be considered. The purpose of this Special Issue is to present the current research trends in the topics of underwater localization, mapping, SLAM, and target detection and localization. To this end, we have collected seven articles from leading researchers in the field, and present the different approaches and methods currently being investigated to improve the performance of underwater robots