Structure-from-motion based image unwrapping and stitching for small bore pipe inspections

Visual inspection is one of the most ubiquitous forms of non-destructive testing, being widely used in routine pipe inspections. For small bore pipes (centimetre diameter), inspectors often have a restricted field of view limiting overall image and inspection quality. Stitching multiple unwrapped images is a common inspection technique to provide a full view inspection image by combining multiple video frames together. A key challenge of this method is knowing the camera pose of each frame. Consequently, mechanical centralisers are often utilised to ensure the camera is located centrally. For the inspection of small-bore pipes, such mechanical centralisers are often too large to fit. This paper presents a post-processing, Structure-from-Motion (SfM) based approach to unwrap and stitch inspection images, captured by a manually deployed commercial videoscope. It advances state-of-the-art approaches which rely on the projection of a laser pattern into the field of view, thus reducing the equipment size. The process consists of camera pose estimation, preliminary point cloud generation, secondary fitting, images unwrapping and stitching to form an undistorted view of the pipe interior. Two industrial focussed demonstrators verified the successful implementation for small-bore pipe inspections. Whereby the new approach does not rely on image features to create the surface texture and is less sensitive to the image quality, more areas can be retrieved from inspections. The reconstructed area was increased by up to 87% using the new approach versus the conventional 3D model

Automatic Color Inspection for Colored Wires in Electric Cables

In this paper, an automatic optical inspection system for checking the sequence of colored wires in electric cable is presented. The system is able to inspect cables with flat connectors differing in the type and number of wires. This variability is managed in an automatic way by means of a self-learning subsystem and does not require manual input from the operator or loading new data to the machine. The system is coupled to a connector crimping machine and once the model of a correct cable is learned, it can automatically inspect each cable assembled by the machine. The main contributions of this paper are: (i) the self-learning system; (ii) a robust segmentation algorithm for extracting wires from images even if they are strongly bent and partially overlapped; (iii) a color recognition algorithm able to cope with highlights and different finishing of the wire insulation. We report the system evaluation over a period of several months during the actual production of large batches of different cables; tests demonstrated a high level of accuracy and the absence of false negatives, which is a key point in order to guarantee defect-free productions

The use of job aids for visual inspection in manufacturing and maintenance

Visual inspection is a task regularly seen in manufacturing applications and is still primarily carried out by human operators. This study explored the use of job aids (anything used to assist the operator with the task, such as lists, check sheets or pictures) to assist with visual inspection within a manufacturing facility that inspects used parts. Job aids in the form of inspection manuals were used regularly during the inspection process, and how accurately they were followed was dependent on a number of factors such as size of part, experience of the operator, and accuracy of the inspection manuals. If the job aids were well structured, well written and accessible, then the inspectors were seen to follow them, however for certain jobs inspectors were seen to change the inspection order making inspection more efficient. The findings of the study suggest that prior experience can help in designing efficient, easy to use job aids and that a collaborative approach to design as well as using pictorial examples for comparison purposes would improve the inspection process

Knowledge-based support in Non-Destructive Testing for health monitoring of aircraft structures

Maintenance manuals include general methods and procedures for industrial maintenance and they contain information about principles of maintenance methods. Particularly, Non-Destructive Testing (NDT) methods are important for the detection of aeronautical defects and they can be used for various kinds of material and in different environments. Conventional non-destructive evaluation inspections are done at periodic maintenance checks. Usually, the list of tools used in a maintenance program is simply located in the introduction of manuals, without any precision as regards to their characteristics, except for a short description of the manufacturer and tasks in which they are employed. Improving the identification concepts of the maintenance tools is needed to manage the set of equipments and establish a system of equivalence: it is necessary to have a consistent maintenance conceptualization, flexible enough to fit all current equipment, but also all those likely to be added/used in the future. Our contribution is related to the formal specification of the system of functional equivalences that can facilitate the maintenance activities with means to determine whether a tool can be substituted for another by observing their key parameters in the identified characteristics. Reasoning mechanisms of conceptual graphs constitute the baseline elements to measure the fit or unfit between an equipment model and a maintenance activity model. Graph operations are used for processing answers to a query and this graph-based approach to the search method is in-line with the logical view of information retrieval. The methodology described supports knowledge formalization and capitalization of experienced NDT practitioners. As a result, it enables the selection of a NDT technique and outlines its capabilities with acceptable alternatives

Interactive inspection of complex multi-object industrial assemblies

The final publication is available at Springer via http://dx.doi.org/10.1016/j.cad.2016.06.005The use of virtual prototypes and digital models containing thousands of individual objects is commonplace in complex industrial applications like the cooperative design of huge ships. Designers are interested in selecting and editing specific sets of objects during the interactive inspection sessions. This is however not supported by standard visualization systems for huge models. In this paper we discuss in detail the concept of rendering front in multiresolution trees, their properties and the algorithms that construct the hierarchy and efficiently render it, applied to very complex CAD models, so that the model structure and the identities of objects are preserved. We also propose an algorithm for the interactive inspection of huge models which uses a rendering budget and supports selection of individual objects and sets of objects, displacement of the selected objects and real-time collision detection during these displacements. Our solution–based on the analysis of several existing view-dependent visualization schemes–uses a Hybrid Multiresolution Tree that mixes layers of exact geometry, simplified models and impostors, together with a time-critical, view-dependent algorithm and a Constrained Front. The algorithm has been successfully tested in real industrial environments; the models involved are presented and discussed in the paper.Peer ReviewedPostprint (author's final draft

Feature and viewpoint selection for industrial car assembly

Abstract. Quality assurance programs of today’s car manufacturers show increasing demand for automated visual inspection tasks. A typical example is just-in-time checking of assemblies along production lines. Since high throughput must be achieved, object recognition and pose estimation heavily rely on offline preprocessing stages of available CAD data. In this paper, we propose a complete, universal framework for CAD model feature extraction and entropy index based viewpoint selection that is developed in cooperation with a major german car manufacturer

Transfer Learning-Based Crack Detection by Autonomous UAVs

Unmanned Aerial Vehicles (UAVs) have recently shown great performance collecting visual data through autonomous exploration and mapping in building inspection. Yet, the number of studies is limited considering the post processing of the data and its integration with autonomous UAVs. These will enable huge steps onward into full automation of building inspection. In this regard, this work presents a decision making tool for revisiting tasks in visual building inspection by autonomous UAVs. The tool is an implementation of fine-tuning a pretrained Convolutional Neural Network (CNN) for surface crack detection. It offers an optional mechanism for task planning of revisiting pinpoint locations during inspection. It is integrated to a quadrotor UAV system that can autonomously navigate in GPS-denied environments. The UAV is equipped with onboard sensors and computers for autonomous localization, mapping and motion planning. The integrated system is tested through simulations and real-world experiments. The results show that the system achieves crack detection and autonomous navigation in GPS-denied environments for building inspection