Weld bead detection based on 3D geometric features and machine learning approaches

14 p.Weld bead detection is essential for automated welding inspection processes. The non-invasive passive techniques, such as photogrammetry, are quickly evolving to provide a 3D point cloud with submillimeter precision and spatial resolution. However, its application in weld visual inspection has not been extensively studied. The derived 3D point clouds, despite the lack of topological information, store significant information for the weld-plaque segmentation. Although the weld bead detection is being carried out over images or based on laser profiles, its characterization by means of 3D geometrical features has not been assessed. Moreover, it is possible to combine machine learning approaches and the 3D features in order to realize the full potential of the weld bead segmentation of 3D submillimeter point clouds. In this paper, the novelty is focused on the study of 3D features on real cases to identify the most relevant ones for weld bead detection on the basis of the information gain. For this novel contribution, the influence of neighborhood size for covariance matrix computation, decision tree algorithms, and split criteria are analyzed to assess the optimal results. The classification accuracy is evaluated by the degree of agreement of the classified data by the kappa index and the area under the receiver operating characteristic (ROC) curve. The experimental results show that the proposed novel methodology performs better than 0.85 for the kappa index and better than 0.95 for ROC area.S

Unsupervised learning-based approach for detecting 3D edges in depth maps

3D edge features, which represent the boundaries between different objects or surfaces in a 3D scene, are crucial for many computer vision tasks, including object recognition, tracking, and segmentation. They also have numerous real-world applications in the field of robotics, such as vision-guided grasping and manipulation of objects. To extract these features in the noisy real-world depth data, reliable 3D edge detectors are indispensable. However, currently available 3D edge detection methods are either highly parameterized or require ground truth labelling, which makes them challenging to use for practical applications. To this extent, we present a new 3D edge detection approach using unsupervised classification. Our method learns features from depth maps at three different scales using an encoder-decoder network, from which edge-specific features are extracted. These edge features are then clustered using learning to classify each point as an edge or not. The proposed method has two key benefits. First, it eliminates the need for manual fine-tuning of data-specific hyper-parameters and automatically selects threshold values for edge classification. Second, the method does not require any labelled training data, unlike many state-of-the-art methods that require supervised training with extensive hand-labelled datasets. The proposed method is evaluated on five benchmark datasets with single and multi-object scenes, and compared with four state-of-the-art edge detection methods from the literature. Results demonstrate that the proposed method achieves competitive performance, despite not using any labelled data or relying on hand-tuning of key parameters.</p

Model Simplification for Efficient Collision Detection in Robotics

Motion planning for industrial robots is a computationally intensive task due to the massive number of potential motions between any two configurations. Calculating all possibilities is generally not feasible. Instead, many motion planners sample a sub-set of the available space until a viable solution is found. Simplifying models to improve collision detection performance, a significant component of motion planning, results in faster and more capable motion planners. Several approaches for simplifying models to improve collision detection performance have been presented in the literature. However, many of them are sub-optimal for an industrial robotics application due to input model limitations, accuracy sacrifices, or the probability of increasing false negatives during collision queries. This thesis focuses on the development of model simplification approaches optimised for industrial robotics applications. Firstly, a new simplification approach, the Bounding Sphere Simplification (BSS), is presented that converts triangle-mesh inputs to a collection of spheres for efficient collision and distance queries. Additionally, BSS removes small features and generates an output model less prone to false negatives

Design and implementation of robot skill programming and control

Abstract. Skill-based approach has been represented as a solution to the raising complicity of robot programming and control. The skills rely heavily on the use of sensors integrating sensor perceptions and robot actions, which enable the robot to adapt to changes and uncertainties in the real world and operate autonomously. The aim of this thesis was to design and implement a programming concept for skill-based control of industrial robots. At the theoretical part of this thesis, the industrial robot system is introduced as well as some basic concepts of robotics. This is followed by the introduction of different robot programming and 3D machine vision methods. At the last section of the theoretical part, the structure of skill-based programs is presented. In the experimental part, structure of the skills required for the “grinding with localization” -task are presented. The task includes skills such as global localization with 3D-depth sensor, scanning the object with 2D-profile scanner, precise localization of the object as well as two grinding skills: level surface grinding and straight seam grinding. Skills are programmed with an off-line programming tool and implemented in a robot cell, composed of a standard industrial robot with grinding tools, 3D-depth sensors and 2D-profile scanners. The results show that global localization can be carried out with consumer class 3D-depth sensors and more accurate local localization with an industrial high-accuracy 2D-profile scanner attached to the robot’s flange. The grinding experiments and tests were focused on finding suitable structures of the skill programs as well as to understand how the different parameters influence on the quality of the grinding.Robotin taitopohjaisten ohjelmien ohjelmointi ja testaus. Tiivistelmä. Robotin taitopohjaisia ohjelmia on esitetty ratkaisuksi robottien jatkuvasti monimutkaistuvaan ohjelmointiin. Taidot pohjautuvat erilaisten antureiden ja robotin toimintojen integroimiseen, joiden avulla robotti pystyy havainnoimaan muutokset reaalimaailmassa ja toimimaan autonomisesti. Tämän työn tavoitteena oli suunnitella ja toteuttaa taitopohjaisia ohjelmia teollisuusrobotille. Aluksi työn teoriaosuudessa esitellään teollisuusrobottijärjestelmään kuuluvia osia ja muutamia robotiikan olennaisimpia käsitteitä. Sen jälkeen käydään läpi eri robotin ohjelmointitapoja ja eri 3D-konenäön toimintaperiaatteita. Teoriaosuuden lopussa esitellään taitopohjaisten ohjelmien rakennetta. Käytännön osuudessa esitellään ”hionta paikoituksella” -tehtävän suoritukseen tarvittavien taitojen rakenne. Tehtävän vaatimia taitoja ovat muun muassa kappaleen globaalipaikoitus 3D-syvyyskameralla, kappaleen skannaus 2D-profiiliskannerilla, kappaleen tarkkapaikoitus ja kaksi eri hiontataitoa: tasomaisen pinnan ja suoran sauman hionta. Taidot ohjelmoidaan off-line ohjelmointityökalulla ja implementoidaan robottisoluun, joka muodostuu hiontatyökaluilla varustetusta teollisuusrobotista, 3D-kameroista ja 2D-profiiliskannereista. Työn tuloksista selviää, että kappaleen globaalipaikoitus voidaan suorittaa kuluttajille suunnatuilla 3D-syvyyskameroilla ja kappaleen tarkempi lokaalipaikoitus robotin ranteeseen kiinnitetyllä teollisuuden käyttämillä 2D-profiiliskannereilla. Hiontojen kokeellisessa osuudessa etsitään ohjelmien oikeanlaista rakennetta sekä muodostetaan käsitys eri parametrien vaikutuksesta hionnan laatuun

Plane-based 3D Mapping for Structured Indoor Environment

Three-dimensional (3D) mapping deals with the problem of building a map of the unknown environments explored by a mobile robot. In contrast to 2D maps, 3D maps contain richer information of the visited places. Besides enabling robot navigation in 3D, a 3D map of the robot surroundings could be of great importance for higher-level robotic tasks, like scene interpretation and object interaction or manipulation, as well as for visualization purposes in general, which are required in surveillance, urban search and rescue, surveying, and others. Hence, the goal of this thesis is to develop a system which is capable of reconstructing the surrounding environment of a mobile robot as a three-dimensional map. Microsoft Kinect camera is a novel sensing sensor that captures dense depth images along with RGB images at high frame rate. Recently, it has dominated the stage of 3D robotic sensing, as it is low-cost, low-power. For this work, it is used as the exteroceptive sensor and obtains 3D point clouds of the surrounding environment. Meanwhile, the wheel odometry of the robot is used to initialize the search for correspondences between different observations. As a single 3D point cloud generated by the Microsoft Kinect sensor is composed of many tens of thousands of data points, it is necessary to compress the raw data to process them efficiently. The method chosen in this work is to use a feature-based representation which simplifies the 3D mapping procedure. The chosen features are planar surfaces and orthogonal corners, which is based on the fact that indoor environments are designed such that walls, ground floors, pillars, and other major parts of the building structures can be modeled as planar surface patches, which are parallel or perpendicular to each other. While orthogonal corners are presented as higher features which are more distinguishable in indoor environment. In this thesis, the main idea is to obtain spatial constraints between pairwise frames by building correspondences between the extracted vertical plane features and corner features. A plane matching algorithm is presented that maximizes the similarity metric between a pair of planes within a search space to determine correspondences between planes. The corner matching result is based on the plane matching results. The estimated spatial constraints form the edges of a pose graph, referred to as graph-based SLAM front-end. In order to build a map, however, a robot must be able to recognize places that it has previously visited. Limitations in sensor processing problem, coupled with environmental ambiguity, make this difficult. In this thesis, we describe a loop closure detection algorithm by compressing point clouds into viewpoint feature histograms, inspired by their strong recognition ability. The estimated roto-translation between detected loop frames is added to the graph representing this newly discovered constraint. Due to the estimation errors, the estimated edges form a non-globally consistent trajectory. With the aid of a linear pose graph optimizing algorithm, the most likely configuration of the robot poses can be estimated given the edges of the graph, referred to as SLAM back-end. Finally, the 3D map is retrieved by attaching each acquired point cloud to the corresponding pose estimate. The approach is validated through different experiments with a mobile robot in an indoor environment