Automatic detection of welding defects using the convolutional neural network

Quality control of welded joints is an important step before commissioning of various types of metal structures. The main obstacles to the commissioning of such facilities are the areas where the welded joint deviates from acceptable defective standards. The defects of welded joints include non-welded, foreign inclusions, cracks, pores, etc. The article describes an approach to the detection of the main types of defects of welded joints using a combination of convolutional neural networks and support vector machine methods. Convolutional neural networks are used for primary classification. The support vector machine is used to accurately define defect boundaries. As a preprocessing in our work, we use the methods of morphological filtration. A series of experiments confirms the high efficiency of the proposed method in comparison with pure CNN method for detecting defects

Automatic Lumbar Vertebrae Segmentation in Fluoroscopic Images via Optimised Concurrent Hough Transform

Low back pain is a very common problem in the industrialised countries and its associated cost is enormous. Diagnosis of the underlying causes can be extremely difficult. Many studies have focused on mechanical disorders of the spine. Digital videofluoroscopy (DVF) was widely used to obtain images for motion studies. This can provide motion sequences of the lumbar spine, but the images obtained often suffer due to noise, exacerbated by the very low radiation dosage. Thus determining vertebrae position within the image sequence presents a considerable challenge. In this paper, we show how our new approach can automatically detect the positions and borders of vertebrae concurrently, relieving many of the problems experienced in other approaches. First, we use phase congruency to relieve difficulty associated with threshold selection in edge detection of the illumination variant DVF images. Then, our new Hough transform approach is applied to determine the moving vertebrae, concurrently. We include optimisation via a genetic algorithm as without it the extraction of moving multiple vertebrae is computationally daunting. Our results show that this new approach can indeed provide extractions of position and rotation which appear to be of sufficient quality to aid therapy and diagnosis of spinal disorders

Machine Vision Application For Automatic Defect Segmentation In Weld Radiographs

Objektif penyelidikan ini adalah untuk membangunkan satu kaedah peruasan kecacatan kimpalan automatik yang boleh meruas pelbagai jenis kecacatan kimpalan yang wujud dalam imej radiografi kimpalan. Kaedah segmentasi kecacatan automatik yang dibangunkan terdir:i daripada tiga algoritma utama, iaitu algoritma penyingkiran label, algoritma pengenalpastian bahagian kimpalan dan algoritma segmentasi kecacatan kimpalan. Algoritma penyingkiran label dibangunkan untuk mengenalpasti dan menyingkirkan label yang terdapat pada imej radiograf kimpalan secara automatik, sebelum algoritma pengenalpastian bahagian kimpalan dan algortima segmentasi kecacatan diaplikasikan ke atas imej radiografi. Satu algoritma pengenalpastian bahagian kimpalan juga dibangunkan dengan tujuan mengenalpasti bahagian kimpalan dalam imej radiogaf secara automatik dengan menggunakan profil keamatan yang diperoleh daripada imej radiografi. The objective of the research is to develop an automatic weld defect segmentation methodology to segment different types of defects in radiographic images of welds. The segmentation methodology consists of three main algorithms. namely label removal algorithm. weld extraction algorithm and defect segmentation algorithm. The label removal algorithm was developed to detect and remove labels that are printed on weld radiographs automatically before weld extraction algorithm and defect detection algorithm are applied. The weld extraction algorithm was developed to locate and extract welds automatically from the intensity profiles taken across the image by using graphical analysis. This algorithm was able to extract weld from a radiograph regardless of whether the intensity profile is Gaussian or otherwise. This method is an improvement compared to the previous weld extraction methods which are limited to weld image with Gaussian intensity profiles. Finally. a defect segmentation algorithm was developed to segment the defects automatically from the image using background subtraction and rank leveling method

A study of hough transform for weld extraction

The process of joining metals is called welding. At times, selecting a poor quality material or improper usage of welding technologies may cause defects in welded joints. Some of these welded joints have to be tested nondestructively, because their failure can cause lot of damage, for instance in power plants. Radiography is a very common method for non-destructive testing of welds. It is done by certified weld inspectors who have knowledge about weld flaws, looking at the radiograph of the welded joint with naked eye. The judgment of the weld inspector can be biased; subjective, because it is dependent on his/her experience. This manual method can also become very time consuming. Many researches were exploring computer aided examination of radiographic images in early 1990’s. With much advancement in computer vision and image processing technologies, they are being used to find more effective ways of automatic weld inspection. These days, fuzzy based methods are being widely used in this area too. The first step in automatic weld inspection is to locate the welds or find a Region of Interest (ROI) in the radiographic image [7]. In this thesis, a Standard Hough Transform (SHT) based methodology is developed for weld extraction. Firstly, we have done binarization of image to remove the background and non-welds. For binarization, optimal binary threshold is found by a metaheuristic –Simulated annealing. Secondly, we use SHT to generate the Hough Transform matrix of all non-zero points in the binary image. Thirdly, we have explored two different paths to find a meaningful set of lines in the binarized image that are welds. Finally, these lines are verified as weld using a weld-peak detection procedure. Weld-peak detection is also helpful to remove any non-welds that were remaining. We have used 25 digitized radiographic images containing 100 welds to test the method in terms of true detection and false alarm rate

심층신경망을 이용한 자동화된 치과 의료영상 분석

학위논문(박사) -- 서울대학교대학원 : 치과대학 치의과학과, 2021.8. 한중석.목 적: 치과 영역에서도 심층신경망(Deep Neural Network) 모델을 이용한 방사선사진에서의 임플란트 분류, 병소 위치 탐지 등의 연구들이 진행되었으나, 최근 개발된 키포인트 탐지(keypoint detection) 모델 또는 전체적 구획화(panoptic segmentation) 모델을 의료분야에 적용한 연구는 아직 미비하다. 본 연구의 목적은 치근단 방사선사진에서 키포인트 탐지를 이용해 임플란트 골 소실 정도를 파악하는 모델과 panoptic segmentation을 파노라마영상에 적용하여 다양한 구조물들을 구획화하는 모델을 학습시켜 진료에 보조적으로 활용되도록 만들어보고, 이 모델들의 추론결과를 평가해보는 것이다. 방 법: 객체 탐지 및 구획화에 있어 널리 연구된 합성곱 신경망 모델인 Mask-RCNN을 키포인트 탐지가 가능한 형태로 준비하여 치근단 방사선사진에서 임플란트의 top, apex, 그리고 bone level 지점을 좌우로 총 6지점 탐지하게끔 학습시킨 뒤, 학습에 사용되지 않은 시험 데이터셋을 대상으로 탐지시킨다. 키포인트 탐지 평가용 지표인 object keypoint similarity (OKS) 및 이를 이용한 average precision (AP) 값을 계산하고, 평균 OKS값을 통해 모델 및 치과의사의 결과를 비교한다. 또한, 탐지된 키포인트를 바탕으로 방사선사진상에서의 골 소실 정도를 수치화한다. Panoptic segmentation을 위해서는 기존의 벤치마크에서 우수한 성적을 거둔 신경망 모델인 Panoptic DeepLab을 파노라마영상에서 주요 구조물(상악동, 상악골, 하악관, 하악골, 자연치, 치료된 치아, 임플란트)을 구획화하도록 학습시킨 뒤, 시험 데이터셋에서의 구획화 결과에 panoptic / semantic / instance segmentation 각각의 평가지표들을 적용하고, 픽셀들의 정답(ground truth) 클래스와 모델이 추론한 클래스에 대한 confusion matrix를 계산한다. 결 과: OKS값을 기반으로 계산한 키포인트 탐지 AP는, 모든 OKS threshold에 대한 평균의 경우, 상악 임플란트에서는 0.761, 하악 임플란트에서는 0.786이었다. 평균 OKS는 모델이 0.8885, 치과의사가 0.9012로, 통계적으로 유의미한 차이가 없었다 (p = 0.41). 모델의 평균 OKS 값은 사람의 키포인트 어노테이션 정규분포상에서 상위 66.92% 수준이었다. 파노라마영상 구조물 구획화에서는, panoptic segmentation 평가지표인 panoptic quality 값의 경우 모든 클래스의 평균은 80.47이었으며, 치료된 치아가 57.13으로 가장 낮았고 하악관이 65.97로 두번째로 낮은 값을 보였다. Semantic segmentation 평가지표인 global한 Intersection over Union (IoU) 값은 모든 클래스 평균 0.795였으며, 하악관이 0.639로 가장 낮았고 치료된 치아가 0.656으로 두번째로 낮은 값을 보였다. Confusion matrix 계산 결과, ground truth 픽셀들 중 올바르게 추론된 픽셀들의 비율은 하악관이 0.802로 가장 낮았다. 개별 객체에 대한 IoU를 기반으로 계산한 Instance segmentation 평가지표인 AP값은, 모든 IoU threshold에 대한 평균의 경우, 치료된 치아가 0.316, 임플란트가 0.414, 자연치가 0.520이었다. 결 론: 키포인트 탐지 신경망 모델을 이용하여, 치근단 방사선사진에서 임플란트의 주요 지점을 사람과 다소 유사한 수준으로 탐지할 수 있었다. 또한, 탐지된 지점들을 기반으로 방사선사진상에서의 임플란트 주위 골 소실 비율 계산을 자동화할 수 있고, 이 값은 임플란트 주위염의 심도 분류에 사용될 수 있다. 파노라마 영상에서는 panoptic segmentation이 가능한 신경망 모델을 이용하여 상악동과 하악관을 포함한 주요 구조물들을 구획화할 수 있었다. 따라서, 이와 같이 각 작업에 맞는 심층신경망을 적절한 데이터로 학습시킨다면 진료 보조 수단으로 활용될 수 있다.Purpose: In dentistry, deep neural network models have been applied in areas such as implant classification or lesion detection in radiographs. However, few studies have applied the recently developed keypoint detection model or panoptic segmentation model to medical or dental images. The purpose of this study is to train two neural network models to be used as aids in clinical practice and evaluate them: a model to determine the extent of implant bone loss using keypoint detection in periapical radiographs and a model that segments various structures on panoramic radiographs using panoptic segmentation. Methods: Mask-RCNN, a widely studied convolutional neural network for object detection and instance segmentation, was constructed in a form that is capable of keypoint detection, and trained to detect six points of an implant in a periapical radiograph: left and right of the top, apex, and bone level. Next, a test dataset was used to evaluate the inference results. Object keypoint similarity (OKS), a metric to evaluate the keypoint detection task, and average precision (AP), based on the OKS values, were calculated. Furthermore, the results of the model and those arrived at by a dentist were compared using the mean OKS. Based on the detected keypoint, the peri-implant bone loss ratio was obtained from the radiograph. For panoptic segmentation, Panoptic DeepLab, a neural network model ranked high in the previous benchmark, was trained to segment key structures in panoramic radiographs: maxillary sinus, maxilla, mandibular canal, mandible, natural tooth, treated tooth, and dental implant. Then, each evaluation metric of panoptic, semantic, and instance segmentation was applied to the inference results of the test dataset. Finally, the confusion matrix for the ground truth class of pixels and the class inferred by the model was obtained. Results: The AP of keypoint detection for the average of all OKS thresholds was 0.761 for the upper implants and 0.786 for the lower implants. The mean OKS was 0.8885 for the model and 0.9012 for the dentist; thus, the difference was not statistically significant (p = 0.41). The mean OKS of the model was in the top 66.92% of the normal distribution of human keypoint annotations. In panoramic radiograph segmentation, the average panoptic quality (PQ) of all classes was 80.47. The treated teeth showed the lowest PQ of 57.13, and the mandibular canal showed the second lowest PQ of 65.97. The Intersection over Union (IoU) was 0.795 on average for all classes, where the mandibular canal showed the lowest IoU of 0.639, and the treated tooth showed the second lowest IoU of 0.656. In the confusion matrix, the proportion of correctly inferred pixels among the ground truth pixels was the lowest in the mandibular canal at 0.802. The AP, averaged for all IoU thresholds, was 0.316 for the treated tooth, 0.414 for the dental implant, and 0.520 for the normal tooth. Conclusion: Using the keypoint detection neural network model, it was possible to detect major landmarks around dental implants in periapical radiographs to a degree similar to that of human experts. In addition, it was possible to automate the calculation of the peri-implant bone loss ratio on periapical radiographs based on the detected keypoints, and this value could be used to classify the degree of peri-implantitis. In panoramic radiographs, the major structures including the maxillary sinus and the mandibular canal could be segmented using a neural network model capable of panoptic segmentation. Thus, if deep neural networks suitable for each task are trained using suitable datasets, the proposed approach can be used to assist dental clinicians.Chapter 1. Introduction 1 Chapter 2. Materials and methods 5 Chapter 3. Results 23 Chapter 4. Discussion 32 Chapter 5. Conclusions 45 Published papers related to this study 46 References 47 Abbreviations 52 Abstract in Korean 53 Acknowledgements 56박