Using deep learning to detect the presence/absence of defects on leather: On the way to build an industry-driven approach

In textile/leather manufacturing environments, as in many other industrial contexts, quality inspection is an essential activity that is commonly performed by human operators. Error, fatigue, ergonomic issues, and related costs associated to this fashion of carrying out fabric validation are aspects concerning companies' strategists, whose mission includes to watch over the physical integrity of their employees, while aiming at enhanced quality control methods implementation towards profit maximization. Considering these challenges from a technical/scientific perspective, machine/deep learning approaches have been showing great skills in adapting a wide range of contexts and, in particular, industrial environments, complementing traditional computer vision methods with characteristics such as increased accuracy while dealing with image classification and segmentation problems, capacity for continuous learning from experts input and feedback, flexibility to easily scale training for new contextualization classes – unknown types of occurrences relevant to characterize a given problem –, among other advantages. The goal of crossing deep learning strategies with fabric inspection processes is pursued in this paper. After providing a brief but representative characterization of the targeted industrial context, in which, typically, fabric rolls of rawmaterial mats must be processed at a relatively low latency, an Automatic Optical Inspection (AOI) system architecture designed for such environments is revisited [1], for contextualization purposes. Afterwards, a set of deep learning-oriented training methods/processes is proposed in combination with neural networks built based on Xception architecture, towards the implementation of one of the components that integrate the aforementioned system, from which is expected the identification of presence/absence of defective textile/leather raw material at a low-latency. Several models powered by Xception were trained with different tunning parameters, resorting to datasets variations that were set up from raw images of leather, following different annotation strategies (meticulous and rough). The model that performed better reached 96% of accuracy.ERDF - European Regional Development Fund(undefined

Iteratively Optimized Patch Label Inference Network for Automatic Pavement Disease Detection

We present a novel deep learning framework named the Iteratively Optimized Patch Label Inference Network (IOPLIN) for automatically detecting various pavement diseases that are not solely limited to specific ones, such as cracks and potholes. IOPLIN can be iteratively trained with only the image label via the Expectation-Maximization Inspired Patch Label Distillation (EMIPLD) strategy, and accomplish this task well by inferring the labels of patches from the pavement images. IOPLIN enjoys many desirable properties over the state-of-the-art single branch CNN models such as GoogLeNet and EfficientNet. It is able to handle images in different resolutions, and sufficiently utilize image information particularly for the high-resolution ones, since IOPLIN extracts the visual features from unrevised image patches instead of the resized entire image. Moreover, it can roughly localize the pavement distress without using any prior localization information in the training phase. In order to better evaluate the effectiveness of our method in practice, we construct a large-scale Bituminous Pavement Disease Detection dataset named CQU-BPDD consisting of 60,059 high-resolution pavement images, which are acquired from different areas at different times. Extensive results on this dataset demonstrate the superiority of IOPLIN over the state-of-the-art image classification approaches in automatic pavement disease detection. The source codes of IOPLIN are released on \url{https://github.com/DearCaat/ioplin}.Comment: Revision on IEEE Trans on IT

A study of machine learning object detection performance for phased array ultrasonic testing of carbon fibre reinforced plastics

The growing adoption of Carbon Fibre Reinforced Plastics (CFRPs) in the aerospace industry has resulted in a significant reliance on Non-Destructive Evaluation (NDE) to ensure the quality and integrity of these materials. The interpretation of large amounts of data acquired from automated robotic ultrasonic scanning by expert operators is often time consuming, tedious, and prone to human error creating a bottleneck in the manufacturing process. However, with ever growing trend of computing power and digitally stored NDE data, intelligent Machine Learning (ML) algorithms have been gaining more traction than before for NDE data analysis. In this study, the performance of ML object detection models, statistical methods for defect detection, and traditional amplitude thresholding approaches for defect detection in CFRPs were compared. A novel augmentation technique was used to enhance synthetically generated datasets used for ML model training. All approaches were tested on real data obtained from an experimental setup mimicking industrial conditions, with ML models showing improvement over amplitude thresholding and statistical thresholding techniques. The advantages and limitations of all methods are reported and discussed

Computer Vision-Based Automatic Railroad Crossing Monitoring and Track Inspection

Currently, there are many imminent challenges in the railroad infrastructure system of the United States, impacting the operation, safety, and management of railroad transportation. In this work, three major challenges which are overcrowded traffic congestion at the grade crossing, low-efficiency and accuracy on inspection of missing or broken rail track components, and dense rail surface defects without quantification, respectively are studied. The congested railroad grade crossing not only introduces significant traffic delays to travelers but also brings potential safety concerns to the first responders. However, limited studies have been devoted on developing an intelligent traffic monitoring system which is significant to deliver real-time information to the travelers and the first responders to improve the traffic operation and safety at the railroad grade crossing. Except to improve the railroad safety related with travelers and the first responders in the first half, the rest of this dissertation focuses on the track safety related to railroad track components and surface defects. The missing or broken components such as spikes, clips, and tie plates can endanger the safety and operation of railroads. Even though various types of inspection approaches such as ground penetrating radar, laser, and LiDAR have been implemented, the operation needs rich experience and extensive training. Meanwhile, track inspections still heavily rely on manual inspection which is low-accurate, low-efficient, and highly subjective. Moreover, rail surface defects negatively impact riding comfort, operational safety, and could even lead to train derailments. During the past decades, there have been many efforts to detect rail surface defects. Unfortunately, previous approaches for detecting and quantifying of rail surface defects are also limited by the high requirements of specialized equipment and personnel training. The main focus of this work is to design and develop computer vision models to address the technical and practical challenges mentioned above. To cope with each challenge, different models including the object detection model, the instance segmentation model, and the semantic segmentation model have been successfully designed and developed. To train, validate, and test different models, three customized image datasets based on the traffic videos at the grade crossing, railroad component images, and dense rail surface defects images have been built. Specifically, a dense traffic detection net (DTDNet) is developed integrating the Transformer Attention (TA) module for better modeling of global context information and the learning-to-match detection head for optimizing object detection and localization using a likelihood probability fashion. A unique grade crossing traffic image dataset including congested and normal traffic during both daytime and nighttime is established. The proposed DTDNet and other state-of-the-art (SOTA) models have been trained, tested, and compared. The proposed DTDNet outperforms other SOTA models in the test cases. Regarding the automatic track components inspection, the real-time instance segmentation model and the YOLOv4-hybrid model have been designed, trained, tested, and evaluated. The first public rail components image database has been built and released online. Compared to the original YOLACT model and the Mask R-CNN model, the training performance has been improved with the improved instance segmentation model. The detection accuracy on the bounding box and the mask has been improved and the inference speed can achieve the real-time speed. With respect to the YOLOv4-hybrid model, it outperforms other SOTA models on the training performance and the field tests with missing or fake rail track components. As for the rail surface defect inspection and quantification, the optimized Mask R-CNN model and the newly proposed lightweight Deeplabv3Plus model using Lovász-Softmax loss (LDL model) have been trained, tested, evaluated, and compared on our rail surface defects image database. Experimental results confirm the robustness and superiority of our model on defect segmentation. Besides, an algorithm is proposed to quantify rail surface defect severities at different levels using our rail surface defects image data. Overall, this dissertation helps to improve the railroad safety by developing and implementing advanced computer vision-based models for better tracking monitoring and inspections

Fusion of colour contrasted images for early detection of oesophageal squamous cell dysplasia from endoscopic videos in real time

Standard white light (WL) endoscopy often misses precancerous oesophageal changes due to their only subtle differences to the surrounding normal mucosa. While deep learning (DL) based decision support systems benefit to a large extent, they face two challenges, which are limited annotated data sets and insufficient generalisation. This paper aims to fuse a DL system with human perception by exploiting computational enhancement of colour contrast. Instead of employing conventional data augmentation techniques by alternating RGB values of an image, this study employs a human colour appearance model, CIECAM, to enhance the colours of an image. When testing on a frame of endoscopic videos, the developed system firstly generates its contrast-enhanced image, then processes both original and enhanced images one after another to create initial segmentation masks. Finally, fusion takes place on the assembled list of masks obtained from both images to determine the finishing bounding boxes, segments and class labels that are rendered on the original video frame, through the application of non-maxima suppression technique (NMS). This deep learning system is built upon real-time instance segmentation network Yolact. In comparison with the same system without fusion, the sensitivity and specificity for detecting early stage of oesophagus cancer, i.e. low-grade dysplasia (LGD) increased from 75% and 88% to 83% and 97%, respectively. The video processing/play back speed is 33.46 frames per second. The main contribution includes alleviation of data source dependency of existing deep learning systems and the fusion of human perception for data augmentation

Transmission line bolts and their defects detection method based on position relationship

Introduction: To solve the problems of small proportion of bolts in aerial images of power transmission lines, small differences between classes, and difficulty in extracting refined features, this paper proposes a method for detecting power transmission line bolts and their defects based on positional relationships.Methods: Firstly, a spatial attention module is added to Faster R-CNN, using two parallel cross attention to obtain cross path features and global features respectively, and spatial feature enhancement is performed on the features output from the convolution layer. Then, starting from the spatial position relationship of bolts and their defects, using the relative geometric features of candidate regions as input, the spatial position relationship of bolts and their defects on the image is modeled. Finally, the position features and regional features are connected to obtain enhanced features. The bolt position knowledge on the connecting plate is added to the detection model to improve the detection accuracy of the model.Results and discussion: The experimental results show that the mAP value of the algorithm in this paper is increased by 6.61% compared to the Faster R-CNN detection model in aerial photography of transmission line bolts and their defect datasets, with the AP value of normal bolts increased by 1.73%, the AP value of pin losing increased by 4.45%, and the AP value of nut losing increased by 13.63%

Enhancing Road Infrastructure Monitoring: Integrating Drones for Weather-Aware Pothole Detection

The abstract outlines the research proposal focused on the utilization of Unmanned Aerial Vehicles (UAVs) for monitoring potholes in road infrastructure affected by various weather conditions. The study aims to investigate how different materials used to fill potholes, such as water, grass, sand, and snow-ice, are impacted by seasonal weather changes, ultimately affecting the performance of pavement structures. By integrating weather-aware monitoring techniques, the research seeks to enhance the rigidity and resilience of road surfaces, thereby contributing to more effective pavement management systems. The proposed methodology involves UAV image-based monitoring combined with advanced super-resolution algorithms to improve image refinement, particularly at high flight altitudes. Through case studies and experimental analysis, the study aims to assess the geometric precision of 3D models generated from aerial images, with a specific focus on road pavement distress monitoring. Overall, the research aims to address the challenges of traditional road failure detection methods by exploring cost-effective 3D detection techniques using UAV technology, thereby ensuring safer roadways for all users

An objective comparison of detection and segmentation algorithms for artefacts in clinical endoscopy

We present a comprehensive analysis of the submissions to the first edition of the Endoscopy Artefact Detection challenge (EAD). Using crowd-sourcing, this initiative is a step towards understanding the limitations of existing state-of-the-art computer vision methods applied to endoscopy and promoting the development of new approaches suitable for clinical translation. Endoscopy is a routine imaging technique for the detection, diagnosis and treatment of diseases in hollow-organs; the esophagus, stomach, colon, uterus and the bladder. However the nature of these organs prevent imaged tissues to be free of imaging artefacts such as bubbles, pixel saturation, organ specularity and debris, all of which pose substantial challenges for any quantitative analysis. Consequently, the potential for improved clinical outcomes through quantitative assessment of abnormal mucosal surface observed in endoscopy videos is presently not realized accurately. The EAD challenge promotes awareness of and addresses this key bottleneck problem by investigating methods that can accurately classify, localize and segment artefacts in endoscopy frames as critical prerequisite tasks. Using a diverse curated multi-institutional, multi-modality, multi-organ dataset of video frames, the accuracy and performance of 23 algorithms were objectively ranked for artefact detection and segmentation. The ability of methods to generalize to unseen datasets was also evaluated. The best performing methods (top 15%) propose deep learning strategies to reconcile variabilities in artefact appearance with respect to size, modality, occurrence and organ type. However, no single method outperformed across all tasks. Detailed analyses reveal the shortcomings of current training strategies and highlight the need for developing new optimal metrics to accurately quantify the clinical applicability of methods