Deep learning computer vision for robotic disassembly and servicing applications

Fastener detection is a necessary step for computer vision (CV) based robotic disassembly and servicing applications. Deep learning (DL) provides a robust approach for creating CV models capable of generalizing to diverse visual environments. Such DL CV systems rely on tuning input resolution and mini-batch size parameters to fit the needs of the detection application. This paper provides a method for determining the optimal compromise between input resolution and mini-batch size to determine the highest performance for cross-recessed screw (CRS) detection while utilizing maximum graphics processing unit resources. The Tiny-You Only Look Once v2 (Tiny-YOLO v2) DL object detection system was chosen to evaluate this method. Tiny-YOLO v2 was employed to solve the specialized task of detecting CRS which are highly common in electronic devices. The method used in this paper for CRS detection is meant to lay the ground-work for multi-class fastener detection, as the method is not dependent on the type or number of object classes. An original dataset of 900 images of 12.3 MPx resolution was manually collected and annotated for training. Three additional distinct datasets of 90 images each were manually collected and annotated for testing. It was found an input resolution of 1664 x 1664 pixels paired with a mini-batch size of 16 yielded the highest average precision (AP) among the seven models tested for all three testing datasets. This model scored an AP of 92.60% on the first testing dataset, 99.20% on the second testing dataset, and 98.39% on the third testing dataset

Automated Defect Detection of Screws in the Manufacturing Industry Using Convolutional Neural Networks

Defect detection in industrial production processes is an important and necessary part of quality control. Many defects can occur during the manufacturing process, causing high manufacturing costs. Thus the inspection of screws, which represent an indispensable element of many mechanical components, is a critical process. To reduce manufacturing costs and increase efficiency, a reliable method for inspection is Deep Learning. It can help simplify the process of quality control and increase the velocity and volume of detected defects in screws. This approach uses a CNN model to classify non-defective and defective screws with different types of defects. Instead of manual quality control methods, which can be easily biased, our CNN approach is accurate, cost-efficient, and fast, with an accuracy of over 97 percent. With this approach corresponding to industrial production processes, different defects in screws and non-defective screws can be classified from images according to a real-world industrial inspection scenario

Quality Control Analysis to Detect Defects in Drywall Fastener Screws

Los tornillos son piezas diseñadas para unir dos o más elementos, compuestos por tres partes principales: cabeza, cuello y rosca. Dependiendo del tipo de material del que estén hechos, su tamaño y funcionalidad pueden adquirir diferentes características. En este artículo se describe la forma en que se evaluó una muestra de 200 tornillos de 2 pulgadas utilizados para drywall con el fin de detectar posibles defectos en su proceso de fabricación, mediante la aplicación de herramientas de calidad, con lo cual se analizan los principales factores que intervienen para lograr la mejor calidad. de estos productos fueron determinados. Además, el estudio de un caso real nos permitió demostrar mejoras factibles que se pueden aplicar no solo en un área determinada, sino en toda la industria, ya que la calidad de un producto también determina el crecimiento potencial de una empresa. Los resultados obtenidos mostraron el grado de afectación de cada defecto, siendo recomendable que la empresa se enfoque en tornillos torcidos, tornillos con puntas deformadas y tornillos con medidas incorrectas, ya que estos representan mayores pérdidas. En conclusión, en la industria es necesario el uso y manejo de herramientas para incrementar la productividad y calidad de los procesos, logrando así un impacto significativo en todas las áreas de producción.Screws are pieces designed to join two or more elements, they are composed of three parts that are thread, neck, and head. Depending on the type of material they are made of their size and functionality can acquire different characteristics. This article describes the way in which a sample of 200 2-inch screws used for drywall were evaluated to detect potential failures in their manufacturing, through the application of quality tools, with which the main factors that intervene to achieve the best quality of these products were determined. In addition, the study of a real case allowed us to demonstrate feasible improvements that can be applied not only in a certain area, but in the whole industry, since the quality of a product also determines the potential growth of a company. The results obtained showed the degree of affectation of each defect, being advisable for the company to focus on bent screws, screws with deformed tips and screws with incorrect measurements, as these represent greater losses. In conclusion, in the industry it is necessary to use and manage tools to increase the productivity and quality of the processes, thus having a significant impact on all areas of production, achieving the best possible final product

Robotic disassembly of waste electrical and electronic equipment

Waste electrical and electronic equipment (WEEE) is the world’s fastest growing form of waste. Inappropriate disposal of WEEE causes damage to ecosystems and local communities due to hazardous materials and toxic chemicals present in electronic products. High value metals in small quantities are dissipated and embodied energy from manufacturing are lost in shredding and crushing treatments of WEEE. On the other hand, manual disassembly is costly and presents safety concerns for human workers. Therefore, robotic disassembly is an ideal approach to addressing the treatment of WEEE. Despite extensive research in the field, large variations and uncertainties in product structures, models, and conditions is a major limitation to the implementation of automation and robotics in the waste industry. The ability of a robotic disassembly system to learn new product structures and reason about existing knowledge of product structure is vital to addressing this challenge. This thesis explores robotic disassembly for WEEE by building upon an existing research disassembly rig for LCD monitors and expanding it to address other product families. The updated disassembly system utilizes a modular framework consisting of a Cognition module, Perception module, and Operation module, in order to address the uncertainties present in end-of-life (EoL) products. A novel disassembly ontology is designed and developed with an upper and lower ontology structure to represent generic disassembly knowledge and product-family-specific knowledge respectively. Furthermore, a Learning framework enables automated expansion of the ontology using past disassembly experiences and user-demonstration. These presented methodologies form the main function of the Cognition module, which aids the Perception module and instructs the Operation module. The disassembly ontology and Learning framework are verified independently from the rest of the system prior to being integrated and validated with real disassembly runs of LCD monitors and keyboards. As such, the disassembly system’s ability to address both known and unknown EoL product types, as well as learn new product types, is demonstrated

Automatic identification of mechanical parts for robotic disassembly using deep neural network techniques

This work addressed the automatic visual identification of mechanical objects from 3D camera scans, and is part of a wider project focusing on automatic disassembly for remanufacturing. The main challenge of the task was the intrinsic uncertainties on the state of end-of-life products, which required a highly robust identification system. The use of point cloud models implied also the need to deal with significant computational overheads. The state-of-the-art PointNet deep neural network was chosen as the classifier system, due to its learning capabilities, suitability to processing 3D models, and ability to recognise objects irrespective of their pose. To obviate the need for collecting a large set of training models, it was decided that PointNet was to be trained using examples generated from 3D CAD models, and used on scans of real objects. Different tests were carried out to assess PointNet ability to deal with imprecise sensor readings and partial views. Due to restrictions on access due to the pandemic, it was not possible to collect a sufficiently systematic set of scans of physical objects in the lab. Various tests were thus carried out using combinations of CAD models of mechanical and everyday objects, primitive geometric shapes, and real scans of everyday objects from popular machine vision benchmarks. The investigation confirmed PointNet’s ability to recognise complex mechanical objects and irregular everyday shapes with good accuracy, generalising the results of learning from geometric shapes and CAD models. The performance of PointNet was not significantly affected by the use of partial views of the objects, a very common case in industrial applications. PointNet showed some limitations when tasked with recognising noisy scenes, and a practical solution was suggested to minimise this problem. To reduce the computational complexity of training a deep architecture using large data sets of 3D scenes, a predator-prey coevolutionary scheme was devised. The proposed algorithm evolves subsets of the training set, selecting for these subsets the most difficult examples. The remaining training samples are discarded by the evolutionary procedure, which thus reduces the number of examples that are presented to the classifier. The experimental results showed that this economy of training samples allows reducing the execution time of the learning procedure, without affecting the neural network recognition accuracy. This simplification of the learning procedure is of general importance for the whole deep learning field, since practical implementations are often hindered by the complexity of the training process