During the assembly of a control cabinet, a worker obstructs many individual configured wires. To distinguish these wires, a printer plots an identifying text on each end of the wires. However, due to the shape of the wires and the printing process, the quality of these markings is often too low, and it is hard or impossible to read the marking. Common reasons are a low contrast or a blurred text. By now, there is no quality check of the marking after a crimping machine produced the wire. This paper investigates methods for wire mark reading that is required to estimate the quality of the marking. By using optical character recognition, the likeliness that a worker can read the marking must be computed. In the final solution, the quality check of the marking will be implemented within an automated quality check that is located after the printing process. With this, the crimping machine can then discard wires of low quality and reproduce them instantly

Bartelt, Stefanie

Herberger, David

Hübner, Marco

Kuhlenkötter, Bernd

English

Institutionelles Repositorium der Leibniz Universität Hannover

CONFERENCE ON PRODUCTION SYSTEMS AND LOGISTICS CPSL 2022 __________________________________________________________________________________ DOI: https://doi.org/10.15488/12175 3rd Conference on Production Systems and Logistics Investigation Of Wire Mark Reading Methods To Support Automatic Quality Control Stefanie Bartelt1, Bernd Kuhlenkötter1 1Ruhr University Bochum, Chair of Production Systems, Bochum, Germany Abstract During the assembly of a control cabinet, a worker obstructs many individual configured wires. To distinguish these wires, a printer plots an identifying text on each end of the wires. However, due to the shape of the wires and the printing process, the quality of these markings is often too low, and it is hard or impossible to read the marking. Common reasons are a low contrast or a blurred text. By now, there is no quality check of the marking after a crimping machine produced the wire. This paper investigates methods for wire mark reading that is required to estimate the quality of the marking. By using optical character recognition, the likeliness that a worker can read the marking must be computed. In the final solution, the quality check of the marking will be implemented within an automated quality check that is located after the printing process. With this, the crimping machine can then discard wires of low quality and reproduce them instantly. Keywords Quality control; preconfigured wires; wire marking; machine learning; optical character recognition. 1. IntroductionControl cabinets are quite common in different domains. Although the specific cabinets can differ largely, their general setup is very similar. Each control cabinet contains a mounting plate in the back. On this plate, wire ducts and top hat rails are fixed. The top hat rails simplify the assembly of further components that are required by the customer. Nevertheless, components may also be fixed directly to the mounting plate. To connect the deployed components, wires with a different color and a different cross section are utilized. The cross section is selected according to the maximum current and the color is often defined by the function of the connection, e.g., signal, power line, or ground. [1] Figure 1 depicts an image of a fully assembled control cabinet. The top hat rails are filled with components DQGFDQQRWEHVHHQ7KHZLUHVDUHJXLGHGIURPWKHFRPSRQHQWV¶FRQQHFWLRQVLn a preferably short way into a nearby wire duct. Considering an average control cabinet, the cabinet embeds about a hundred or more wires. As the figure indicates, the wire ducts often contain a larger number of different wires. Hence, the identification of a certain wire can be quite challenging. To simplify this identification, markings can be printed on the wires. A common practice is to use the source as well as the target. Each component has a reference designator that is unique within a project. Standards, e.g., the EN 81346, specify rules for the naming and are commonly used in industry. However, obsoletespecifications like the DIN 40719 are still used for naming. For instance, a terminal block may be identifiedE\WKHVWULQJ³ /$-X10:2-´ Thereby, the first part starting with the equal sign indicates the facility,217and the second part starting with a plus sign indicates the location. Both do not change for a control cabinet DQGFDQEHRPLWWHGZLWKLQDPDUNLQJ+HQFH³-X10:2-´ZRXOGEHDPHDningful marking for one end of a wire and identifies a certain component in the setup/HWXVDVVXPHWKDW³-X1063(´ is another marking for the other end of the wire. As soon as a worker reads the end marking, he will know where to connect the wire to. However, the length of the wire might be up to a few meters and to identify a wire at an arbitrary position, intermediate markings are added to the wire. Angle brackets gives a hint to the direction. In the example DERYH³-X10:2 < > -;63(´LVVXFKDQLQWHrmediate marking. Following the wire to the left, one will end XSDWWKHHQGZLWKWKHPDUNLQJ³-;´, and so on. Of course, other rules are possible, but creating markings according to these rules is simple and effective.  Figure 1: Partial image of a fully assembled control cabinet. Only the covers of the wire ducts are missing. 2. Problem statement During assembly as well as for maintenance, it is important to identify a certain wire. For this, a sufficient quality of the above-mentioned marking is important. However, an inkjet printer applies the marking to the wire. This process is very susceptible to errors, and a proper quality check is missing. The most common errors are a blurred or faded print. A suitable solution for a quality check would be the utilization of optical character recognition (OCR). 7KH2&5DOJRULWKPFDQLGHQWLI\WKHPDUNLQJ¶VWH[WZKLFKLVWKHQFRPSDUHGto the known one that is applied by the printer. To illustrate the problem, figure 2 depicts some wires with different diameter and color. As shown, also the color of the ink may be different. Due to this, the contrast of the printing and the color of the wire can be very low. Although such an inspection can be done in a defined environment, some aspects must be considered for the quality check: x The wires are most likely not in a straight line, but they are usually slightly bent. As mentioned above, there is a defined environment for the inspection. This can prevent greater bending, but some remains. With respect to the character recognition, this will be a minor issue. x The printer applies the marking only from one direction. Hence, due to the shape of the wires, the marking may not be fully visible when using only one camera for inspection. Due to a twist of the wire, it might even happen that some characters are fully visible, and some are not. 218x The font of the printer is a dot-matrix font. While common serif or sans-serif fonts and even handwritten text is state-of-the-art, dot-matrix fonts are very hard to identify. The main reason for this failure seems to be the disconnected points that prevent common methods to work properly.  Figure 2: Image of wires with different size and colors. 3. Related work Optical character recognition has been subject of research for many decades. The general approach is to acquire an image, detect text lines and positions of single characters, and identify the characters found [2]. Finally, the identified characters are grouped to words. In addition, the image may be pre-processed, and the identified text can be post-processed to reduce errors. However, for the above-described problem, only the steps up to the building of words is relevant, because to decide whether the quality is sufficient or not, this identified text must be compared with a known one. Furthermore, the building of words is very simple since there is only one text line and white spaces can be neglected. In recent years, the utilization of machine learning for OCR applications has increased. Although the main steps are the same as described above. Thereby, different models for text detection can be selected, i.e., an object detection model or a text instance segmentation model. Eskenazi et al. gives a review on several segmentation algorithms [3]. A succeeding transcription model yields then to the final text. Furthermore, a character instance segmentation model can be used for both the text detection and text transcription [4]. There are two major applications for OCR. The first one is the text recognition of printed documents, for instance, books or invoices. Such documents contain a huge number of characters and have a good structure, i.e., lines and columns of text. The second application is text recognition in real life. Thereby, it is most likely that texts are rotated or distorted, and a main issue is to find the positions of the characters within an image [5]. In [6], Zheng et al. present a smart assistance system based on OCR. The system combines augmented reality methods and visual inspection methods. With this, the system can identify certain wires and present corresponding information to the user. Although the system can detect text printed to the wires, the text font is a sans-serif font and off-the-shelve methods are able to recognize them. Dot-matrix characters are formed from single dots that are not connected. In addition, the appearance of a character may change significantly already if a single dot is missing. As a result, recognition of dot-matrix fonts is different from other fonts [7]. Approaches are either an enhanced pre-processing, like connecting 219the dots, or sophisticated training-based methods. An efficient solution to find dots within an image is the computation of the cross correlation of the image with a given image of a single dot. For this, fast algorithms exist, e.g. [8]. In [9], the authors proposes a combination of pre-processing to identify regions of interest and a convolutional neural network for character recognition. Nevertheless, there is no general solution for recognition of dot-matrix fonts. Szajna et al. are developing a system to read wire markings by means of artificial intelligence, i.e., a deep neural network [10]. The presented solution takes a picture from a wire inserted into the system. With this, advanced methods recognize the wire marking. Thereby, the focus is on identifying any character including faulty ones. Although the system may be adapted to a quality control, the project does not analyze different fonts used for the marking, since it is assumed that the certain specification for the labeling varies from company to company. Even though the examined wires were marked with a dot-matrix font, the individual dots are sufficiently large and connected, which makes recognition much easier. 4. Wire mark reading There are two main steps to read a wire marking. The first step is the image acquisition with an optional pre-processing of the image. The second step is the character recognition. From the related work, a setup with a fixture for the wire, illumination, and a camera that takes images with a medium resolution is proven to yield good results. To find an appropriate solution, different variants of pre-processing as well as various character recognition solutions were compared. 4.1 Acquisition setup The first step in identifying the marking was to build a well-defined environment. For this, a camera module and illumination are located above a frosted glass plate. Although the frosted glass creates a slight reflection of the camera module, it eliminates almost completely shadows of the wire. Opaque plates are mounted to the sides to eliminate effects from outside. The wires can be installed through holes on two opposite sides. The OV2640 camera module is controlled by an ESP32 microcontroller. Next to the camera module, a white color LED is located that is used for illumination. Via a serial connection, a dedicated application can acquire an image with a resolution of 1024x768 pixel. Figure 3 shows the development state of the hardware and software prototype.    Figure 3: Photo of the hardware (left) and software (right) prototype to identify the markings. 2204.2 Pre-processing With the above-described setup, 65 images of different wires were taken and processed. In a first test, it turns out that gray scaling, blurring, and cross correlation increases the recognition. Figure 4 gives two examples of an original image as well as three pre-processed variants. The first variant applies a scatter filter before a Gaussian blur. The next one applies the same filter and adds a gray scaling. The last variant is a cross correlation with a black dot on a white background. Other variants were also tested, but they do not yield better results. #1     #2      (a) (b) (c) (d) Figure 4: Two examples of an original image (a), blurred image (b), blurred grey-scaled image (c), and cross-correlation image (d). As described in the next section, this pre-processing improves the character recognition. However, the effect is limited. Furthermore, recognition for wires with a low contrast, for instance a blue wire with black font such as example #2 in figure 4 shows, was not possible in any case.  ª  ª  ª  (a)  ª  ª  ª  (b) Figure 5: Three applied steps of pre-processing consisting of straitening, color removal, and normalization. 221To improve the contrast, the three-step pre-processing was selected, as figure 5 depicts. The first step is to straighten the wire. For this, the contour of the wire is detected by applying a Canny edge detection. With this, a perspective transformation computes a straight image of the wire. The second step estimates the wire color and removes it by using formula (1). ௬തതതതതതത is the normalized mean value of line y and may has a value from zero to one. ௫ǡ௬ is the gray value of the pixel at line y and column x. The other colors are equivalent. The last step is a linear normalization to increase the contrast. ݕǡݔݕܽݎܩ ൌ ൫ͳ െ തതതതതത൯ݕ ή ݕǡݔ ൅ ൫ͳ െ തതതതതതതത൯ݕ ή ݕǡݔ ൅ ൫ͳ െ തതതതതതത൯ݕ ή  (1) ݕǡݔ4.3 Character recognition The pre-processed images as well as the original image were tested with several OCR engines. Table 1 states the result of the recognized text for the given example. Further engines were also tested, e.g., Tesseract, IronOcr, and Aspose OCR. Except of Matrox SureDotOCR, these engines do not compute viable results for any of the image variants. As shown, only one engine can recognize text in the original image. The detected WH[WFRUUHVSRQGVWRWKH³;´WKDWZDVUHFRJQL]HGDVDURWDWHG³6;´$OVRWKHEOXUUHGLPDJHKDVDEDGperformance. The grey scaled variant has a quite good recognition compared to the colored one, and even better than the cross-correlation image. ,QWHUHVWLQJO\WKHFKDUDFWHUV³´DQG³!´ZHUHQRWGHWHFWHGE\DQ\engine. Nevertheless, all tested OCR engines have failure rates that are far from acceptable. An exception is SureDotOCR. This engine is specialized for reading dot matrix fonts and can detect the marking properly. For this, the used 5x5 dot font is defined within the engine. A major drawback is the requirement to specify the expected number of characters. As soon as this number does not match to the image, the results are rather bad. For example, when trying to read 11 characters in the example given in figure 5 a, the correct string of ³-X5:10 < > -X8´ZDVUHDG:KHQWU\LQJWRUHDGFKDUDFWHUVWKHHQJLQHVJLYHV³--:P-P-:--:-´.  Table 1: Character recognition results of different engines. image Google vision OCRSpace fintract Matrox SureDotOCR Figure 4 #1a SX no text detected no text detected -X5:10 < > -X8 Figure 4 #1b no text detected -X5 10 no text detected -X5:10 < > -X8 Figure 4 #1c - X5:10 -XB -X5:1O 8 - X5:10 -XB Xe -X5:10 < > -X8 Figure 4 #1d SS: Aus-XS -X5 1O - XE: 40 -XE -X5:10 < > -X8 Figure 4 #2a no text detected no text detected no text detected no text detected Figure 4 #2b no text detected no text detected no text detected no text detected Figure 4 #2c no text detected no text detected no text detected no text detected Figure 4 #2d no text detected no text detected no text detected no text detected Figure 5 a s no text detected 000 -X5:10 < > -X8 Figure 5 b ûðŲĄĿŔǿ no text detected no text detected -X1:4 < > - X9: 2225. Conclusion and future workWithin this paper, the problem of recognizing text on electrical wires was stated. An optical character recognition algorithm should identify the printed characters, and the resulting text can be compared with the known marking text. If both texts match, the quality of the printing is sufficient. While most commonly available engines cannot read the marking properly, one engine that is dedicated to dot-matrix fonts yield appropriate results. Although this library is suitable for a quality check of a known marking, a general detection is not possible due to limitations of the library in terms of flexibility of the number of characters. Hence, further developments are required, which will be done in future work. Additionally, further work will elaborate the challenge that all sides of the wire must be considered. By now, the wires are manually rotated to ensure the marking to be on top. Acknowledgements This Project is supported by the Federal Ministry for Economic Affairs and Climate Action (BMWK) on the basis of a decision by the German Bundestag (funding number: KK5055201MS0). References [1] Tempel, P., Eger, F., Lechler, A., Verl, A., 2017. Schaltschrankbau 4.0: Eine Studie über die Automatisierungs-und Digitalisierungspotentiale in der Fertigung von Schaltschränken und Schaltanlagen im klassischenMaschinen- und Anlagenbau. Universität Stuttgart. Institut für Steuerungstechnik der Werkzeugmaschinen undFertigungseinrichtungen., Stuttgart.[2] Islam, N., Islam, Z., Noor, N., 2017. A Survey on Optical Character Recognition System.http://arxiv.org/pdf/1710.05703v1.[3] Eskenazi, S., Gomez-Krämer, P., Ogier, J.-M., 2017. A comprehensive survey of mostly textual documentsegmentation algorithms since 2008. Pattern Recognition 64, 1±14.[4] Subramani, N., Matton, A., Greaves, M., Lam, A., 2020. A Survey of Deep Learning Approaches for OCR andDocument Understanding, 15 pp. http://arxiv.org/pdf/2011.13534v2.[5] Zhu, Y., Yao, C., Bai, X., 2016. Scene text detection and recognition: recent advances and future trends. Front.Comput. Sci. 10 (1), 19±36.[6] Zheng, L., Liu, X., An, Z., Li, S., Zhang, R., 2020. A smart assistance system for cable assembly by combiningwearable augmented reality with portable visual inspection. Virtual Reality & Intelligent Hardware 2 (1), 12±27.[7] Endo, K., Ohyama, W., Wakabayashi, T., Kimura, F., 2015. Performance Improvement of Dot-Matrix CharacterRecognition by Variation Model Based Learning, in: Jawahar, C.V., Shan, S. (Eds.), Computer Vision - ACCV2014 Workshops, vol. 9009. Springer International Publishing, Cham, pp. 147±156.[8] Briechle, K., Hanebeck, U.D., 2001. Template matching using fast normalized cross correlation, in: OpticalPattern Recognition XII. Aerospace/Defense Sensing, Simulation, and Controls, Orlando, FL. Monday 16 April2001. SPIE, pp. 95±102.[9] Muresan, M.P., Szabo, P.A., Nedevschi, S., 92019. Dot Matrix OCR for Bottle Validity Inspection, in: 2019 IEEE15th International Conference on Intelligent Computer Communication and Processing (ICCP). 2019 IEEE 15thInternational Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca,Romania. 05.09.2019 - 07.09.2019. IEEE, pp. 395±401.[10] 6]DMQD$.RVWU]HZVNL0&LHELHUD.6WU\MVNL5:RĨQLDN:$SSOLFDWLRQRIWKH'HHS&11-BasedMethod in Industrial System for Wire Marking Identification. Energies 14 (12), 3659.223Biography Stefanie Bartelt (*1991) is a research assistant at the Chair of Production Systems at Ruhr University Bochum. She conducts research in the field of industrial robotics and is currently working on a project for the automated handling and quality control of bendable components for the wiring of control cabinets. Bernd Kuhlenkötter (*1971) is Professor and head of the Chair of Production Systems (LPS) at Ruhr University Bochum. His research focuses on the planning, simulation and implementation of production systems and smart product-service systems. Prof. Kuhlenkötter is, among others, a member of the Scientific Society for Production Engineering (WGP) and the Scientific Society for Assembly, Handling and Industrial Robotics (MHI) as well as scientific director of the Research Center for the Engineering of Smart Product-Service Systems (ZESS). 224

Investigation of Wire Mark Reading Methods to Support Automatic Quality Control

https://www.repo.uni-hannover.de/bitstream/123456789/12273/1/Bartelt-CPSL2022.pdf

Investigation of Wire Mark Reading Methods to Support Automatic Quality Control

Abstract

Similar works

Full text

Available Versions

Institutionelles Repositorium der Leibniz Universität Hannover