Berndt, Dirk

Sauer, Steffen

Schnee, Jost

Teutsch, Christian

Digitale Bibliothek Thüringen

urn:nbn:de:gbv:ilm1-2011imeko-046:8 Joint International IMEKO TC1+ TC7+ TC13 Symposium August 31st− September 2nd, 2011, Jena, Germany urn:nbn:de:gbv:ilm1-2011imeko:2    WORKER ASSISTANCE AND QUALITY INSPECTION –  APPLICATION OF OPTICAL 3D METROLOGY AND AUGMENTED REALITY TECHNOLOGIES  Steffen Sauer, Dirk Berndt, Jost Schnee, Christian Teutsch Fraunhofer Institut für Fabrikbetrieb und –automatisierung IFF, Magdeburg, Germany   Abstract − Numerous industrial mount processes are characterized by a high variance due to the individuality of the final products. Manual assembly work subjectively influences the quality of an end product and errors thereby cannot be excluded. Errors that occur in production increase costs as it is likely to happen in the assembly of clamping which is used in CNC machining centers. As they are usu-ally assembled manually, errors might occur.  Keywords: augmented reality, mount assistance, quality inspection   1.  INTRODUCTION  Systematic assistance is an option to support hu-man action and prevent errors. We present an assis-tance based on an integrated Augmented Reality-based approach. Augmented Reality (AR) is the en-hancement of the real world with additional virtual information. A comprehensive introduction is given in [1]. The field of industrial application is much diversi-fied. It reaches from maintenance tasks [2] over con-struction tasks [3] to mount instruction systems. The presented AR-based approach helps a worker to avoid errors by mounting wrong parts. By providing visual information on a component or assembly being as-sembled, clear instructions are furnished, which make processes highly reliable even when variants and types of assembly keep changing. Such systems can be extended to supplement the assistance function with automatic quality inspection after every step of as-sembly. This further enhances process reliability.    2. FUNCTIONAL PRINCIPLE OF ASSISTANCE  Data from a video camera monitoring the work-place and 3-D CAD models of design data on an as-sembled assembly including every single component are available to the system as input information. For stationary structures, the camera’s spatial position and orientation (perspective) relative to the assembly field is initially determined one time with the aid of an external reference system located in the camera’s field of vision and a fixed reference to the assembled as-sembly or its holder. Assistance or inspection of an assembly for one assembly step may be organized by employing data on camera position and orientation to align the visual perspective of the 3-D CAD model identically to the camera perspective and generate a virtual image of the real camera view. Afterward, the CAD models of the components being assembled may be overlaid in the correct position and orientation of the camera image's view of the real assembly scene. This "virtually" adds the view of the real camera im-age to the current work step. This information on the position and orientation of the component being as-sembled systematically supports workers. A simulta-neously displayed text version of the assembly opera-tion presents the type of component and the sequence of assembly steps. Additional information such as bolting torque and the like is displayed for the current work step and special instruction is integrated in the visualization. Alternatively to this fixed camera arrangement that monitors the assembly scene, another camera that is fixed or even has defined movement may be used. This allows different perspectives of the assembly scene and has many benefits particularly when assemblies are large and complex. In addition to overlaid and textual assistance information, an interac-tive 3D CAD model viewer provides workers further support.    3. FUNCTIONAL PRINCIPLE OF ASSEMBLY INSPECTION  Accompanying inspection of the result of assem-bly further enhances process reliability. Once an as-sembly step has been completed, the presence of the mounted component and the correctness of its position and orientation as well as the completeness of the step are inspected by scanning the 3D geometry. The assis-tance setup is extended by another camera to generate a second view of the assembly scene. The mounted component’s object geometry is determined three-dimensionally in real time with the aid of stereo trian-gulation. The resultant set of points is compared with the CAD data to generate quality data on a compo-nent’s presence, correctness and correct installation position. The result is visualized immediately upon the urn:nbn:de:gbv:ilm1-2011imeko-046:8 Joint International IMEKO TC1+ TC7+ TC13 Symposium August 31st− September 2nd, 2011, Jena, Germany urn:nbn:de:gbv:ilm1-2011imeko:2  conclusion of an assembly step and enables workers to directly inspect their own work themselves. The developed AR-based system for modern CNC systems displays timely and precisely positioned mounting instructions on the operators’ workplace. Here, one or more cameras monitor the working place, and the camera images are shown on a display in front of the worker. An overlay, generated live from the CAD-model of the parts, is then rendered on the cam-era images. These overlaid images give precise visual instructions for the worker by showing exactly what to mount (see Fig. 1). If the worker follows all AR in-structions, mistakes will not occur.    Fig. 1. Overlay visualization of assistance renderings. Some parts have already been mounted. Note that there are small but clearly visible differences between assistance and reality on top of the parts.    4.  ASSEMBLY OPERATION GENERATIONS  The sequence of an assembly operation’s steps may be defined by an assembly operation editor or already in the assembly's design in the CAD program. The assembly operation editor computes and proposes a possible assembly sequence based on the CAD data of the component to be mounted. The user may choose between alternative proposals or define a sequence manually. The assistance and inspection system can be adapted to changing assembly scenarios flexibly and quickly by selecting an assembly sequence dataset.    5.  VISUALIZATION AND OPERATION  In the simplest case, the assistance functions and the inspection results can be visualized by a monitor set up in the worker’s field of vision. An alternative is expedient for select applications. A projector projects assistance information directly onto the component and thus provides workers intuitive support. Workers may enter operating instructions by touch screens or by manual or pedal buttons. Alternatively, hand ges-tures may be employed to instruct the system.  6.  SYSTEM SETUP FOR CNC MACHINES  The developed assistance setup consists of one or more cameras that are statically directed towards the workers mounting place. For CNC machines that process setups with base plates of approximate square dimensions, a turntable is installed (see Fig. 2(a)). A turntable helps workers to access the base plate from arbitrary direction. On the table there are fixed sockets that allow base plates to be fixed in an exact prede-fined position reproducibly. One or more touch screens are installed directly in front of the worker. They show overlaid images from each camera view. A typical setup has a camera that is aligned vertical to the working place and that way covers the x-y-plane. A second camera aligned nearly horizontal presents the y-z plane and primarily displays height informa-tion. By rotating the base plate on the turntable it is ensured that the second camera can see all faces of the assembly. For elongated base plates typically more cameras but no turntable are used (Fig. 2(b)). The process of mounting a complete assembly is split in several steps. These steps again consist of several parts that have to be mounted in an arbitrary sequence. For each step and each camera view a special render-ing of the parts is generated and overlaid on the cam-era images. The rendered parts are created by using the original clamping tool CAD files. Then a virtual camera is placed at exact the same location as in real-ity and exactly the same camera parameters are ap-plied. A perfect rendering thus has the same appear-ance as the original clampings or, in turn, the worker has to place the clampings in reality equal to the vir-tual visualization. To sum up, we have the CAD data of a complete assembly, a description that defines the assembly process sequences, and an accurate camera calibration, which provides the required camera parameters. The output is an augmented camera image showing the parts which have to be mounted in each step. If the workplace is equipped with a turntable, a simple ro-tary encoder provides angle information which is applied on the CAD model transformation.    7.  CONCLUSIONS  Augmented Reality assistance systems help effi-ciently organize complex and varied assembly proc-esses with a high level of objective product quality. They speed up operations significantly while enhanc-ing process reliability at the same time. In-process integration of assistance and quality inspection gener-ates direct feedback, thus eliminating time-consuming and costly reworking.   urn:nbn:de:gbv:ilm1-2011imeko-046:8 Joint International IMEKO TC1+ TC7+ TC13 Symposium August 31st− September 2nd, 2011, Jena, Germany urn:nbn:de:gbv:ilm1-2011imeko:2  (a)  (b)   Fig. 2. Examples of worker assistance setups: (a) A typical setup containing a turntable for square sized base plates (both cameras outside the image) and (b) a setup for elongated base plates. Three of five cameras are mounted on top of the work-place. Touch screens for visualization and interaction are mounted facing towards the worker.    REFERENCES  [1]   R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, B. MacIntyre, "Recent advances in augmented reality”, IEEE Computer Graphics and Applications, 21(6), pp. 34-47, 2001. [2]  T. Haritos, N. Macchiarella, "A mobile application of augmented reality for aerospace maintenance training”, Digital Avionics Systems Conferences, 2005. [3]   D. Reiners, D. Stricker, G. Klinker, S. Müller, "Augmented reality for construction tasks: doorlock assembly”, IWAR '98: Proceedings of the international workshop on Augmented reality. A. K. Peters, Ltd., 1999.                           Author(s): Steffen Sauer, Dirk Berndt, Jost Schnee, Christi-an Teutsch, Fraunhofer Institut für Fabrikbetrieb und –auto-matisierung IFF, Sandtorstr. 22, 39106 Magdeburg, Ger-many, phone +49 391 4090 265, fax +49 391 4090 93 265, steffen.sauer@iff.fraunhofer.de. 

Worker assistance and quality inspection – application of optical 3D metrology and augmented reality technologies

https://www.db-thueringen.de/servlets/MCRFileNodeServlet/dbt_derivate_00024435/ilm1-2011imeko-046.pdf

Worker assistance and quality inspection – application of optical 3D metrology and augmented reality technologies

Abstract

Similar works

Full text

Available Versions

Digitale Bibliothek Thüringen