Dynamic update of a virtual cell for programming and safe monitoring of an industrial robot

A hardware/software architecture for robot motion planning and on-line safe monitoring has been developed with the objective to assure high flexibility in production control, safety for workers and machinery, with user-friendly interface. The architecture, developed using Microsoft Robotics Developers Studio and implemented for a six-dof COMAU NS 12 robot, established a bidirectional communication between the robot controller and a virtual replica of the real robotic cell. The working space of the real robot can then be easily limited for safety reasons by inserting virtual objects (or sensors) in such a virtual environment. This paper investigates the possibility to achieve an automatic, dynamic update of the virtual cell by using a low cost depth sensor (i.e., a commercial Microsoft Kinect) to detect the presence of completely unknown objects, moving inside the real cell. The experimental tests show that the developed architecture is able to recognize variously shaped mobile objects inside the monitored area and let the robot stop before colliding with them, if the objects are not too small

Robotic path planning for non-destructive testing - a custom MATLAB toolbox approach

The requirement to increase inspection speeds for non-destructive testing (NDT) of composite aerospace parts is common to many manufacturers. The prevalence of complex curved surfaces in the industry provides motivation for the use of 6 axis robots in these inspections. The purpose of this paper is to present work undertaken for the development of a KUKA robot manipulator based automated NDT system. A new software solution is presented that enables flexible trajectory planning to be accomplished for the inspection of complex curved surfaces often encountered in engineering production. The techniques and issues associated with conventional manual inspection techniques and automated systems for the inspection of large complex surfaces were reviewed. This approach has directly influenced the development of a MATLAB toolbox targeted to NDT automation, capable of complex path planning, obstacle avoidance, and external synchronization between robots and associated external NDT systems. This paper highlights the advantages of this software over conventional off-line-programming approaches when applied to NDT measurements. An experimental validation of path trajectory generation, on a large and curved composite aerofoil component, is presented. Comparative metrology experiments were undertaken to evaluate the real path accuracy of the toolbox when inspecting a curved 0.5 m2 and a 1.6 m2 surface using a KUKA KR16 L6-2 robot. The results have shown that the deviation of the distance between the commanded TCPs and the feedback positions were within 2.7 mm. The variance of the standoff between the probe and the scanned surfaces was smaller than the variance obtainable via commercial path-planning software. Tool paths were generated directly on the triangular mesh imported from the CAD models of the inspected components without need for an approximating analytical surface. By implementing full external control of the robotic hardware, it has been possible to synchronise the NDT data collection with positions at all points along the path, and our approach allows for the future development of additional functionality that is specific to NDT inspection problems. For the current NDT application, the deviations from CAD design and the requirements for both coarse and fine inspections, dependent on measured NDT data, demand flexibility in path planning beyond what is currently available from existing off-line robot programming software

Teleoperation of industrial robot manipulators based on augmented reality

This research develops a novel teleoperation for robot manipulators based on augmented reality. The proposed interface is equipped with full capabilities in order to replace the classical teach pendant of the robot for carrying out teleoperation tasks. The proposed interface is based on an augmented reality headset for projecting computer-generated graphics onto the real environment and a gamepad to interact with the computer-generated graphics and provide robot commands. In order to demonstrate the benefits of the proposed method, several usability tests were conducted using a 6R industrial robot manipulator in order to compare the proposed interface and the conventional teach pendant interface for teleoperation tasks. In particular, the results of these usability tests show that the proposed approach is more intuitive, ergonomic, and easy to use. Furthermore, the comparison results also show that the proposed method clearly improves the velocity of the teleoperation task, regardless of the user's previous experience in robotics and augmented reality technology

Sensor data-based decision making

Increasing globalization and growing industrial system complexity has amplified the interest in the use of information provided by sensors as a means of improving overall manufacturing system performance and maintainability. However, utilization of sensors can only be effective if the real-time data can be integrated into the necessary business processes, such as production planning, scheduling and execution systems. This integration requires the development of intelligent decision making models that can effectively process the sensor data into information and suggest appropriate actions. To be able to improve the performance of a system, the health of the system also needs to be maintained. In many cases a single sensor type cannot provide sufficient information for complex decision making including diagnostics and prognostics of a system. Therefore, a combination of sensors should be used in an integrated manner in order to achieve desired performance levels. Sensor generated data need to be processed into information through the use of appropriate decision making models in order to improve overall performance. In this dissertation, which is presented as a collection of five journal papers, several reactive and proactive decision making models that utilize data from single and multi-sensor environments are developed. The first paper presents a testbed architecture for Auto-ID systems. An adaptive inventory management model which utilizes real-time RFID data is developed in the second paper. In the third paper, a complete hardware and inventory management solution, which involves the integration of RFID sensors into an extremely low temperature industrial freezer, is presented. The last two papers in the dissertation deal with diagnostic and prognostic decision making models in order to assure the healthy operation of a manufacturing system and its components. In the fourth paper a Mahalanobis-Taguchi System (MTS) based prognostics tool is developed and it is used to estimate the remaining useful life of rolling element bearings using data acquired from vibration sensors. In the final paper, an MTS based prognostics tool is developed for a centrifugal water pump, which fuses information from multiple types of sensors in order to take diagnostic and prognostics decisions for the pump and its components --Abstract, page iv

Automated freeform assembly of threaded fasteners

Over the past two decades, a major part of the manufacturing and assembly market has been driven by its customer requirements. Increasing customer demand for personalised products create the demand for smaller batch sizes, shorter production times, lower costs, and the flexibility to produce families of products - or different parts - with the same sets of equipment. Consequently, manufacturing companies have deployed various automation systems and production strategies to improve their resource efficiency and move towards right-first-time production. However, many of these automated systems, which are involved with robot-based, repeatable assembly automation, require component- specific fixtures for accurate positioning and extensive robot programming, to achieve flexibility in their production. Threaded fastening operations are widely used in assembly. In high-volume production, the fastening processes are commonly automated using jigs, fixtures, and semi-automated tools. This form of automation delivers reliable assembly results at the expense of flexibility and requires component variability to be adequately controlled. On the other hand, in low- volume, high- value manufacturing, fastening processes are typically carried out manually by skilled workers. This research is aimed at addressing the aforementioned issues by developing a freeform automated threaded fastener assembly system that uses 3D visual guidance. The proof-of-concept system developed focuses on picking up fasteners from clutter, identifying a hole feature in an imprecisely positioned target component and carry out torque-controlled fastening. This approach has achieved flexibility and adaptability without the use of dedicated fixtures and robot programming. This research also investigates and evaluates different 3D imaging technology to identify the suitable technology required for fastener assembly in a non-structured industrial environment. The proposed solution utilises the commercially available technologies to enhance the precision and speed of identification of components for assembly processes, thereby improving and validating the possibility of reliably implementing this solution for industrial applications. As a part of this research, a number of novel algorithms are developed to robustly identify assembly components located in a random environment by enhancing the existing methods and technologies within the domain of the fastening processes. A bolt identification algorithm was developed to identify bolts located in a random clutter by enhancing the existing surface-based matching algorithm. A novel hole feature identification algorithm was developed to detect threaded holes and identify its size and location in 3D. The developed bolt and feature identification algorithms are robust and has sub-millimetre accuracy required to perform successful fastener assembly in industrial conditions. In addition, the processing time required for these identification algorithms - to identify and localise bolts and hole features - is less than a second, thereby increasing the speed of fastener assembly

Task Oriented Programming and Service Algorithms for Smart Robotic Cells

L'abstract è presente nell'allegato / the abstract is in the attachmen

Automated NDT inspection for large and complex geometries of composite materials

Large components with complex geometries, made of composite materials, have become very common in modern structures. To cope with future demand projections, it is necessary to overcome the current non-destructive testing (NDT) bottlenecks encountered during the inspection phase of manufacture. This thesis investigates several aspects of the introduction of automation within the inspection process of complex parts. The use of six-axis robots for product inspection and non-destructive testing systems is the central investigation of this thesis. The challenges embraced by the research include the development of a novel controlling approach for robotic manipulators and of novel path-planning strategies. The integration of robot manipulators and NDT data acquisition instruments is optimized. An effective and reliable way to encode the NDT data through the interpolated robot feedback positions is implemented. The viability of the new external control method is evaluated experimentally. The observed maximum position and orientation errors are respectively within 2mm and within 1 degree, over an operating envelope of 3m³. A new software toolbox (RoboNDT), aimed at NDT technicians, has been developed during this work. RoboNDT is intended to transform the robot path-planning problem into an easy step of the inspection process. The software incorporates the novel path-planning algorithms developed during this research and is shaped to overcome practical limitations of current OLP software. The software has been experimentally validated using scans on real high value aerospace components. RoboNDT delivers tool-path errors that are lower than the errors given by commercial off-line path-planning software. For example the variability of the standoff is within 10 mm for the tool-paths created with the commercial software and within 4.5 mm for the RoboNDT tool-paths, over a scanned area of 1.6m². The output of this research was used to support a 3-year industrial project, called IntACom and led by TWI on behalf of major aerospace sponsors. The result is a demonstrator system, currently in use at TWI Technology Centre, which is capable of inspecting complex geometries with high throughput. The IntACom system can scan real components 2.8 times faster than traditional 3-DoF scanners deploying phased-array inspection and 6.7 times faster than commercial gantry systems deploying traditional single-element inspection.Large components with complex geometries, made of composite materials, have become very common in modern structures. To cope with future demand projections, it is necessary to overcome the current non-destructive testing (NDT) bottlenecks encountered during the inspection phase of manufacture. This thesis investigates several aspects of the introduction of automation within the inspection process of complex parts. The use of six-axis robots for product inspection and non-destructive testing systems is the central investigation of this thesis. The challenges embraced by the research include the development of a novel controlling approach for robotic manipulators and of novel path-planning strategies. The integration of robot manipulators and NDT data acquisition instruments is optimized. An effective and reliable way to encode the NDT data through the interpolated robot feedback positions is implemented. The viability of the new external control method is evaluated experimentally. The observed maximum position and orientation errors are respectively within 2mm and within 1 degree, over an operating envelope of 3m³. A new software toolbox (RoboNDT), aimed at NDT technicians, has been developed during this work. RoboNDT is intended to transform the robot path-planning problem into an easy step of the inspection process. The software incorporates the novel path-planning algorithms developed during this research and is shaped to overcome practical limitations of current OLP software. The software has been experimentally validated using scans on real high value aerospace components. RoboNDT delivers tool-path errors that are lower than the errors given by commercial off-line path-planning software. For example the variability of the standoff is within 10 mm for the tool-paths created with the commercial software and within 4.5 mm for the RoboNDT tool-paths, over a scanned area of 1.6m². The output of this research was used to support a 3-year industrial project, called IntACom and led by TWI on behalf of major aerospace sponsors. The result is a demonstrator system, currently in use at TWI Technology Centre, which is capable of inspecting complex geometries with high throughput. The IntACom system can scan real components 2.8 times faster than traditional 3-DoF scanners deploying phased-array inspection and 6.7 times faster than commercial gantry systems deploying traditional single-element inspection

Simulation-Based and Data-Driven Approaches to Industrial Digital Twinning Towards Autonomous Smart Manufacturing Systems

A manufacturing paradigm shift from conventional control pyramids to decentralized, service-oriented, and cyber-physical systems (CPSs) is taking place in today’s Industry 4.0 revolution. Generally accepted roles and implementation recipes of cyber systems are expected to be standardized in the future of manufacturing industry. Developing affordable and customizable cyber-physical production system (CPPS) and digital twin implementations infuses new vitality for current Industry 4.0 and Smart Manufacturing initiatives. Specially, Smart Manufacturing systems are currently looking for methods to connect factories to control processes in a more dynamic and open environment by filling the gaps between virtual and physical systems. The work presented in this dissertation first utilizes industrial digital transformation methods for the automation of robotic manufacturing systems, constructing a simulation-based surrogate system as a digital twin to visually represent manufacturing cells, accurately simulate robot behaviors, promptly predict system faults and adaptively control manipulated variables. Then, a CPS-enabled control architecture is presented that accommodates: intelligent information systems involving domain knowledge, empirical model, and simulation; fast and secured industrial communication networks; cognitive automation by rapid signal analytics and machine learning (ML) based feature extraction; and interoperability between machine and human. A successful semantic integration of process indicators is fundamental to future control autonomy. Hence, a product-centered signature mapping approach to automated digital twinning is further presented featuring a hybrid implementation of smart sensing, signature-based 3D shape feature extractor, and knowledge taxonomy. Furthermore, capabilities of members in the family of Deep Reinforcement Learning (DRL) are explored within the context of manufacturing operational control intelligence. Preliminary training results are presented in this work as a trial to incorporate DRL-based Artificial Intelligence (AI) to industrial control processes. The results of this dissertation demonstrate a digital thread of autonomous Smart Manufacturing lifecycle that enables complex signal processing, semantic integration, automatic derivation of manufacturing strategies, intelligent scheduling of operations and virtual verification at a system level. The successful integration of currently available industrial platforms not only provides facile environments for process verification and optimization, but also facilitates derived strategies to be readily deployable to physical shop floor. The dissertation finishes with summary, conclusions, and suggestions for further work

Utveckling av en virtuell robotcell för programmering och optimering

Cyberfysiska system, robotik och simulering är områden i stort fokus eftersom det har en stor del i industri 4.0. Syftet med projektet är att utveckla robotlaboratoriet på Högskolan i Gävle till att vara mer anpassat för införandet av CPS genom att skapa en virtuell robotcell och utveckla griparen hos robotarna. Den virtuella robotcellen kommer att öka säkerheten i laboratoriet och gör så fler personer kan arbeta med robotarna samtidigt. Robotarna och programvarorna som användes i projektet är utvecklade av företaget KUKA. Det användes tre programvaror till den virtuella robotcellen i projektet. De två som användes mest utförde simulering och programmering. Den sista programvaran hade möjligheten att sammankoppla och styra de andra programvarorna. Den virtuella robotcellen utvecklades i simuleringsprogramvaran och där skapades även program som utfördes av de fysiska robotarna i laboratoriet. De fysiska robotarna rörde sig inte på samma sätt som i den virtuella modellen. En felsökning utfördes och problemet fanns hos de fysiska robotarna. KUKA kommer därför utföra ett underhållsarbete på robotarna. Valideringen av den virtuella robotcellen utfördes därför i simuleringsprogrammet. När den fysiska robotcellen fungerar korrekt så kommer den också att kunna användas för att validera den virtuella robotcellen. En programvara som gav griparen fler funktionaliteter installerades på robotarna. Programvaran som användes var utvecklad av KUKA och fanns i laboratoriet. Funktionerna gör griparen lättare att använda vid manuell körning och programmering. Programvaran som utför sammankopplingen av simulering- och programmeringprogramvaran behöver undersökas vidare för att den virtuella robotcellen ska kunna utföra fler funktioner. Den virtuella robotcellen finns tillgänglig för alla på högskolan och kan användas för att få en förståelse för hur robotarna fungerar. Program kan optimeras i den virtuella miljön och utföras av de fysiska robotarna.Cyber physical systems, robotics and simulation are areas of major focus as they play a big part in industry 4.0. The aim of the project is to develop the robotic laboratory at the University of Gävle to be more adapted to the introduction of CPS by creating a virtual robotic cell and developing the gripper of the robots. The virtual robotic cell are going to increase the safety in the laboratory and make it possible for more people to work with the robots at the same time. The robots and software used in the project were developed by the company KUKA. Three different software’s were used for the virtual robotic cell in the project. The two most commonly used were simulation and programming. The third software had the possibility to interconnect and control the other two software’s. The virtual robotic cell was developed in the simulation software and also programs were created that the physical robots in the laboratory performed. The physical robots did not move in the same way as the virtual model. Troubleshooting was performed and it showed that the problem was with the physical robots. Therefore KUKA will accomplish maintenance work on the robots. The validation of the virtual robotic cell were for that reason performed in the simulation program. When the physical robotic cell is operating correctly it will also be able to validate that the virtual robotic cell is working. A software that gave the gripper more functionality was installed on the robots. The software that was used was developed by KUKA and were available in the laboratory. The functions make the gripper easier to manage in manual driving and programming. The software that performs the interconnection of simulation and programming software is in need of further analyzation in order for the virtual robotic cell to be able to execute more functions. The virtual robotic cell is available for everyone at the university and can be used to understand how the robots work. Programs can be optimized in the virtual environment and performed by the physical robots