Event-driven industrial robot control architecture for the Adept V+ platform

Modern industrial robotic systems are highly interconnected. They operate in a distributed environment and communicate with sensors, computer vision systems, mechatronic devices, and computational components. On the fundamental level, communication and coordination between all parties in such distributed system are characterized by discrete event behavior. The latter is largely attributed to the specifics of communication over the network, which, in terms, facilitates asynchronous programming and explicit event handling. In addition, on the conceptual level, events are an important building block for realizing reactivity and coordination. Eventdriven architecture has manifested its effectiveness for building loosely-coupled systems based on publish-subscribe middleware, either general-purpose or robotic-oriented. Despite all the advances in middleware, industrial robots remain difficult to program in context of distributed systems, to a large extent due to the limitation of the native robot platforms. This paper proposes an architecture for flexible event-based control of industrial robots based on the Adept V+ platform. The architecture is based on the robot controller providing a TCP/IP server and a collection of robot skills, and a high-level control module deployed to a dedicated computing device. The control module possesses bidirectional communication with the robot controller and publish/subscribe messaging with external systems. It is programmed in asynchronous style using pyadept, a Python library based on Python coroutines, AsyncIO event loop and ZeroMQ middleware. The proposed solution facilitates integration of Adept robots into distributed environments and building more flexible robotic solutions with eventbased logic

Flexible Composition of Robot Logic with Computer Vision Services

Vision-based robotics is an ever-growing field within industrial automation. Demands for greater flexibility and higher quality motivate manufacturing companies to adopt these technologies for such tasks as material handling, assembly, and inspection. In addition to the direct use in the manufacturing setting, robots combined with vision systems serve as highly flexible means for realization of prototyping test-beds in the R&D context.Traditionally, the problem areas of robotics and computer vision are attacked separately. An exception is the study of vision-based servo control, the focus of which constitutes control-theoretic aspects of vision-based robot guidance under assumption that robot joints can be controlled directly. The missing part is a systemic approach to implementing robotic application with vision sensing given industrial robots constrained by their programming interface. This thesis targets the development process of vision-based robotic systems in an event-driven environment. It focuses on design and composition of three functional components: (1) robot control function, (2) image acquisition function, and (3) image processing function. The thesis approaches its goal by a combination of laboratory results, a case study of an industrial company (Kongsberg Automotive AS), and formalization of computational abstractions and architectural solutions. The image processing function is tackled with the application of reactive pipelines. The proposed system development method allows for smooth transition from early-stage vision algorithm prototyping to the integration phase. The image acquisition function in this thesis is exposed in a service-oriented manner with the help of a flexible set of concurrent computational primitives. To realize control of industrial robots, a distributed architecture is devised, which supports composability of communication-heavy robot logic, as well as flexible coupling of the robot control node with vision services

Search Methods for Mobile Manipulator Performance Measurement

Mobile manipulators are a potential solution to the increasing need for additional flexibility and mobility in industrial robotics applications. However, they tend to lack the accuracy and precision achieved by fixed manipulators, especially in scenarios where both the manipulator and the autonomous vehicle move simultaneously. This thesis analyzes the problem of dynamically evaluating the positioning error of mobile manipulators. In particular, it investigates the use of Bayesian methods to predict the position of the end-effector in the presence of uncertainty propagated from the mobile platform. Simulations and real-world experiments are carried out to test the proposed method against a deterministic approach. These experiments are carried out on two mobile manipulators - a proof-of-concept research platform and an industrial mobile manipulator - using ROS and Gazebo. The precision of the mobile manipulator is evaluated through its ability to intercept retroreflective markers using a photoelectric sensor attached to the end-effector. Compared to the deterministic search approach, we observed improved interception capability with comparable search times, thereby enabling the effective performance measurement of the mobile manipulator

An Overview of Industrial Robots Control and Programming Approaches

Nowadays, manufacturing plants are required to be flexible to respond quickly to customer demands, adapting production and processes without affecting their efficiency. In this context, Industrial Robots (IRs) are a primary resource for modern factories due to their versatility which allows the execution of flexible, reconfigurable, and zero-defect manufacturing tasks. Even so, the control and programming of the commercially available IRs are limiting factors for their effective implementation, especially for dynamic production environments or when complex applications are required. These issues have stimulated the development of new technologies that support more efficient methods for robot control and programming. The goal of this research is to identify and evaluate the main approaches proposed in scientific papers and by the robotics industry in the last decades. After a critical review of the standard IR control schematic, the paper discusses the available control alternatives and summarizes their characteristics, range of applications, and remaining limitations

Development of PVDF tactile dynamic sensing in a behaviour-based assembly robot

The research presented in this thesis focuses on the development of tactile event sig¬ nature sensors and their application, especially in reactive behaviour-based robotic assembly systems.In pursuit of practical and economic sensors for detecting part contact, the application ofPVDF (polyvinylidene fluoride) film, a mechanical vibration sensitive piezo material, is investigated. A Clunk Sensor is developed which remotely detects impact vibrations, and a Push Sensor is developed which senses small changes in the deformation of a compliant finger surface. The Push Sensor is further developed to provide some force direction and force pattern sensing capability.By being able to detect changes of state in an assembly, such as a change of contact force, an assembly robot can be well informed of current conditions. The complex structure of assembly tasks provides a rich context within which to interpret changes of state, so simple binary sensors can conveniently supply a lot more information than in the domain of mobile robots. Guarded motions, for example, which require sensing a change of state, have long been recognised as very useful in part mating tasks. Guarded motions are particularly well suited to be components of assembly behavioural modules.In behaviour-based robotic assembly systems, the high level planner is endowed with as little complexity as possible while the low level planning execution agent deals with actual sensing and action. Highly reactive execution agents can provide advantages by encapsulating low level sensing and action, hiding the details of sensori-motor complexity from the higher levels.Because behaviour-based assembly systems emphasise the utility of this kind of quali¬ tative state-change sensor (as opposed to sensors which measure physical quantities), the robustness and utility of the Push Sensor was tested in an experimental behaviourbased system. An experimental task of pushing a ring along a convoluted stiff wire is chosen, in which the tactile sensors developed here are aided by vision. Three differ¬ ent methods of combining these different sensors within the general behaviour-based paradigm are implemented and compared. This exercise confirms the robustness and utility of the PVDF-based tactile sensors. We argue that the comparison suggests that for behaviour-based assembly systems using multiple concurrent sensor systems, bottom-level motor control in terms of force or velocity would be more appropriate than positional control. Behaviour-based systems have traditionally tried to avoid symbolic knowledge. Considering this in the light of the above work, it was found useful to develop a taxonomy of type of knowledge and refine the prohibition

Advancement in robot programming with specific reference to graphical methods

This research study is concerned with the derivation of advanced robot programming methods. The methods include the use of proprietary simulation modelling and design software tools for the off-line programming of industrial robots. The study has involved the generation of integration software to facilitate the co-operative operation of these software tools. The three major researcli'themes7of "ease of usage", calibration and the integration of product design data have been followed to advance robot programming. The "ease of usage" is concerned with enhancements in the man-machine interface for robo t simulation systems in terms of computer assisted solid modelling and computer assisted task generation. Robot simulation models represent an idealised situation, and any off-line robot programs generated from'them may contain'discrepancies which could seriously effect thq programs' performance; Calibration techniques have therefore been investigated as 'a method of overcoming discrepancies between the simulation model and the real world. At the present time, most computer aided design systems operate as isolated islands of computer technology, whereas their product databases should be used to support decision making processes and ultimately facilitate the generation of machine programs. Thus the integration of product design data has been studied as an important step towards truly computer integrated manufacturing. The functionality of the three areas of study have been generalised and form the basis for recommended enhancements to future robot programming systems

Self-managed Workflows for Cyber-physical Systems

Workflows are a well-established concept for describing business logics and processes in web-based applications and enterprise application integration scenarios on an abstract implementation-agnostic level. Applying Business Process Management (BPM) technologies to increase autonomy and automate sequences of activities in Cyber-physical Systems (CPS) promises various advantages including a higher flexibility and simplified programming, a more efficient resource usage, and an easier integration and orchestration of CPS devices. However, traditional BPM notations and engines have not been designed to be used in the context of CPS, which raises new research questions occurring with the close coupling of the virtual and physical worlds. Among these challenges are the interaction with complex compounds of heterogeneous sensors, actuators, things and humans; the detection and handling of errors in the physical world; and the synchronization of the cyber-physical process execution models. Novel factors related to the interaction with the physical world including real world obstacles, inconsistencies and inaccuracies may jeopardize the successful execution of workflows in CPS and may lead to unanticipated situations. This thesis investigates properties and requirements of CPS relevant for the introduction of BPM technologies into cyber-physical domains. We discuss existing BPM systems and related work regarding the integration of sensors and actuators into workflows, the development of a Workflow Management System (WfMS) for CPS, and the synchronization of the virtual and physical process execution as part of self-* capabilities for WfMSes. Based on the identified research gap, we present concepts and prototypes regarding the development of a CPS WFMS w.r.t. all phases of the BPM lifecycle. First, we introduce a CPS workflow notation that supports the modelling of the interaction of complex sensors, actuators, humans, dynamic services and WfMSes on the business process level. In addition, the effects of the workflow execution can be specified in the form of goals defining success and error criteria for the execution of individual process steps. Along with that, we introduce the notion of Cyber-physical Consistency. Following, we present a system architecture for a corresponding WfMS (PROtEUS) to execute the modelled processes-also in distributed execution settings and with a focus on interactive process management. Subsequently, the integration of a cyber-physical feedback loop to increase resilience of the process execution at runtime is discussed. Within this MAPE-K loop, sensor and context data are related to the effects of the process execution, deviations from expected behaviour are detected, and compensations are planned and executed. The execution of this feedback loop can be scaled depending on the required level of precision and consistency. Our implementation of the MAPE-K loop proves to be a general framework for adding self-* capabilities to WfMSes. The evaluation of our concepts within a smart home case study shows expected behaviour, reasonable execution times, reduced error rates and high coverage of the identified requirements, which makes our CPS~WfMS a suitable system for introducing workflows on top of systems, devices, things and applications of CPS.:1. Introduction 15 1.1. Motivation 15 1.2. Research Issues 17 1.3. Scope & Contributions 19 1.4. Structure of the Thesis 20 2. Workflows and Cyber-physical Systems 21 2.1. Introduction 21 2.2. Two Motivating Examples 21 2.3. Business Process Management and Workflow Technologies 23 2.4. Cyber-physical Systems 31 2.5. Workflows in CPS 38 2.6. Requirements 42 3. Related Work 45 3.1. Introduction 45 3.2. Existing BPM Systems in Industry and Academia 45 3.3. Modelling of CPS Workflows 49 3.4. CPS Workflow Systems 53 3.5. Cyber-physical Synchronization 58 3.6. Self-* for BPM Systems 63 3.7. Retrofitting Frameworks for WfMSes 69 3.8. Conclusion & Deficits 71 4. Modelling of Cyber-physical Workflows with Consistency Style Sheets 75 4.1. Introduction 75 4.2. Workflow Metamodel 76 4.3. Knowledge Base 87 4.4. Dynamic Services 92 4.5. CPS-related Workflow Effects 94 4.6. Cyber-physical Consistency 100 4.7. Consistency Style Sheets 105 4.8. Tools for Modelling of CPS Workflows 106 4.9. Compatibility with Existing Business Process Notations 111 5. Architecture of a WfMS for Distributed CPS Workflows 115 5.1. Introduction 115 5.2. PROtEUS Process Execution System 116 5.3. Internet of Things Middleware 124 5.4. Dynamic Service Selection via Semantic Access Layer 125 5.5. Process Distribution 126 5.6. Ubiquitous Human Interaction 130 5.7. Towards a CPS WfMS Reference Architecture for Other Domains 137 6. Scalable Execution of Self-managed CPS Workflows 141 6.1. Introduction 141 6.2. MAPE-K Control Loops for Autonomous Workflows 141 6.3. Feedback Loop for Cyber-physical Consistency 148 6.4. Feedback Loop for Distributed Workflows 152 6.5. Consistency Levels, Scalability and Scalable Consistency 157 6.6. Self-managed Workflows 158 6.7. Adaptations and Meta-adaptations 159 6.8. Multiple Feedback Loops and Process Instances 160 6.9. Transactions and ACID for CPS Workflows 161 6.10. Runtime View on Cyber-physical Synchronization for Workflows 162 6.11. Applicability of Workflow Feedback Loops to other CPS Domains 164 6.12. A Retrofitting Framework for Self-managed CPS WfMSes 165 7. Evaluation 171 7.1. Introduction 171 7.2. Hardware and Software 171 7.3. PROtEUS Base System 174 7.4. PROtEUS with Feedback Service 182 7.5. Feedback Service with Legacy WfMSes 213 7.6. Qualitative Discussion of Requirements and Additional CPS Aspects 217 7.7. Comparison with Related Work 232 7.8. Conclusion 234 8. Summary and Future Work 237 8.1. Summary and Conclusion 237 8.2. Advances of this Thesis 240 8.3. Contributions to the Research Area 242 8.4. Relevance 243 8.5. Open Questions 245 8.6. Future Work 247 Bibliography 249 Acronyms 277 List of Figures 281 List of Tables 285 List of Listings 287 Appendices 28