Combining Vision Verification with a High Level Robot Programming Language

This thesis describes work on using vision verification within an object level language for describing robot assembly (RAPT). The motivation for this thesis is provided by two problems. The first is how to enhance a high level robot programming language so that it can encompass vision commands to locate workpieces of an assembly. The second is how to find a way of making full use of sensory information to update the robot system's knowledge about the environment. The work described in this thesis consists of three parts: (1) adding vision commands into the RAPT input language so that the user can specify vision verification tasks; (2) implementing a symbolic geometrical reasoning system so that vision data can be reasoned about symbolically at compile time in order to speed up run time operations; (3) providing a framework which enables the RAPT system to make full use of the sensory information. The vision commands allow partial information about positions to be combined with sensory information in a general way, and the symbolic reasoning system allows much of the reasoning work about vision information to be done before the actual information is obtained. The framework combines a verification vision facility with an object level language in an intelligent way so that all ramifications of the effects of sensory data are taken account of. The heart of the framework is the modifying factor array. The position of each object is expressed as the product of two parts: the planned position and the difference between this and "he actual one. This difference, referred to as the modifying factor of an object, is stored in the modifying factor array. The planned position is described by the user in the usual way in a RAPT program and its value is inferred by the RAPT reasoning system. Modifying factors of objects whose positions are directly verified are defined at compile time as symbolic expressions containing variables whose value will become known at run time. The modifying factors of other objects (not directly verified) may be dependent upon positions of objects which are verified. At compile time the framework reasons about the influence of the sensory information on the objects which are not verified directly by the vision system, and establishes connections among modifying factors of objects in each situation. This framework makes the representation of the influence of vision information on the robot's knowledge of the environment compact and simple. All the programming has been done. It has been tested with simulated data and works successfully

Achieving reliability using behavioural modules in a robotic assembly system

The research in this thesis looks at improving the reliability of robotic as¬ sembly while still retaining the flexibility to change the system to cope with dif¬ ferent assemblies. The lack of a truly flexible robotic assembly system presents a problem which current systems have yet to overcome. An experimental sys¬ tem has been designed and implemented to demonstrate the ideas presented in this work. Runs of this system have also been performed to test and assess the scheme which has been developed.The Behaviour-based SOMASS system looks at decomposing the task into modular units, called Behavioural Modules, which reliably perform the as¬ sembly task by using variation reducing strategies. The thesis work looks at expanding this framework to produce a system which relaxes the constraints of complete reliability within a Behavioural Module by embedding these in a re¬ liable system architecture. This means that Behavioural Modules do not have to guarantee to successfully perform their given task but instead can perform it adequately, with occasional failures dealt with by the appropriate introduction of alternative actionsTo do this, the concepts of Exit States, the Ideal Execution Path, and Alter¬ native Execution Paths have been described. The Exit State of a Behavioural Module gives an indication of the control path which has actually been taken during its execution. This information, along with appropriate information available to the execution system (such as sensor and planner data), allows the Ideal Execution Path and Alternative Execution Paths to be defined. These show, respectively, the best control path through the system (as determined by the system designer) and alternative control routes which can be taken when necessary

The design and implementation of vision-based behavioural modules for a robotic assembly system

The work drsrrihrd in this thesis ia about, how to program robots to work re liably in the presence of uncertainty. Some architectural principle!: are proposed which address the problem of decomposing robotic assembly tasks into modular units such that a robot program can be implemented efficiently, tested easily, and can be maintained or modified without undue complexity. This architecture also provides a framework to integrate sensors into a robotic, assembly system.These modular units arc called behavioural modules. They perforin their tasks reliably. The problem of uncertainty is dealt with by encapsulating sensing and variation reducing strategies inside these modules. Experiments are performed with a working robotic assembly system using vision based behavioural modules. Analy sis of this system validates the principles presented in this thesis

Multi-object recognition and retrieval using Puma560 robot

The objective of the research described here is to develop efficient algorithm and software tools for multiobject recognition and retrieval. This research project addresses two major issues: The first issue is the identification of features and efficient methods for feature extraction which can completely describe an object. These features can be acquired using visual and ultra-sonic sensors. The second issue is the development of efficient algorithms for the retrieval of multi-objects based on their features; The methods and algorithms developed in this research are verified on a Unimation PUMA 560 robot. Non contact sensors (a vision and a range sensor) are employed for feature detection. The information from both sensors will be combined for feature extraction and feature mapping (sensor fusion). The sensors and the robot have been integrated for this purpose with a Pentium 133 Mhz Personal Computer

Plethora : a framework for the intelligent control of robotic assembly systems

Plethora : a framework for the intelligent control of robotic assembly system

Accomplishing task-invariant assembly strategies by means of an inherently accommodating robot arm

Despite the fact that the main advantage of robot manipulators was always meant to be their flexibility, they have not been applied widely to the assembly of industrial components in situations other than those where hard automation might be used. We identify the two main reasons for this as the 'fragility' of robot operation during tasks that involve contact, and the lack of an appropriate user interface. This thesis describes an attempt to address these problems.We survey the techniques that have been proposed to bring the performance of cur¬ rent industrial robot manipulators in line with expectations, and conclude that the main obstacle in realising a flexible assembly robot that exhibits robust and reliable behaviour is the problem of spatial uncertainty.Based on observations of the performance of position-controlled robot manipulators and what is involved during rigid-body part mating, we propose a model of assembly tasks that exploits the shape invariance of the part geometry across instances of a task. This allows us to escape from the problem of spatial uncertainty because we are 110 longer working in spatial terms. In addition, because the descriptions of assembly tasks that we derive are task-invariant, i.e. they are not dependent on part size or location, they lend themselves naturally to a task-level programming interface, thereby simplifying the process of programming an assembly robot.the process of programming an assembly robot. However, to test this approach empirically requires a manipulator that is able to control the force that it applies, as well as being sensitive to environmental constraints. The inertial properties of standard industrial manipulators preclude them from exhibiting this kind of behaviour. In order to solve this problem we designed and constructed a three degree of freedom, planar, direct-drive arm that is open-loop force-controllable (with respect to its end-point), and inherently accommodating during contact.In order to demonstrate the forgiving nature of operation of our robot arm we imple¬ mented a generic crank turning program that is independent of the geometry of the crank involved, i.e. no knowledge is required of the location or length of the crank. I11 order to demonstrate the viability of our proposed approach to assembly we pro¬ grammed our robot system to perform some representative tasks; the insertion of a peg into a hole, and the rotation of a block into a corner. These programs were tested on parts of various size and material, and in various locations in order to illustrate their invariant nature.We conclude that the problem of spatial uncertainty is in fact an artefact of the fact that current industrial manipulators are designed to be position controlled. The work described in this thesis shows that assembly robots, when appropriately designed, controlled and programmed, can be the reliable and flexible devices they were always meant to be

Programming of Path Specific Robot Operations with Optimal Part Placement

In this paper we describe a task level programming system for path specific robot operations. We define path specific tasks as those robot tasks in which the path the manipulator end effector has to follow is fixed and is given, such operations may include welding or sealant application. The initial path selection is made through a graphical interface using a pointing device (such as a mouse) to outline the desired path on a CAD model of the workpiece. The final result of the system is the part location, which enables the chosen manipulator to optimally perform the desired task. Optimality is based on maximizing the manipulability of the manipulator performing the task using a function of the jacobian. User defined constraints, joint limit constraints, and collision avoidance constraints are used to guide the optimal location selection. The workable task is then executed using calls to a C language based motion control library outlined in [Guptill88] [Guptill & Stahura 87]. The usefulness of the system described in this paper is indicated by an example of two robotic devices performing a down-hand welding operation

Task Planner for Simultaneous Fulfillment of Operational, Geometric and Uncertainty-Reduction Goals

Our ultimate goal in robot planning is to develop a planner which can create complete assembly plans given as input a high level description of assembly goals, geometric models of the components of the assembly, and a description of the capabilities of the work cell (including the robot and the sensory system). In this paper, we introduce SPAR, a planning system which reasons about high level operational goals, geometric goals and uncertainty-reduction goals in order to create assembly plans which consist of manipulations as well as sensory operations when appropriate. Operational planning is done using a nonlinear, constraint posting planner. Geometric planning is accomplished by constraining the execution of operations in the plan so that geometric goals are satisfied, or, if the geometric configuration of the world prevents this, by introducing new operations into the plan with the appropriate constraints. When the uncertainty in the world description exceeds that specified by the uncertainty-reduction goals, SPAR introduces either sensing operations or manipulations to reduce that uncertainty to acceptable levels. If SPAR cannot find a way to sufficiently reduce uncertainties, it does not abandon the plan. Instead, it augments the plan with sensing operations to be used to verify the execution of the action, and, when possible, posts possible error recovery plans, although at this point, the verification operations and recovery plans are predefined

Advancement in robot programming with specific reference to graphical methods

This research study is concerned with the derivation of advanced robot programming methods. The methods include the use of proprietary simulation modelling and design software tools for the off-line programming of industrial robots. The study has involved the generation of integration software to facilitate the co-operative operation of these software tools. The three major researcli'themes7of "ease of usage", calibration and the integration of product design data have been followed to advance robot programming. The "ease of usage" is concerned with enhancements in the man-machine interface for robo t simulation systems in terms of computer assisted solid modelling and computer assisted task generation. Robot simulation models represent an idealised situation, and any off-line robot programs generated from'them may contain'discrepancies which could seriously effect thq programs' performance; Calibration techniques have therefore been investigated as 'a method of overcoming discrepancies between the simulation model and the real world. At the present time, most computer aided design systems operate as isolated islands of computer technology, whereas their product databases should be used to support decision making processes and ultimately facilitate the generation of machine programs. Thus the integration of product design data has been studied as an important step towards truly computer integrated manufacturing. The functionality of the three areas of study have been generalised and form the basis for recommended enhancements to future robot programming systems