Metaphor-based negotiation and its application in AGV movement planning

The theme of this thesis is "metaphor-based negotiation". By metaphor-based negotiation I mean a category of approaches for problem-solving in Distributed Artificial Intelligence (DAI) that mimic some aspects of human negotiation behaviour. The research in this dissertation is divided into two closely related parts. Cooperative interaction among agents in a multiagent system (MAS) is discussed in general, and the discussion leads to a formal definition of metaphor-based negotiation. Then, as a specific application, a "spring-based" computational model for metaphor-based negotiation is developed as an approach to solving movement planning, specifically the AGV scheduling problem (AGVSP) — determing the timings of AGVs' activities, of automated guided vehicles (AGVs) in a factory.By formally addressing the multi-agent cooperative interaction problem and assuming that agents in a MAS are rational, benevolent and fully informed, an initial strategy set of cooperative interaction can be reduced to a strategy set by eliminating strategies that are irrational in a group sense. However, it is proved in this dissertation that, in the remaining strategy set, no unique strategy can be found that is acceptable to all agents according their individual preferences. More specifically, in this smaller strategy set, if one agent moves from one strategy to another in an attempt to better its individual goal achievement, then there is at least one agent whose goal achievement will be negatively affected by such a move. So, the cooperative interaction problem can only be partially solved if no further knowledge is given to those agents. The idea of a common sense principle is introduced in this dissertation to overcome the deficiencies of the assumptions of rationality, benevolence and full-informedness.In reality, the assumption of full-informedness of agents may not be practical. Communication is needed for agents to (1) exchange their local problem solving information, and (2) exchange proposals for global problem solving, when their views are in conflict. Based on the discussion of cooperative interaction, a formal definition of metaphorbased negotiation is proposed to formally indicate what is a proposal and what is the condition for accepting a proposal from another agent. In this definition, the common sense principle is one of the most important features, not found in definitions of negotiation available so far in the literature, which guides agents to find an agreement when negotiation is running into difficulties.The AGVSP involves timing activities for each AGV in a AGV-based factory. The AGVSP is naturally distributed: the whole problem can be easily divided into several subproblems each of which involves timing of activities of one AGV. Therefore, it is intuitively straightforward for us to seek DAI approaches to solving the AGVSP. In spired by Kwa's Iterative Negotiation Model [Kwa 88b] [Kwa 88a] for the AGVSP, we developed a spring-based (metaphor-based) negotiation model for the AGVSP to overcome some vital problems in Kwa's model. The idea of the spring-based negotiation model is described below:The AGVSP can be regarded as a Distributed Constraint Satisfaction Problem (DCSP) and solved in a MAS. Each agent in the MAS is designed to solve a subproblem — a local scheduling problem which is a small Constraint Satisfaction Problem (CSP). Conflicts exist when intra-agent constraints or inter-agent constraints are violated. These constraints can be classified into hard constraints— those that can not be relaxed at the agent level unless the system designer permits (e.g., by providing an arbitrator), and soft constraints — those that can be relaxed at the agent level when necessary. When agents are in conflict, i.e, when some inter-agent constraints are violated (or say, when one agent's timings of its activities overlap those of some other agents), these agents involved will resolve the conflicts through a (metaphor-based) negotiation procedure in which conflicts will be gradually resolved by each agent's relaxation of its intra-agent constraints, i.e, by yielding some amount of its initially allocated resources to other agents or by shifting its initially allocated resources. The negotiation can be viewed as a process of exchanging proposals (of cooperative strategies) between conflicting agents, where a cooperative strategy is a possible resolution to a conflict according to the viewpoint of the proposing agent. However, since agents are designed to be rational, each agent that is involved in the conflicts will try hard to relax its intra-agent constraints as little as possible. Further, it is reasonably acceptable that the more an intra-agent constraint has been relaxed the less the respective agent is willing to relax it further. This feature can be modeled by a spring — the more it has been compressed the harder it is to compress it further. Based on this inspiration, a spring-based computational model of metaphor-based negotiation is proposed: each agent's local schedule is represented by a local spring network in which each spring element represents a soft intra-agent constraint. Relaxation of an intra-agent constraint is likened to a spring being compressed by external forces from other agents. As a consequence, the compressed spring will also show a reacting force upon those compressing agents. An agreement will be reached when those forces and reacting forces are balanced. This is the common sense principle in the spring-based negotiation. The model solves some key issues, e.g., how to select negotiation techniques and skills during the process of negotiation, that have not been solved by Kwa's iterative negotiation model. Some experimental evidence of the value of this model is presented

Engineering coordination : eine Methodologie für die Koordination von Planungssystemen

Planning problems, like real-world planning and scheduling problems, are complex tasks. As an efficient strategy for handing such problems is the ‘divide and conquer’ strategy has been identified. Each sub problem is then solved independently. Typically the sub problems are solved in a linear way. This approach enables the generation of sub-optimal plans for a number of real world problems. Today, this approach is widely accepted and has been established e.g. in the organizational structure of companies. But existing interdependencies between the sub problems are not sufficiently regarded, as each problem are solved sequentially and no feedback information is given. The field of coordination has been covered by a number of academic fields, like the distributed artificial intelligence, economics or game theory. An important result is, that there exist no method that leads to optimal results in any given coordination problem. Consequently, a suitable coordination mechanism has to be identified for each single coordination problem. Up to now, there exists no process for the selection of a coordination mechanism, neither in the engineering of distributed systems nor in agent oriented software engineering. Within the scope of this work the ECo process is presented, that address exactly this selection problem. The Eco process contains the following five steps. • Modeling of the coordination problem • Defining the coordination requirements • Selection / Design of the coordination mechanism • Implementation • Evaluation Each of these steps is detailed in the thesis. The modeling has to be done to enable a systemic analysis of the coordination problem. Coordination mechanisms have to respect the given situation and the context in which the coordination has to be done. The requirements imposed by the context of the coordination problem are formalized in the coordination requirements. The selection process is driven by these coordination requirements. Using the requirements as a distinction for the selection of a coordination mechanism is a central aspect of this thesis. Additionally these requirements can be used for documentation of design decisions. Therefore, it is reasonable to annotate the coordination mechanisms with the coordination requirements they fulfill and fail to ease the selection process, for a given situation. For that reason we present a new classification scheme for coordination methods within this thesis that classifies existing coordination methods according to a set of criteria that has been identified as important for the distinction between different coordination methods. The implementation phase of the ECo process is supported by the CoPS process and CoPS framework that has been developed within this thesis, as well. The CoPS process structures the design making that has to be done during the implementation phase. The CoPS framework provides a set of basic features software agents need for realizing the selected coordination method. Within the CoPS process techniques are presented for the design and implementation of conversations between agents that can be applied not only within the context of the coordination of planning systems, but for multiagent systems in general. The ECo-CoPS approach has been successfully validated in two case studies from the logistic domain.Reale Planungsprobleme, wie etwa die Produktionsplanung in einer Supply Chain, sind komplex Planungsprobleme. Eine übliche Strategie derart komplexen Problemen zu lösen, ist es diese Probleme in einfachere Teilprobleme zu zerlegen und diese dann separat, meist sequentiell, zu lösen (divide-and-conquer Strategie). Dieser Ansatz erlaubt die Erstellung von (suboptimalen) Plänen für eine Reihe von realen Anwendungen, und ist heute in den Organisationsstrukturen von größeren Unternehmen institutionalisiert worden. Allerdings werden Abhängigkeiten zwischen den Teilproblemen nicht ausreichend berücksichtigt, da die Partialprobleme sequentiell ohne Feedback gelöst werden. Die erstellten Teillösungen müssen deswegen oft nachträglich koordiniert werden. Das Gebiet der Koordination wird in verschiedenen Forschungsgebieten, wie etwa der verteilten Künstlichen Intelligenz, den Wirtschaftswissenschaften oder der Spieltheorie untersucht. Ein zentrales Ergebnis dieser Forschung ist, dass es keinen für alle Situationen geeigneten Koordinationsmechanismus gibt. Es stellt sich also die Aufgabe aus den zahlreichen vorgeschlagenen Koordinationsmechanismen eine Auswahl zu treffen, die für die aktuelle Situation den geeigneten Mechanismus identifiziert. Für die Auswahl eines solchen Mechanismus existiert bisher jedoch kein strukturiertes Verfahren für die Entwicklung von verteilten Systems und insbesondere im Bereich der Agenten orientierter Softwareentwicklung. Im Rahmen dieser Arbeit wird genau hierfür ein Verfahren vorgestellt, der ECo-Prozess. Mit Hilfe dieses Prozesses wird der Auswahlprozess in die folgenden Schritte eingeteilt: • Modellierung der Problemstellung und des relevante Kontextes • Formulierung von Anforderungen an einen Koordinationsmechanismus (coordination requirements) • Auswahl/Entwurf eines Koordinationsmechanismuses • Implementierung des Koordinationsverfahrens • Evaluation des Koordinationsverfahrens Diese Schritte werden im Rahmen der vorliegenden Arbeit detailliert beschrieben. Die Modellierung der Problemstellung stellt dabei den ersten Schritt dar, um die Problemstellung analytisch zugänglich zu machen. Koordinationsverfahren müssen die Gegebenheiten, den Kontext und die Domäne, in der sie angewendet werden sollen hinreichend berücksichtigen um anwendbar zu sein. Dieses kann über Anforderungen an den Koordinationsprozess formalisiert werden. Der von den Anforderungen getrieben Auswahlprozess ist ein Kernstück der hier vorgestellten Arbeit. Durch die Formulierung der Anforderungen und der Annotation eines Koordinationsmechanismus bezüglich der erfüllten und nicht erfüllten Anforderungen werden die Motive für Designentscheidungen dieses Verfahren expliziert. Wenn Koordinationsverfahren anhand dieser Anforderungen klassifiziert werden können, ist es weiterhin möglich den Auswahlprozess (unabhängig vom ECo-Ansatz) zu vereinfachen und zu beschleunigen. Im Rahmen dieser Arbeit wird eine Klassifikation von Koordinationsansätzen anhand von allgemeinen Kriterien vorgestellt, die die Identifikation von geeigneten Kandidaten erleichtern. Diese Kandidaten können dann detaillierter untersucht werden. Dies wurde in den vorgestellten Fallstudien erfolgreich demonstriert. Für die Unterstützung der Implementierung eines Koordinationsansatzes wird in dieser Arbeit zusätzlich der CoPS Prozess vorgeschlagen. Der CoPS Prozess erlaubt einen ganzheitlichen systematischen Ansatz für den Entwurf und die Implementierung eines Koordinationsverfahrens. Unterstürzt wird der CoPS Prozess durch das CoPS Framework, das die Implementierung erleichtert, indem es als eine Plattform mit Basisfunktionalität eines Agenten bereitstellt, der für die Koordination von Planungssystemen verantwortlich ist. Im Rahmen des CoPS Verfahrens werden Techniken für den Entwurf und die Implementierung von Konversation im Kontext des agenten-orientiertem Software Engineerings ausführlich behandelt. Der Entwurf von Konversationen geht dabei weit über Fragestellung der Formatierung von Nachrichten hinaus, wie dies etwa in den FIPA Standards geregelt ist, und ist für die Implementierung von agentenbasierten Systemen im Allgemeinen von Bedeutung. Die Funktionsweise des ECo-CoPS Ansatzes wird anhand von zweierfolgreich durchgeführten Fallstudien aus dem betriebswirtschaftlichen Kontext vorgestellt

Automated Negotiation Among Web Services

Software as a service is well accepted software deployment and distribution model that is grown exponentially in the last few years. One of the biggest benefits of SaaS is the automated composition of these services in a composite system. It allows users to automatically find and bind these services, as to maximize the productivity of their composed systems, meeting both functional and non-functional requirements. In this paper we present a framework for modeling the dependency relationship of different Quality of Service parameters of a component service. Our proposed approach considers the different invocation patterns of component services in the system and models the dependency relationship for optimum values of these QoS parameters. We present a service composition framework that models the dependency relations ship among component services and uses the global QoS for service selection

Collaborative and adaptive supply chain planning

Dans le contexte industriel d'aujourd'hui, la compétitivité est fortement liée à la performance de la chaîne d'approvisionnement. En d'autres termes, il est essentiel que les unités d'affaires de la chaîne collaborent pour coordonner efficacement leurs activités de production, de façon a produire et livrer les produits à temps, à un coût raisonnable. Pour atteindre cet objectif, nous croyons qu'il est nécessaire que les entreprises adaptent leurs stratégies de planification, que nous appelons comportements, aux différentes situations auxquelles elles font face. En ayant une connaissance de l'impact de leurs comportements de planification sur la performance de la chaîne d'approvisionnement, les entreprises peuvent alors adapter leur comportement plutôt que d'utiliser toujours le même. Cette thèse de doctorat porte sur l'adaptation des comportements de planification des membres d'une même chaîne d'approvisionnement. Chaque membre pouvant choisir un comportement différent et toutes les combinaisons de ces comportements ayant potentiellement un impact sur la performance globale, il est difficile de connaître à l'avance l'ensemble des comportements à adopter pour améliorer cette performance. Il devient alors intéressant de simuler les différentes combinaisons de comportements dans différentes situations et d'évaluer les performances de chacun. Pour permettre l'utilisation de plusieurs comportements dans différentes situations, en utilisant la technologie à base d'agents, nous avons conçu un modèle d'agent à comportements multiples qui a la capacité d'adapter son comportement de planification selon la situation. Les agents planificateurs ont alors la possibilité de se coordonner de façon collaborative pour améliorer leur performance collective. En modélisant les unités d'affaires par des agents, nous avons simulé avec la plateforme de planification à base d'agents de FORAC des agents utilisant différents comportements de planification dits de réaction et de négociation. Cette plateforme, développée par le consortium de recherche FORAC de l'Université Laval, permet de simuler des décisions de planification et de planifier les opérations de la chaîne d'approvisionnement. Ces comportements de planification sont des métaheurisciques organisationnelles qui permettent aux agents de générer des plans de production différents. La simulation est basée sur un cas illustrant la chaîne d'approvisionnement de l'industrie du bois d'œuvre. Les résultats obtenus par l'utilisation de multiples comportements de réaction et de négociation montrent que les systèmes de planification avancée peuvent tirer avantage de disposer de plusieurs comportements de planification, en raIson du contexte dynamique des chaînes d'approvisionnement. La pertinence des résultats de cette thèse dépend de la prémisse que les entreprises qui adapteront leurs comportements de planification aux autres et à leur environnement auront un avantage concurrentiel important sur leurs adversaires

Time constraint agents? coordination and learning in cooperative multi-agent system

Ph.DDOCTOR OF PHILOSOPH

Agents for educational games and simulations

This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications

Coalition based approach for shop floor agility – a multiagent approach

Dissertation submitted for a PhD degree in Electrical Engineering, speciality of Robotics and Integrated Manufacturing from the Universidade Nova de Lisboa, Faculdade de Ciências e TecnologiaThis thesis addresses the problem of shop floor agility. In order to cope with the disturbances and uncertainties that characterise the current business scenarios faced by manufacturing companies, the capability of their shop floors needs to be improved quickly, such that these shop floors may be adapted, changed or become easily modifiable (shop floor reengineering). One of the critical elements in any shop floor reengineering process is the way the control/supervision architecture is changed or modified to accommodate for the new processes and equipment. This thesis, therefore, proposes an architecture to support the fast adaptation or changes in the control/supervision architecture. This architecture postulates that manufacturing systems are no more than compositions of modularised manufacturing components whose interactions when aggregated are governed by contractual mechanisms that favour configuration over reprogramming. A multiagent based reference architecture called Coalition Based Approach for Shop floor Agility – CoBASA, was created to support fast adaptation and changes of shop floor control architectures with minimal effort. The coalitions are composed of agentified manufacturing components (modules), whose relationships within the coalitions are governed by contracts that are configured whenever a coalition is established. Creating and changing a coalition do not involve programming effort because it only requires changes to the contract that regulates it

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic