Synthesized cooperative strategies for intelligent multi-robots in a real-time distributed environment : a thesis presented in partial fulfillment of the requirements for the degree of Master of Science in Computer Science at Massey University, Albany, New Zealand

In the robot soccer domain, real-time response usually curtails the development of more complex Al-based game strategies, path-planning and team cooperation between intelligent agents. In light of this problem, distributing computationally intensive algorithms between several machines to control, coordinate and dynamically assign roles to a team of robots, and allowing them to communicate via a network gives rise to real-time cooperation in a multi-robotic team. This research presents a myriad of algorithms tested on a distributed system platform that allows for cooperating multi- agents in a dynamic environment. The test bed is an extension of a popular robot simulation system in the public domain developed at Carnegie Mellon University, known as TeamBots. A low-level real-time network game protocol using TCP/IP and UDP were incorporated to allow for a conglomeration of multi-agent to communicate and work cohesively as a team. Intelligent agents were defined to take on roles such as game coach agent, vision agent, and soccer player agents. Further, team cooperation is demonstrated by integrating a real-time fuzzy logic-based ball-passing algorithm and a fuzzy logic algorithm for path planning. Keywords Artificial Intelligence, Ball Passing, the coaching system, Collaborative, Distributed Multi-Agent, Fuzzy Logic, Role Assignmen

BEHAVIOR BASED CONTROL AND FUZZY Q-LEARNING FOR AUTONOMOUS FIVE LEGS ROBOT NAVIGATION

This paper presents collaboration of behavior based control and fuzzy Q-learning for five legs robot navigation systems. There are many fuzzy Q-learning algorithms that have been proposed to yield individual behavior like obstacle avoidance, find target and so on. However, for complicated tasks, it is needed to combine all behaviors in one control schema using behavior based control. Based this fact, this paper proposes a control schema that incorporate fuzzy q-learning in behavior based schema to overcome complicated tasks in navigation systems of autonomous five legs robot. In the proposed schema, there are two behaviors which is learned by fuzzy q-learning. Other behaviors is constructed in design step. All behaviors are coordinated by hierarchical hybrid coordination node. Simulation results demonstrate that the robot with proposed schema is able to learn the right policy, to avoid obstacle and to find the target. However, Fuzzy q-learning failed to give right policy for the robot to avoid collision in the corner location. Keywords : behavior based control, fuzzy q-learnin

Application of Fuzzy State Aggregation and Policy Hill Climbing to Multi-Agent Systems in Stochastic Environments

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually even as the operating environment changes. Applying this learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing (PHC) and fuzzy state aggregation (FSA) function approximation is tested in two stochastic environments; Tileworld and the robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning lone. Results from the RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing

Fuzzy State Aggregation and Policy Hill Climbing for Stochastic Environments

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually learn even as the operating environment changes. Additionally, by applying reinforcement learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the fastest policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing and fuzzy state aggregation function approximation is tested in two stochastic environments: Tileworld and the simulated robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning reinforcement learning alone. Results from the multi-agent RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing

Using ABC² in the RoboCup domain

Proceeding of: Robot Soccer World Cup I, RoboCup-97, Nagoya, Japan, 1997This paper presents an architecture for the control of autonomus agents that allows explicit cooperation among them. The structure of the software agents controlling the robots is based on a general purpose multi-agent architecture based on a two level approach. One level is composed of reactive skills capable of achieving simple actions by their own. The other is based on an agenda used as an opportunistic planning mechanism to compound, activate and coordinate the basic skills. This agenda handles actions both from the internal goals of the robot or from other robots. This paper describes the work already accomplished, as well as the issues arising from the implementation of the architecture and its use in the RoboCup domain.Publicad