Distributed Architecture for Dynamic Role Behaviour in Humanoid Soccer Robots

Non

Architecting centralized coordination of soccer robots based on principle solution

This is an Accepted Manuscript of an article published by Taylor & Francis in Advanced Robotics on 2015, available online:http://www.tandfonline.com/10.1080/01691864.2015.1017534Coordination strategy is a relevant topic in multi-robot systems, and robot soccer offers a suitable domain to conduct research in multi-robot coordination. Team strategy collects and uses environmental information to derive optimal team reactions, through cooperation among individual soccer robots. This paper presents a diagrammatic approach to architecting the coordination strategy of robot soccer teams by means of a principle solution. The proposed model focuses on robot soccer leagues that possess a central decision-making system, involving the dynamic selection of the roles and behaviors of the robot soccer players. The work sets out from the conceptual design phase, facilitating cross-domain development efforts, where different layers must be interconnected and coordinated to perform multiple tasks. The principle solution allows for intuitive design and the modeling of team strategies in a highly complex robot soccer environment with changing game conditions. Furthermore, such an approach enables systematic realization of collaborative behaviors among the teammates.This work was partially supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under the CICYT project Mission Based Control (COBAMI): DPI2011-28507-C02-01/02. Jose G. Guarnizo was supported by a scholarship from the Administrative Department of Science, Technology and Innovation COLCIENCIAS, Colombia.Guarnizo Marín, JG.; Mellado Arteche, M.; Low, CY.; Blanes Noguera, F. (2015). Architecting centralized coordination of soccer robots based on principle solution. Advanced Robotics. 29(15):989-1004. https://doi.org/10.1080/01691864.2015.1017534S98910042915Farinelli, A., Iocchi, L., & Nardi, D. (2004). Multirobot Systems: A Classification Focused on Coordination. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), 34(5), 2015-2028. doi:10.1109/tsmcb.2004.832155Tews, A., & Wyeth, G. (2000). MAPS: a system for multi-agent coordination. Advanced Robotics, 14(1), 37-50. doi:10.1163/156855300741429Stulp, F., Utz, H., Isik, M., & Mayer, G. (2010). Implicit Coordination with Shared Belief: A Heterogeneous Robot Soccer Team Case Study. Advanced Robotics, 24(7), 1017-1036. doi:10.1163/016918610x496964Guarnizo, J. G., Mellado, M., Low, C. Y., & Aziz, N. (2013). Strategy Model for Multi-Robot Coordination in Robotic Soccer. Applied Mechanics and Materials, 393, 592-597. doi:10.4028/www.scientific.net/amm.393.592Riley, P., & Veloso, M. (2002). Recognizing Probabilistic Opponent Movement Models. Lecture Notes in Computer Science, 453-458. doi:10.1007/3-540-45603-1_59Ros, R., Arcos, J. L., Lopez de Mantaras, R., & Veloso, M. (2009). A case-based approach for coordinated action selection in robot soccer. Artificial Intelligence, 173(9-10), 1014-1039. doi:10.1016/j.artint.2009.02.004Atkinson, J., & Rojas, D. (2009). On-the-fly generation of multi-robot team formation strategies based on game conditions. Expert Systems with Applications, 36(3), 6082-6090. doi:10.1016/j.eswa.2008.07.039Costelha, H., & Lima, P. (2012). Robot task plan representation by Petri nets: modelling, identification, analysis and execution. Autonomous Robots, 33(4), 337-360. doi:10.1007/s10514-012-9288-xAbreu, P. H., Silva, D. C., Almeida, F., & Mendes-Moreira, J. (2014). Improving a simulated soccer team’s performance through a Memory-Based Collaborative Filtering approach. Applied Soft Computing, 23, 180-193. doi:10.1016/j.asoc.2014.06.021Duan, Y., Liu, Q., & Xu, X. (2007). Application of reinforcement learning in robot soccer. Engineering Applications of Artificial Intelligence, 20(7), 936-950. doi:10.1016/j.engappai.2007.01.003Hwang, K.-S., Jiang, W.-C., Yu, H.-H., & Li, S.-Y. (2011). Cooperative Reinforcement Learning Based on Zero-Sum Games. Mobile Robots - Control Architectures, Bio-Interfacing, Navigation, Multi Robot Motion Planning and Operator Training. doi:10.5772/26620Gausemeier, J., Dumitrescu, R., Kahl, S., & Nordsiek, D. (2011). Integrative development of product and production system for mechatronic products. Robotics and Computer-Integrated Manufacturing, 27(4), 772-778. doi:10.1016/j.rcim.2011.02.005Klančar, G., Zupančič, B., & Karba, R. (2007). Modelling and simulation of a group of mobile robots. Simulation Modelling Practice and Theory, 15(6), 647-658. doi:10.1016/j.simpat.2007.02.002Gausemeier, J., Frank, U., Donoth, J., & Kahl, S. (2009). Specification technique for the description of self-optimizing mechatronic systems. Research in Engineering Design, 20(4), 201-223. doi:10.1007/s00163-008-0058-

Predicting performance in team games: The automatic coach

This is an electronic version of the paper presented at the 3rd International Conference on Agents and Artificial Intelligence, held in Rome on 2011A wide range of modern videogames involves a number of players collaborating to obtain a common goal. The way the players are teamed up is usually based on a measure of performance that makes players with a similar level of performance play together. We propose a novel technique based on clustering over observed behaviour in the game that seeks to exploit the particular way of playing of every player to find other players with a gameplay such that in combination will constitute a good team, in a similar way to a human coach. This paper describes the preliminary results using these techniques for the characterization of player and team behaviours. Experiments are performed in the domain of Soccerbots.This work has been partly supported by: Spanish Ministry of Science and Education under grant TIN2009-13692-C03-03, TIN2010-19872 and Spanish Ministry of Industry under grant TSI, 020110- 2009-205

Denoising Opponents Position in Partial Observation Environment

The RoboCup competitions hold various leagues, and the Soccer Simulation 2D League is a major among them. Soccer Simulation 2D (SS2D) match involves two teams, including 11 players and a coach for each team, competing against each other. The players can only communicate with the Soccer Simulation Server during the game. Several code bases are released publicly to simplify team development. So researchers can easily focus on decision-making and implementing machine learning methods. SS2D actions and behaviors are only partially accurate due to different challenges, such as noise and partial observation. Therefore, one strategy is to implement alternative denoising methods to tackle observation inaccuracy. Our idea is to predict opponent positions while they have yet to be seen in a finite number of cycles using machine learning methods to make more accurate actions such as pass. We will explain our position prediction idea powered by Long Short-Term Memory models (LSTM) and Deep Neural Networks (DNN). The results show that the LSTM and DNN predict the opponents' position more accurately than the standard algorithm, such as the last-seen method

Exploiting Opponent Modeling For Learning In Multi-agent Adversarial Games

An issue with learning effective policies in multi-agent adversarial games is that the size of the search space can be prohibitively large when the actions of both teammates and opponents are considered simultaneously. Opponent modeling, predicting an opponent’s actions in advance of execution, is one approach for selecting actions in adversarial settings, but it is often performed in an ad hoc way. In this dissertation, we introduce several methods for using opponent modeling, in the form of predictions about the players’ physical movements, to learn team policies. To explore the problem of decision-making in multi-agent adversarial scenarios, we use our approach for both offline play generation and real-time team response in the Rush 2008 American football simulator. Simultaneously predicting the movement trajectories, future reward, and play strategies of multiple players in real-time is a daunting task but we illustrate how it is possible to divide and conquer this problem with an assortment of data-driven models. By leveraging spatio-temporal traces of player movements, we learn discriminative models of defensive play for opponent modeling. With the reward information from previous play matchups, we use a modified version of UCT (Upper Conference Bounds applied to Trees) to create new offensive plays and to learn play repairs to counter predicted opponent actions. iii In team games, players must coordinate effectively to accomplish tasks while foiling their opponents either in a preplanned or emergent manner. An effective team policy must generate the necessary coordination, yet considering all possibilities for creating coordinating subgroups is computationally infeasible. Automatically identifying and preserving the coordination between key subgroups of teammates can make search more productive by pruning policies that disrupt these relationships. We demonstrate that combining opponent modeling with automatic subgroup identification can be used to create team policies with a higher average yardage than either the baseline game or domain-specific heuristics

Discovering Strategic Behaviour of Multi-Agent Systems in Adversary Settings

Can specific behaviour strategies be induced from low-level observations of two adversary groups of agents with limited domain knowledge? This paper presents a domain-independent Multi-Agent Strategy Discovering Algorithm (MASDA), which discovers strategic behaviour patterns of a group of agents under the described conditions. The algorithm represents the observed multi-agent activity as a graph, where graph connections correspond to performed actions and graph nodes correspond to environment states at action starts. Based on such data representation, the algorithm applies hierarchical clustering and rule induction to extract and describe strategic behaviour. The discovered strategic behaviour is represented visually as graph paths and symbolically as rules. MASDA was evaluated on RoboCup. Both soccer experts and quantitative evaluation confirmed the relevance of the discovered behaviour patterns

Abstracting Multidimensional Concepts for Multilevel Decision Making in Multirobot Systems

Multirobot control architectures often require robotic tasks to be well defined before allocation. In complex missions, it is often difficult to decompose an objective into a set of well defined tasks; human operators generate a simplified representation based on experience and estimation. The result is a set of robot roles, which are not best suited to accomplishing those objectives. This thesis presents an alternative approach to generating multirobot control algorithms using task abstraction. By carefully analysing data recorded from similar systems a multidimensional and multilevel representation of the mission can be abstracted, which can be subsequently converted into a robotic controller. This work, which focuses on the control of a team of robots to play the complex game of football, is divided into three sections: In the first section we investigate the use of spatial structures in team games. Experimental results show that cooperative teams beat groups of individuals when competing for space and that controlling space is important in the game of robot football. In the second section, we generate a multilevel representation of robot football based on spatial structures measured in recorded matches. By differentiating between spatial configurations appearing in desirable and undesirable situations, we can abstract a strategy composed of the more desirable structures. In the third section, five partial strategies are generated, based on the abstracted structures, and a suitable controller is devised. A set of experiments shows the success of the method in reproducing those key structures in a multirobot system. Finally, we compile our methods into a formal architecture for task abstraction and control. The thesis concludes that generating multirobot control algorithms using task abstraction is appropriate for problems which are complex, weakly-defined, multilevel, dynamic, competitive, unpredictable, and which display emergent properties