On environment difficulty and discriminating power

The final publication is available at Springer via http://dx.doi.org/10.1007/s10458-014-9257-1This paper presents a way to estimate the difficulty and discriminating power of any task instance. We focus on a very general setting for tasks: interactive (possibly multiagent) environments where an agent acts upon observations and rewards. Instead of analysing the complexity of the environment, the state space or the actions that are performed by the agent, we analyse the performance of a population of agent policies against the task, leading to a distribution that is examined in terms of policy complexity. This distribution is then sliced by the algorithmic complexity of the policy and analysed through several diagrams and indicators. The notion of environment response curve is also introduced, by inverting the performance results into an ability scale. We apply all these concepts, diagrams and indicators to two illustrative problems: a class of agent-populated elementary cellular automata, showing how the difficulty and discriminating power may vary for several environments, and a multiagent system, where agents can become predators or preys, and may need to coordinate. Finally, we discuss how these tools can be applied to characterise (interactive) tasks and (multi-agent) environments. These characterisations can then be used to get more insight about agent performance and to facilitate the development of adaptive tests for the evaluation of agent abilities.I thank the reviewers for their comments, especially those aiming at a clearer connection with the field of multi-agent systems and the suggestion of better approximations for the calculation of the response curves. The implementation of the elementary cellular automata used in the environments is based on the library 'CellularAutomaton' by John Hughes for R [58]. I am grateful to Fernando Soler-Toscano for letting me know about their work [65] on the complexity of 2D objects generated by elementary cellular automata. I would also like to thank David L. Dowe for his comments on a previous version of this paper. This work was supported by the MEC/MINECO projects CONSOLIDER-INGENIO CSD2007-00022 and TIN 2010-21062-C02-02, GVA project PROMETEO/2008/051, the COST - European Cooperation in the field of Scientific and Technical Research IC0801 AT, and the REFRAME project, granted by the European Coordinated Research on Long-term Challenges in Information and Communication Sciences & Technologies ERA-Net (CHIST-ERA), and funded by the Ministerio de Economia y Competitividad in Spain (PCIN-2013-037).José Hernández-Orallo (2015). On environment difficulty and discriminating power. Autonomous Agents and Multi-Agent Systems. 29(3):402-454. https://doi.org/10.1007/s10458-014-9257-1S402454293Anderson, J., Baltes, J., & Cheng, C. T. (2011). Robotics competitions as benchmarks for ai research. The Knowledge Engineering Review, 26(01), 11–17.Andre, D., & Russell, S. J. (2002). State abstraction for programmable reinforcement learning agents. In Proceedings of the National Conference on Artificial Intelligence (pp. 119–125). Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.Antunes, L., Fortnow, L., van Melkebeek, D., & Vinodchandran, N. V. (2006). Computational depth: Concept and applications. Theoretical Computer Science, 354(3), 391–404. Foundations of Computation Theory (FCT 2003), 14th Symposium on Fundamentals of Computation Theory 2003.Arai, K., Kaminka, G. A., Frank, I., & Tanaka-Ishii, K. (2003). Performance competitions as research infrastructure: Large scale comparative studies of multi-agent teams. Autonomous Agents and Multi-Agent Systems, 7(1–2), 121–144.Ashcraft, M. H., Donley, R. D., Halas, M. A., & Vakali, M. (1992). Chapter 8 working memory, automaticity, and problem difficulty. In Jamie I.D. Campbell (Ed.), The nature and origins of mathematical skills, volume 91 of advances in psychology (pp. 301–329). North-Holland.Ay, N., Müller, M., & Szkola, A. (2010). Effective complexity and its relation to logical depth. IEEE Transactions on Information Theory, 56(9), 4593–4607.Barch, D. M., Braver, T. S., Nystrom, L. E., Forman, S. D., Noll, D. C., & Cohen, J. D. (1997). Dissociating working memory from task difficulty in human prefrontal cortex. Neuropsychologia, 35(10), 1373–1380.Bordini, R. H., Hübner, J. F., & Wooldridge, M. (2007). Programming multi-agent systems in AgentSpeak using Jason. London: Wiley. com.Boutilier, C., Reiter, R., Soutchanski, M., Thrun, S. et al. (2000). Decision-theoretic, high-level agent programming in the situation calculus. In Proceedings of the National Conference on Artificial Intelligence (pp. 355–362). Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.Busoniu, L., Babuska, R., & De Schutter, B. (2008). A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 38(2), 156–172.Chaitin, G. J. (1977). Algorithmic information theory. IBM Journal of Research and Development, 21, 350–359.Chedid, F. B. (2010). Sophistication and logical depth revisited. In 2010 IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) (pp. 1–4). IEEE.Cheeseman, P., Kanefsky, B. & Taylor, W. M. (1991). Where the really hard problems are. In Proceedings of IJCAI-1991 (pp. 331–337).Dastani, M. (2008). 2APL: A practical agent programming language. Autonomous Agents and Multi-agent Systems, 16(3), 214–248.Delahaye, J. P. & Zenil, H. (2011). Numerical evaluation of algorithmic complexity for short strings: A glance into the innermost structure of randomness. Applied Mathematics and Computation, 219(1), 63–77Dowe, D. L. (2008). Foreword re C. S. Wallace. Computer Journal, 51(5), 523–560. Christopher Stewart WALLACE (1933–2004) memorial special issue.Dowe, D. L., & Hernández-Orallo, J. (2012). IQ tests are not for machines, yet. Intelligence, 40(2), 77–81.Du, D. Z., & Ko, K. I. (2011). Theory of computational complexity (Vol. 58). London: Wiley-Interscience.Elo, A. E. (1978). The rating of chessplayers, past and present (Vol. 3). London: Batsford.Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. London: Lawrence Erlbaum.Fatès, N. & Chevrier, V. (2010). How important are updating schemes in multi-agent systems? an illustration on a multi-turmite model. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1-Volume 1 (pp. 533–540). International Foundation for Autonomous Agents and Multiagent Systems.Ferber, J. & Müller, J. P. (1996). Influences and reaction: A model of situated multiagent systems. In Proceedings of Second International Conference on Multi-Agent Systems (ICMAS-96) (pp. 72–79).Ferrando, P. J. (2009). Difficulty, discrimination, and information indices in the linear factor analysis model for continuous item responses. Applied Psychological Measurement, 33(1), 9–24.Ferrando, P. J. (2012). Assessing the discriminating power of item and test scores in the linear factor-analysis model. Psicológica, 33, 111–139.Gent, I. P., & Walsh, T. (1994). Easy problems are sometimes hard. Artificial Intelligence, 70(1), 335–345.Gershenson, C. & Fernandez, N. (2012). Complexity and information: Measuring emergence, self-organization, and homeostasis at multiple scales. Complexity, 18(2), 29–44.Gruner, S. (2010). Mobile agent systems and cellular automata. Autonomous Agents and Multi-agent Systems, 20(2), 198–233.Hardman, D. K., & Payne, S. J. (1995). Problem difficulty and response format in syllogistic reasoning. The Quarterly Journal of Experimental Psychology, 48(4), 945–975.He, J., Reeves, C., Witt, C., & Yao, X. (2007). A note on problem difficulty measures in black-box optimization: Classification, realizations and predictability. Evolutionary Computation, 15(4), 435–443.Hernández-Orallo, J. (2000). Beyond the turing test. Journal of Logic Language & Information, 9(4), 447–466.Hernández-Orallo, J. (2000). On the computational measurement of intelligence factors. In A. Meystel (Ed.), Performance metrics for intelligent systems workshop (pp. 1–8). Gaithersburg, MD: National Institute of Standards and Technology.Hernández-Orallo, J. (2000). Thesis: Computational measures of information gain and reinforcement in inference processes. AI Communications, 13(1), 49–50.Hernández-Orallo, J. (2010). A (hopefully) non-biased universal environment class for measuring intelligence of biological and artificial systems. In M. Hutter et al. (Ed.), 3rd International Conference on Artificial General Intelligence (pp. 182–183). Atlantis Press Extended report at http://users.dsic.upv.es/proy/anynt/unbiased.pdf .Hernández-Orallo, J., & Dowe, D. L. (2010). Measuring universal intelligence: Towards an anytime intelligence test. Artificial Intelligence, 174(18), 1508–1539.Hernández-Orallo, J., Dowe, D. L., España-Cubillo, S., Hernández-Lloreda, M. V., & Insa-Cabrera, J. (2011). On more realistic environment distributions for defining, evaluating and developing intelligence. In J. Schmidhuber, K. R. Thórisson, & M. Looks (Eds.), LNAI series on artificial general intelligence 2011 (Vol. 6830, pp. 82–91). Berlin: Springer.Hernández-Orallo, J., Dowe, D. L., & Hernández-Lloreda, M. V. (2014). Universal psychometrics: Measuring cognitive abilities in the machine kingdom. Cognitive Systems Research, 27, 50–74.Hernández-Orallo, J., Insa, J., Dowe, D. L. & Hibbard, B. (2012). Turing tests with turing machines. In A. Voronkov (Ed.), The Alan Turing Centenary Conference, Turing-100, Manchester, 2012, volume 10 of EPiC Series (pp. 140–156).Hernández-Orallo, J. & Minaya-Collado, N. (1998). A formal definition of intelligence based on an intensional variant of Kolmogorov complexity. In Proceedings of International Symposium of Engineering of Intelligent Systems (EIS’98) (pp. 146–163). ICSC Press.Hibbard, B. (2009). Bias and no free lunch in formal measures of intelligence. Journal of Artificial General Intelligence, 1(1), 54–61.Hoos, H. H. (1999). Sat-encodings, search space structure, and local search performance. In 1999 International Joint Conference on Artificial Intelligence (Vol. 16, pp. 296–303).Insa-Cabrera, J., Benacloch-Ayuso, J. L., & Hernández-Orallo, J. (2012). On measuring social intelligence: Experiments on competition and cooperation. In J. Bach, B. Goertzel, & M. Iklé (Eds.), AGI, volume 7716 of lecture notes in computer science (pp. 126–135). Berlin: Springer.Insa-Cabrera, J., Dowe, D. L., España-Cubillo, S., Hernández-Lloreda, M. V., & Hernández-Orallo, J. (2011). Comparing humans and AI agents. In J. Schmidhuber, K. R. Thórisson, & M. Looks (Eds.), LNAI series on artificial general intelligence 2011 (Vol. 6830, pp. 122–132). Berlin: Springer.Knuth, D. E. (1973). Sorting and searching, volume 3 of the art of computer programming. Reading, MA: Addison-Wesley.Kotovsky, K., & Simon, H. A. (1990). What makes some problems really hard: Explorations in the problem space of difficulty. Cognitive Psychology, 22(2), 143–183.Legg, S. (2008). Machine super intelligence. PhD thesis, Department of Informatics, University of Lugano, June 2008.Legg, S., & Hutter, M. (2007). Universal intelligence: A definition of machine intelligence. Minds and Machines, 17(4), 391–444.Leonetti, M. & Iocchi, L. (2010). Improving the performance of complex agent plans through reinforcement learning. In Proceedings of the 2010 International Conference on Autonomous Agents and Multiagent Systems (Vol. 1, pp. 723–730). International Foundation for Autonomous Agents and Multiagent Systems.Levin, L. A. (1973). Universal sequential search problems. Problems of Information Transmission, 9(3), 265–266.Levin, L. A. (1986). Average case complete problems. SIAM Journal on Computing, 15, 285.Li, M., & Vitányi, P. (2008). An introduction to Kolmogorov complexity and its applications (3rd ed.). Berlin: Springer.Low, C. K., Chen, T. Y., & Rónnquist, R. (1999). Automated test case generation for bdi agents. Autonomous Agents and Multi-agent Systems, 2(4), 311–332.Madden, M. G., & Howley, T. (2004). Transfer of experience between reinforcement learning environments with progressive difficulty. Artificial Intelligence Review, 21(3), 375–398.Mellenbergh, G. J. (1994). Generalized linear item response theory. Psychological Bulletin, 115(2), 300.Michel, F. (2004). Formalisme, outils et éléments méthodologiques pour la modélisation et la simulation multi-agents. PhD thesis, Université des sciences et techniques du Languedoc, Montpellier.Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81.Orponen, P., Ko, K. I., Schöning, U., & Watanabe, O. (1994). Instance complexity. Journal of the ACM (JACM), 41(1), 96–121.Simon, H. A., & Kotovsky, K. (1963). Human acquisition of concepts for sequential patterns. Psychological Review, 70(6), 534.Team, R., et al. (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.Whiteson, S., Tanner, B., & White, A. (2010). The reinforcement learning competitions. The AI Magazine, 31(2), 81–94.Wiering, M., & van Otterlo, M. (Eds.). (2012). Reinforcement learning: State-of-the-art. Berlin: Springer.Wolfram, S. (2002). A new kind of science. Champaign, IL: Wolfram Media.Zatuchna, Z., & Bagnall, A. (2009). Learning mazes with aliasing states: An LCS algorithm with associative perception. Adaptive Behavior, 17(1), 28–57.Zenil, H. (2010). Compression-based investigation of the dynamical properties of cellular automata and other systems. Complex Systems, 19(1), 1–28.Zenil, H. (2011). Une approche expérimentale à la théorie algorithmique de la complexité. PhD thesis, Dissertation in fulfilment of the degree of Doctor in Computer Science, Université de Lille.Zenil, H., Soler-Toscano, F., Delahaye, J. P. & Gauvrit, N. (2012). Two-dimensional kolmogorov complexity and validation of the coding theorem method by compressibility. arXiv, preprint arXiv:1212.6745

Strategies for cooperation emergence in distributed service discovery

This is an Accepted Manuscript of an article published by Taylor & Francis in Cybernetics and Systems on APR 3 2014], available online:http://www.tandfonline.com/10.1080/01969722.2014.894848[EN] In distributed environments where entities only have a partial view of the system, cooperation plays a key issue. In the case of decentralized service discovery in open agent societies, agents only know about the services they provide and who are their direct neighbors. Therefore, they need the cooperation of their neighbors in order to locate the required services. However, cooperation is not always present in open systems. Non-cooperative agents pursuing their own goals could refuse to forward queries from other agents to avoid the cost of this action; therefore, the efficiency of the decentralized service discovery could be seriously damaged. In this paper, we propose the ombination of incentives and local structural changes in order to promote cooperation in the service discovery process. The results show that, even in scenarios where the predominant behavior is not collaborative cooperation emerges.The work was partially supported by the Spanish Ministry of Science and Innovation through grants TIN2009-13839-C03-01, TIN2012-36586-C03-01, CSD2007-0022 (CONSOLIDER-INGENIO 2010).Del Val Noguera, E.; Rebollo Pedruelo, M.; Botti, V. (2014). Strategies for cooperation emergence in distributed service discovery. Cybernetics and Systems. 45(3):220-240. https://doi.org/10.1080/01969722.2014.894848S220240453Blanc , A. , Y.K. Liu , and A. Vahdat . “Designing Incentives for Peer-to-Peer Routing.” InProceedings of the 24th Annual Joint Conference of the IEEE Computer and Communications Societies, Vol. 1, pp. 374–385, 2005 .del Val , E. “Semantic Service Management in Service-Oriented Multi-Agent Systems.” Ph.D. thesis, Departament de Sistemes Informàtics i Computació, Universitat Politècnica de València, 2013 .Del Val, E., Rebollo, M., & Botti, V. (2012). Enhancing decentralized service discovery in open service-oriented multi-agent systems. Autonomous Agents and Multi-Agent Systems, 28(1), 1-30. doi:10.1007/s10458-012-9210-0DORAN, J. E., FRANKLIN, S., JENNINGS, N. R., & NORMAN, T. J. (1997). On cooperation in multi-agent systems. The Knowledge Engineering Review, 12(3), 309-314. doi:10.1017/s0269888997003111Eguíluz, V. M., Zimmermann, M. G., Cela‐Conde, C. J., & Miguel, M. S. (2005). Cooperation and the Emergence of Role Differentiation in the Dynamics of Social Networks. American Journal of Sociology, 110(4), 977-1008. doi:10.1086/428716Griffiths , N. and M. Luck . “Changing Neighbours: Improving Tag-Based Cooperation.” InProceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1.(AAMAS'10), 249–256. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems, 2010 .Gu , B. and S. Jarvenpaa . “Are Contributions to p2p Technical Forums Private or Public Goods? An Empirical Investigation.” Paper presented at the 1st Workshop on Economics of Peer-to-Peer Systems, June 4–5, 2004, Harvard University .Hauert, C., Traulsen, A., Brandt, H., Nowak, M. A., & Sigmund, K. (2007). Via Freedom to Coercion: The Emergence of Costly Punishment. Science, 316(5833), 1905-1907. doi:10.1126/science.1141588Hofmann , L.M. , N. Chakraborty , and K. Sycara . “The Evolution of Cooperation in Self-Interested Agent Societies: A Critical Study.” InProceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems, Volume 2 , edited by K. Tumer , P. Yolum , L. Sonenberg , and P. Stone , 685–692. IFAAMAS, 2011 .Lin, W. S., Zhao, H. V., & Liu, K. J. R. (2009). Incentive Cooperation Strategies for Peer-to-Peer Live Multimedia Streaming Social Networks. IEEE Transactions on Multimedia, 11(3), 396-412. doi:10.1109/tmm.2009.2012915Nowak, M. A. (2006). Five Rules for the Evolution of Cooperation. Science, 314(5805), 1560-1563. doi:10.1126/science.1133755Nowak, M. A., & Sigmund, K. (1998). Evolution of indirect reciprocity by image scoring. Nature, 393(6685), 573-577. doi:10.1038/31225Ohtsuki, H., Hauert, C., Lieberman, E., & Nowak, M. A. (2006). A simple rule for the evolution of cooperation on graphs and social networks. Nature, 441(7092), 502-505. doi:10.1038/nature04605Santos, F. C., Santos, M. D., & Pacheco, J. M. (2008). Social diversity promotes the emergence of cooperation in public goods games. Nature, 454(7201), 213-216. doi:10.1038/nature06940Shneidman , J. and D. C. Parkes . “Rationality and Self-Interest in Peer to Peer Networks.” Paper presented at the 2nd Int. Workshop on Peer-to-Peer Systems (IPTPS’03), February 20–21, 2003, Berkeley, CA .Sigmund, K. (2007). Punish or perish? Retaliation and collaboration among humans. Trends in Ecology & Evolution, 22(11), 593-600. doi:10.1016/j.tree.2007.06.012Sigmund, K. (2009). Sympathy and similarity: The evolutionary dynamics of cooperation. Proceedings of the National Academy of Sciences, 106(21), 8405-8406. doi:10.1073/pnas.0903947106Sigmund, K., Hauert, C., & Nowak, M. A. (2001). Reward and punishment. Proceedings of the National Academy of Sciences, 98(19), 10757-10762. doi:10.1073/pnas.161155698Sun , Q. and H. Garcia-Molina . “Slic: A Selfish Link-Based Incentive Mechanism for Unstructured Peer-To-Peer Networks.” Paper presented at the 24th International Conference on Distributed Computing Systems (ICDCS’04), March 23–26, 2004, Washington, DC .Villatoro , D. , J. Sabater-Mir , and S. Sen . “Social Instruments for Robust Convention Emergence.”Proceedings of the International Joint Conference on Artificial Intelligence, edited by T. Walsh, 420–425, 2011

Multi-level agent-based modeling - A literature survey

During last decade, multi-level agent-based modeling has received significant and dramatically increasing interest. In this article we present a comprehensive and structured review of literature on the subject. We present the main theoretical contributions and application domains of this concept, with an emphasis on social, flow, biological and biomedical models.Comment: v2. Ref 102 added. v3-4 Many refs and text added v5-6 bibliographic statistics updated. v7 Change of the name of the paper to reflect what it became, many refs and text added, bibliographic statistics update

Decentralized Multi-Subgroup Formation Control With Connectivity Preservation and Collision Avoidance

This paper proposes a formation control algorithm to create separated multiple formations for an undirected networked multi-agent system while preserving the network connectivity and avoiding collision among agents. Through the modified multi-consensus technique, the proposed algorithm can simultaneously divide a group of multiple agents into any arbitrary number of desired formations in a decentralized manner. Furthermore, the agents assigned to each formation group can be easily reallocated to other formation groups without network topological constraints as long as the entire network is initially connected; an operator can freely partition agents even if there is no spanning tree within each subgroup. Besides, the system can avoid collision without loosing the connectivity even during the transient period of formation by applying the existing potential function based on the network connectivity estimation. If the estimation is correct, the potential function not only guarantees the connectivity maintenance but also allows some extra edges to be broken if the network remains connected. Numerical simulations are performed to verify the feasibility and performance of the proposed multi-subgroup formation control

Hybrid automata dicretising agents for formal modelling of robots

Some of the fundamental capabilities required by autonomous vehicles and systems for their intelligent decision making are: modelling of the environment and forming data abstractions for symbolic, logic based reasoning. The paper formulates a discrete agent framework that abstracts and controls a hybrid system that is a composition of hybrid automata modelled continuous individual processes. Theoretical foundations are laid down for a class of general model composition agents (MCAs) with an advanced subclass of rational physical agents (RPAs). We define MCAs as the most basic structures for the description of complex autonomous robotic systems. The RPA’s have logic based decision making that is obtained by an extension of the hybrid systems concepts using a set of abstractions. The theory presented helps the creation of robots with reliable performance and safe operation in their environment. The paper emphasizes the abstraction aspects of the overall hybrid system that emerges from parallel composition of sets of RPAs and MCAs

Acceptance conditions in automated negotiation

In every negotiation with a deadline, one of the negotiating parties has to accept an offer to avoid a break off. A break off is usually an undesirable outcome for both parties, therefore it is important that a negotiator employs a proficient mechanism to decide under which conditions to accept. When designing such conditions one is faced with the acceptance dilemma: accepting the current offer may be suboptimal, as better offers may still be presented. On the other hand, accepting too late may prevent an agreement from being reached, resulting in a break off with no gain for either party. Motivated by the challenges of bilateral negotiations between automated agents and by the results and insights of the automated negotiating agents competition (ANAC), we classify and compare state-of-the-art generic acceptance conditions. We focus on decoupled acceptance conditions, i.e. conditions that do not depend on the bidding strategy that is used. We performed extensive experiments to compare the performance of acceptance conditions in combination with a broad range of bidding strategies and negotiation domains. Furthermore we propose new acceptance conditions and we demonstrate that they outperform the other conditions that we study. In particular, it is shown that they outperform the standard acceptance condition of comparing the current offer with the offer the agent is ready to send out. We also provide insight in to why some conditions work better than others and investigate correlations between the properties of the negotiation environment and the efficacy of acceptance condition