On environment difficulty and discriminating power

The final publication is available at Springer via http://dx.doi.org/10.1007/s10458-014-9257-1This paper presents a way to estimate the difficulty and discriminating power of any task instance. We focus on a very general setting for tasks: interactive (possibly multiagent) environments where an agent acts upon observations and rewards. Instead of analysing the complexity of the environment, the state space or the actions that are performed by the agent, we analyse the performance of a population of agent policies against the task, leading to a distribution that is examined in terms of policy complexity. This distribution is then sliced by the algorithmic complexity of the policy and analysed through several diagrams and indicators. The notion of environment response curve is also introduced, by inverting the performance results into an ability scale. We apply all these concepts, diagrams and indicators to two illustrative problems: a class of agent-populated elementary cellular automata, showing how the difficulty and discriminating power may vary for several environments, and a multiagent system, where agents can become predators or preys, and may need to coordinate. Finally, we discuss how these tools can be applied to characterise (interactive) tasks and (multi-agent) environments. These characterisations can then be used to get more insight about agent performance and to facilitate the development of adaptive tests for the evaluation of agent abilities.I thank the reviewers for their comments, especially those aiming at a clearer connection with the field of multi-agent systems and the suggestion of better approximations for the calculation of the response curves. The implementation of the elementary cellular automata used in the environments is based on the library 'CellularAutomaton' by John Hughes for R [58]. I am grateful to Fernando Soler-Toscano for letting me know about their work [65] on the complexity of 2D objects generated by elementary cellular automata. I would also like to thank David L. Dowe for his comments on a previous version of this paper. This work was supported by the MEC/MINECO projects CONSOLIDER-INGENIO CSD2007-00022 and TIN 2010-21062-C02-02, GVA project PROMETEO/2008/051, the COST - European Cooperation in the field of Scientific and Technical Research IC0801 AT, and the REFRAME project, granted by the European Coordinated Research on Long-term Challenges in Information and Communication Sciences & Technologies ERA-Net (CHIST-ERA), and funded by the Ministerio de Economia y Competitividad in Spain (PCIN-2013-037).José Hernández-Orallo (2015). On environment difficulty and discriminating power. Autonomous Agents and Multi-Agent Systems. 29(3):402-454. https://doi.org/10.1007/s10458-014-9257-1S402454293Anderson, J., Baltes, J., & Cheng, C. T. (2011). Robotics competitions as benchmarks for ai research. The Knowledge Engineering Review, 26(01), 11–17.Andre, D., & Russell, S. J. (2002). State abstraction for programmable reinforcement learning agents. In Proceedings of the National Conference on Artificial Intelligence (pp. 119–125). Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.Antunes, L., Fortnow, L., van Melkebeek, D., & Vinodchandran, N. V. (2006). Computational depth: Concept and applications. Theoretical Computer Science, 354(3), 391–404. Foundations of Computation Theory (FCT 2003), 14th Symposium on Fundamentals of Computation Theory 2003.Arai, K., Kaminka, G. A., Frank, I., & Tanaka-Ishii, K. (2003). Performance competitions as research infrastructure: Large scale comparative studies of multi-agent teams. Autonomous Agents and Multi-Agent Systems, 7(1–2), 121–144.Ashcraft, M. H., Donley, R. D., Halas, M. A., & Vakali, M. (1992). Chapter 8 working memory, automaticity, and problem difficulty. In Jamie I.D. Campbell (Ed.), The nature and origins of mathematical skills, volume 91 of advances in psychology (pp. 301–329). North-Holland.Ay, N., Müller, M., & Szkola, A. (2010). Effective complexity and its relation to logical depth. IEEE Transactions on Information Theory, 56(9), 4593–4607.Barch, D. M., Braver, T. S., Nystrom, L. E., Forman, S. D., Noll, D. C., & Cohen, J. D. (1997). Dissociating working memory from task difficulty in human prefrontal cortex. Neuropsychologia, 35(10), 1373–1380.Bordini, R. H., Hübner, J. F., & Wooldridge, M. (2007). Programming multi-agent systems in AgentSpeak using Jason. London: Wiley. com.Boutilier, C., Reiter, R., Soutchanski, M., Thrun, S. et al. (2000). Decision-theoretic, high-level agent programming in the situation calculus. In Proceedings of the National Conference on Artificial Intelligence (pp. 355–362). Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.Busoniu, L., Babuska, R., & De Schutter, B. (2008). A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 38(2), 156–172.Chaitin, G. J. (1977). Algorithmic information theory. IBM Journal of Research and Development, 21, 350–359.Chedid, F. B. (2010). Sophistication and logical depth revisited. In 2010 IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) (pp. 1–4). IEEE.Cheeseman, P., Kanefsky, B. & Taylor, W. M. (1991). Where the really hard problems are. In Proceedings of IJCAI-1991 (pp. 331–337).Dastani, M. (2008). 2APL: A practical agent programming language. Autonomous Agents and Multi-agent Systems, 16(3), 214–248.Delahaye, J. P. & Zenil, H. (2011). Numerical evaluation of algorithmic complexity for short strings: A glance into the innermost structure of randomness. Applied Mathematics and Computation, 219(1), 63–77Dowe, D. L. (2008). Foreword re C. S. Wallace. Computer Journal, 51(5), 523–560. Christopher Stewart WALLACE (1933–2004) memorial special issue.Dowe, D. L., & Hernández-Orallo, J. (2012). IQ tests are not for machines, yet. Intelligence, 40(2), 77–81.Du, D. Z., & Ko, K. I. (2011). Theory of computational complexity (Vol. 58). London: Wiley-Interscience.Elo, A. E. (1978). The rating of chessplayers, past and present (Vol. 3). London: Batsford.Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. London: Lawrence Erlbaum.Fatès, N. & Chevrier, V. (2010). How important are updating schemes in multi-agent systems? an illustration on a multi-turmite model. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1-Volume 1 (pp. 533–540). International Foundation for Autonomous Agents and Multiagent Systems.Ferber, J. & Müller, J. P. (1996). Influences and reaction: A model of situated multiagent systems. In Proceedings of Second International Conference on Multi-Agent Systems (ICMAS-96) (pp. 72–79).Ferrando, P. J. (2009). Difficulty, discrimination, and information indices in the linear factor analysis model for continuous item responses. Applied Psychological Measurement, 33(1), 9–24.Ferrando, P. J. (2012). Assessing the discriminating power of item and test scores in the linear factor-analysis model. Psicológica, 33, 111–139.Gent, I. P., & Walsh, T. (1994). Easy problems are sometimes hard. Artificial Intelligence, 70(1), 335–345.Gershenson, C. & Fernandez, N. (2012). Complexity and information: Measuring emergence, self-organization, and homeostasis at multiple scales. Complexity, 18(2), 29–44.Gruner, S. (2010). Mobile agent systems and cellular automata. Autonomous Agents and Multi-agent Systems, 20(2), 198–233.Hardman, D. K., & Payne, S. J. (1995). Problem difficulty and response format in syllogistic reasoning. The Quarterly Journal of Experimental Psychology, 48(4), 945–975.He, J., Reeves, C., Witt, C., & Yao, X. (2007). A note on problem difficulty measures in black-box optimization: Classification, realizations and predictability. Evolutionary Computation, 15(4), 435–443.Hernández-Orallo, J. (2000). Beyond the turing test. Journal of Logic Language & Information, 9(4), 447–466.Hernández-Orallo, J. (2000). On the computational measurement of intelligence factors. In A. Meystel (Ed.), Performance metrics for intelligent systems workshop (pp. 1–8). Gaithersburg, MD: National Institute of Standards and Technology.Hernández-Orallo, J. (2000). Thesis: Computational measures of information gain and reinforcement in inference processes. AI Communications, 13(1), 49–50.Hernández-Orallo, J. (2010). A (hopefully) non-biased universal environment class for measuring intelligence of biological and artificial systems. In M. Hutter et al. (Ed.), 3rd International Conference on Artificial General Intelligence (pp. 182–183). Atlantis Press Extended report at http://users.dsic.upv.es/proy/anynt/unbiased.pdf .Hernández-Orallo, J., & Dowe, D. L. (2010). Measuring universal intelligence: Towards an anytime intelligence test. Artificial Intelligence, 174(18), 1508–1539.Hernández-Orallo, J., Dowe, D. L., España-Cubillo, S., Hernández-Lloreda, M. V., & Insa-Cabrera, J. (2011). On more realistic environment distributions for defining, evaluating and developing intelligence. In J. Schmidhuber, K. R. Thórisson, & M. Looks (Eds.), LNAI series on artificial general intelligence 2011 (Vol. 6830, pp. 82–91). Berlin: Springer.Hernández-Orallo, J., Dowe, D. L., & Hernández-Lloreda, M. V. (2014). Universal psychometrics: Measuring cognitive abilities in the machine kingdom. Cognitive Systems Research, 27, 50–74.Hernández-Orallo, J., Insa, J., Dowe, D. L. & Hibbard, B. (2012). Turing tests with turing machines. In A. Voronkov (Ed.), The Alan Turing Centenary Conference, Turing-100, Manchester, 2012, volume 10 of EPiC Series (pp. 140–156).Hernández-Orallo, J. & Minaya-Collado, N. (1998). A formal definition of intelligence based on an intensional variant of Kolmogorov complexity. In Proceedings of International Symposium of Engineering of Intelligent Systems (EIS’98) (pp. 146–163). ICSC Press.Hibbard, B. (2009). Bias and no free lunch in formal measures of intelligence. Journal of Artificial General Intelligence, 1(1), 54–61.Hoos, H. H. (1999). Sat-encodings, search space structure, and local search performance. In 1999 International Joint Conference on Artificial Intelligence (Vol. 16, pp. 296–303).Insa-Cabrera, J., Benacloch-Ayuso, J. L., & Hernández-Orallo, J. (2012). On measuring social intelligence: Experiments on competition and cooperation. In J. Bach, B. Goertzel, & M. Iklé (Eds.), AGI, volume 7716 of lecture notes in computer science (pp. 126–135). Berlin: Springer.Insa-Cabrera, J., Dowe, D. L., España-Cubillo, S., Hernández-Lloreda, M. V., & Hernández-Orallo, J. (2011). Comparing humans and AI agents. In J. Schmidhuber, K. R. Thórisson, & M. Looks (Eds.), LNAI series on artificial general intelligence 2011 (Vol. 6830, pp. 122–132). Berlin: Springer.Knuth, D. E. (1973). Sorting and searching, volume 3 of the art of computer programming. Reading, MA: Addison-Wesley.Kotovsky, K., & Simon, H. A. (1990). What makes some problems really hard: Explorations in the problem space of difficulty. Cognitive Psychology, 22(2), 143–183.Legg, S. (2008). Machine super intelligence. PhD thesis, Department of Informatics, University of Lugano, June 2008.Legg, S., & Hutter, M. (2007). Universal intelligence: A definition of machine intelligence. Minds and Machines, 17(4), 391–444.Leonetti, M. & Iocchi, L. (2010). Improving the performance of complex agent plans through reinforcement learning. In Proceedings of the 2010 International Conference on Autonomous Agents and Multiagent Systems (Vol. 1, pp. 723–730). International Foundation for Autonomous Agents and Multiagent Systems.Levin, L. A. (1973). Universal sequential search problems. Problems of Information Transmission, 9(3), 265–266.Levin, L. A. (1986). Average case complete problems. SIAM Journal on Computing, 15, 285.Li, M., & Vitányi, P. (2008). An introduction to Kolmogorov complexity and its applications (3rd ed.). Berlin: Springer.Low, C. K., Chen, T. Y., & Rónnquist, R. (1999). Automated test case generation for bdi agents. Autonomous Agents and Multi-agent Systems, 2(4), 311–332.Madden, M. G., & Howley, T. (2004). Transfer of experience between reinforcement learning environments with progressive difficulty. Artificial Intelligence Review, 21(3), 375–398.Mellenbergh, G. J. (1994). Generalized linear item response theory. Psychological Bulletin, 115(2), 300.Michel, F. (2004). Formalisme, outils et éléments méthodologiques pour la modélisation et la simulation multi-agents. PhD thesis, Université des sciences et techniques du Languedoc, Montpellier.Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81.Orponen, P., Ko, K. I., Schöning, U., & Watanabe, O. (1994). Instance complexity. Journal of the ACM (JACM), 41(1), 96–121.Simon, H. A., & Kotovsky, K. (1963). Human acquisition of concepts for sequential patterns. Psychological Review, 70(6), 534.Team, R., et al. (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.Whiteson, S., Tanner, B., & White, A. (2010). The reinforcement learning competitions. The AI Magazine, 31(2), 81–94.Wiering, M., & van Otterlo, M. (Eds.). (2012). Reinforcement learning: State-of-the-art. Berlin: Springer.Wolfram, S. (2002). A new kind of science. Champaign, IL: Wolfram Media.Zatuchna, Z., & Bagnall, A. (2009). Learning mazes with aliasing states: An LCS algorithm with associative perception. Adaptive Behavior, 17(1), 28–57.Zenil, H. (2010). Compression-based investigation of the dynamical properties of cellular automata and other systems. Complex Systems, 19(1), 1–28.Zenil, H. (2011). Une approche expérimentale à la théorie algorithmique de la complexité. PhD thesis, Dissertation in fulfilment of the degree of Doctor in Computer Science, Université de Lille.Zenil, H., Soler-Toscano, F., Delahaye, J. P. & Gauvrit, N. (2012). Two-dimensional kolmogorov complexity and validation of the coding theorem method by compressibility. arXiv, preprint arXiv:1212.6745

Double blind randomized placebo-controlled trial on the effects of testosterone supplementation in elderly men with moderate to low testosterone levels: design and baseline characteristics [ISRCTN23688581]

In ageing men testosterone levels decline, while cognitive function, muscle and bone mass, sexual hair growth, libido and sexual activity decline and the risk of cardiovascular diseases increase. We set up a double-blind, randomized placebo-controlled trial to investigate the effects of testosterone supplementation on functional mobility, quality of life, body composition, cognitive function, vascular function and risk factors, and bone mineral density in older hypogonadal men. We recruited 237 men with serum testosterone levels below 13.7 nmol/L and ages 60–80 years. They were randomized to either four capsules of 40 mg testosterone undecanoate (TU) or placebo daily for 26 weeks. Primary endpoints are functional mobility and quality of life. Secondary endpoints are body composition, cognitive function, aortic stiffness and cardiovascular risk factors and bone mineral density. Effects on prostate, liver and hematological parameters will be studied with respect to safety. Measure of effect will be the difference in change from baseline visit to final visit between TU and placebo. We will study whether the effect of TU differs across subgroups of baseline waist girth (< 100 cm vs. ≥ 100 cm; testosterone level (<12 versus ≥ 12 nmol/L), age (< median versus ≥ median), and level of outcome under study (< median versus ≥ median). At baseline, mean age, BMI and testosterone levels were 67 years, 27 kg/m(2 )and 10.72 nmol/L, respectively

Differences in Muscle Protein Synthesis and Anabolic Signaling in the Postabsorptive State and in Response to Food in 65–80 Year Old Men and Women

Women have less muscle than men but lose it more slowly during aging. To discover potential underlying mechanism(s) for this we evaluated the muscle protein synthesis process in postabsorptive conditions and during feeding in twenty-nine 65–80 year old men (n = 13) and women (n = 16). We discovered that the basal concentration of phosphorylated eEF2Thr56 was ∼40% less (P<0.05) and the basal rate of MPS was ∼30% greater (P = 0.02) in women than in men; the basal concentrations of muscle phosphorylated AktThr308, p70s6kThr389, eIF4ESer209, and eIF4E-BP1Thr37/46 were not different between the sexes. Feeding increased (P<0.05) AktThr308 and p70s6kThr389 phosphorylation to the same extent in men and women but increased (P<0.05) the phosphorylation of eIF4ESer209 and eIF4E-BP1Thr37/46 in men only. Accordingly, feeding increased MPS in men (P<0.01) but not in women. The postabsorptive muscle mRNA concentrations for myoD and myostatin were not different between sexes; feeding doubled myoD mRNA (P<0.05) and halved that of myostatin (P<0.05) in both sexes. Thus, there is sexual dimorphism in MPS and its control in older adults; a greater basal rate of MPS, operating over most of the day may partially explain the slower loss of muscle in older women

Differences in Muscle Protein Synthesis and Anabolic Signaling in the Postabsorptive State and in Response to Food in 65–80 Year Old Men and Women

Women have less muscle than men but lose it more slowly during aging. To discover potential underlying mechanism(s) for this we evaluated the muscle protein synthesis process in postabsorptive conditions and during feeding in twenty-nine 65–80 year old men (n = 13) and women (n = 16). We discovered that the basal concentration of phosphorylated eEF2Thr56 was ∼40% less (P<0.05) and the basal rate of MPS was ∼30% greater (P = 0.02) in women than in men; the basal concentrations of muscle phosphorylated AktThr308, p70s6kThr389, eIF4ESer209, and eIF4E-BP1Thr37/46 were not different between the sexes. Feeding increased (P<0.05) AktThr308 and p70s6kThr389 phosphorylation to the same extent in men and women but increased (P<0.05) the phosphorylation of eIF4ESer209 and eIF4E-BP1Thr37/46 in men only. Accordingly, feeding increased MPS in men (P<0.01) but not in women. The postabsorptive muscle mRNA concentrations for myoD and myostatin were not different between sexes; feeding doubled myoD mRNA (P<0.05) and halved that of myostatin (P<0.05) in both sexes. Thus, there is sexual dimorphism in MPS and its control in older adults; a greater basal rate of MPS, operating over most of the day may partially explain the slower loss of muscle in older women

Fibre Distribution and the Process-Property Dilemma

The options for the fibre reinforcement of polymer matrix composites cover a range from short-fibre chopped strand mat, through woven fabric to unidirectional pre-impregnated (prepreg) reinforcements. The modelling of such materials may be simplified by assumptions such as perfect regular packing of fibres and the total absence of fibre waviness. However, these and other features such as the crimp or waviness in woven fabrics make real materials more complex than the simplified models. Clustering of fibres creates fibre-rich and resin-rich volumes (FRV and RRV respectively) in the composites. Prior to impregnation, large RRV will be pore-space that can expedite the flow of resin in liquid composite moulding processes (especially resin transfer moulding (RTM) and resin infusion under flexible tooling (RIFT). In the composite, the clustering of fibres tends to reduce the mechanical properties. The use of image processing and analysis can permit micro-/meso-structural characterisation which may correlate to the respective properties. This chapter considers the quantification of microstructure images in the context of the process-property dilemma for woven carbon-fibre reinforced composites with the aim of increasing understanding of the balance between processability and mechanical performance

The dependence of dijet production on photon virtuality in ep collisions at HERA

The dependence of dijet production on the virtuality of the exchanged photon, Q^2, has been studied by measuring dijet cross sections in the range 0 < Q^2 < 2000 GeV^2 with the ZEUS detector at HERA using an integrated luminosity of 38.6 pb^-1. Dijet cross sections were measured for jets with transverse energy E_T^jet > 7.5 and 6.5 GeV and pseudorapidities in the photon-proton centre-of-mass frame in the range -3 < eta^jet <0. The variable xg^obs, a measure of the photon momentum entering the hard process, was used to enhance the sensitivity of the measurement to the photon structure. The Q^2 dependence of the ratio of low- to high-xg^obs events was measured. Next-to-leading-order QCD predictions were found to generally underestimate the low-xg^obs contribution relative to that at high xg^obs. Monte Carlo models based on leading-logarithmic parton-showers, using a partonic structure for the photon which falls smoothly with increasing Q^2, provide a qualitative description of the data.Comment: 35 pages, 6 eps figures, submitted to Eur.Phys.J.

Beauty photoproduction measured using decays into muons in dijet events in ep collisions at s\sqrt{s}=318 GeV

The photoproduction of beauty quarks in events with two jets and a muon has been measured with the ZEUS detector at HERA using an integrated luminosity of 110 pb$^{- 1}$. The fraction of jets containing b quarks was extracted from the transverse momentum distribution of the muon relative to the closest jet. Differential cross sections for beauty production as a function of the transverse momentum and pseudorapidity of the muon, of the associated jet and of $x_{\gamma}^{jets}$, the fraction of the photon's momentum participating in the hard process, are compared with MC models and QCD predictions made at next-to-leading order. The latter give a good description of the data.Comment: 32 pages, 6 tables, 7 figures Table 6 and Figure 7 revised September 200

Search for a narrow charmed baryonic state decaying to D^*+/- p^-/+ in ep collisions at HERA

A resonance search has been made in the D^*+/- p^-/+ invariant-mass spectrum with the ZEUS detector at HERA using an integrated luminosity of 126 pb^-1. The decay channels D^*+ -> D^0 pi^+_s -> (K^- pi^+) pi^+_s and D^*+ -> D^0 pi^+_s -> (K^- pi^+ pi^+ pi^-) pi^+_s (and the corresponding antiparticle decays) were used to identify D^*+/- mesons. No resonance structure was observed in the D^*+/- p^-/+ mass spectrum from more than 60000 reconstructed D^*+/- mesons. The results are not compatible with a report of the H1 Collaboration of a charmed pentaquark, Theta^0_c.Comment: 22 pages, 7 figures, 1 table; minor text revisions; 2 references adde

Measurement of the inclusive and dijet cross-sections of b-jets in pp collisions at sqrt(s) = 7 TeV with the ATLAS detector

The inclusive and dijet production cross-sections have been measured for jets containing b-hadrons (b-jets) in proton-proton collisions at a centre-of-mass energy of sqrt(s) = 7 TeV, using the ATLAS detector at the LHC. The measurements use data corresponding to an integrated luminosity of 34 pb^-1. The b-jets are identified using either a lifetime-based method, where secondary decay vertices of b-hadrons in jets are reconstructed using information from the tracking detectors, or a muon-based method where the presence of a muon is used to identify semileptonic decays of b-hadrons inside jets. The inclusive b-jet cross-section is measured as a function of transverse momentum in the range 20 < pT < 400 GeV and rapidity in the range |y| < 2.1. The bbbar-dijet cross-section is measured as a function of the dijet invariant mass in the range 110 < m_jj < 760 GeV, the azimuthal angle difference between the two jets and the angular variable chi in two dijet mass regions. The results are compared with next-to-leading-order QCD predictions. Good agreement is observed between the measured cross-sections and the predictions obtained using POWHEG + Pythia. MC@NLO + Herwig shows good agreement with the measured bbbar-dijet cross-section. However, it does not reproduce the measured inclusive cross-section well, particularly for central b-jets with large transverse momenta.Comment: 10 pages plus author list (21 pages total), 8 figures, 1 table, final version published in European Physical Journal