^(238)U-^(230)Th dating of a geomagnetic excursion in Quaternary basalts of the Albuquerque Volcanoes Field, New Mexico (USA)

We report the ^(238)U-^(230)Th age derived from samples from three basalt flows from the Albuquerque Volcanoes Field, central New Mexico (USA), interpreted to have erupted during one of the Bruhnes Chron geomagnetic field polarity events. Whole-rock samples and magnetite separates from two of the samples define an isochron with an age of 156±29 ka (2σ). It is now possible to determine the ages of geomagnetic excursions with reasonable precision, by the ^(238)U-^(230)Th method, using basalts erupted between possibly 25 and up to 250 ka. Although a strict correlation of the polarity event recorded by the Albuquerque lavas with other high amplitude field phenomena in the latest Quaternary is not yet possible, the Blake and the less well-documented Old Crow and Biwa I/Jamaica events are likely candidates

Magnetochronology of the Entire Chinle Formation (Norian Age) in a Scientific Drill Core From Petrified Forest National Park (Arizona, USA) and Implications for Regional and Global Correlations in the Late Triassic

Building on an earlier study that confirmed the stability of the 405‐kyr eccentricity climate cycle and the timing of the Newark‐Hartford astrochronostratigraphic polarity time scale back to 215 Ma, we extend the magnetochronology of the Late Triassic Chinle Formation to its basal unconformity in scientific drill core PFNP‐1A from Petrified Forest National Park (Arizona, USA). The 335‐m‐thick Chinle section is imprinted with paleomagnetic polarity zones PF1r to PF10n, which we correlate to chrons E17r to E9n (~209 to 224 Ma) of the Newark‐Hartford astrochronostratigraphic polarity time scale. A sediment accumulation rate of ~34 m/Myr can be extended down to ~270 m, close to the base of the Sonsela Member and the base of magnetozone PF5n, which we correlate to chron E14n that onsets at 216.16 Ma. Magnetozones PF5r to PF10n in the underlying 65‐m‐thick section of the mudstone‐dominated Blue Mesa and Mesa Redondo members plausibly correlate to chrons E13r to E9n, indicating a sediment accumulation rate of only ~10 m/Myr. Published high‐precision U‐Pb detrital zircon dates from the lower Chinle tend to be several million years older than the magnetochronological age model. The source of this discrepancy is unclear but may be due to sporadic introduction of juvenile zircons that get recycled. The new magnetochronological constraint on the base of the Sonsela Member brings the apparent timing of the included Adamanian‐ Revueltian land vertebrate faunal zone boundary and the Zone II to Zone III palynofloral transition closer to the temporal range of the ~215 Ma Manicouagan impact structure in Canada

The Advanced Composition Explorer

The Advanced Composition Explorer (ACE) was recently selected as one of two new Explorer‐class missions to be developed for launch during the mid‐1990’s ACE will observe particles of solar, interplanetary, interstellar, and galactic origins, spanning the energy range from that of the solar wind (∼1 keV/nucleon) to galactic cosmic ray energies (several hundred MeV/nucleon). Definitive studies will be made of the abundance of nearly all isotopes from H to Zn (1≤Z≤30), with exploratory isotope studies extending to Zr(Z=40). To accomplish this, the ACE payload includes six high‐resolution spectrometers, each designed to provide the optimum charge, mass, or charge‐state resolution in its particular energy range, and each having a geometry factor optimized for the expected flux levels, so as to provide a collecting power a factor of 10 to 1000 times greater than previous or planned experiments. The payload also includes several instruments of standard design that will monitor solar wind and magnetic field conditions and energetic H, He, and electron fluxes. We summarize here the scientific objectives, instrumentation, spacecraft, and mission approach that were defined for ACE during the Phase‐A study period

LA-ICPMS U-Pb geochronology of detrital zircon grains from the Coconino, Moenkopi, and Chinle Formations in the Petrified Forest National Park (Arizona)

Uranium–lead (U–Pb) geochronology was conducted by laser ablation – inductively coupled plasma mass spectrometry (LA-ICPMS) on 7175 detrital zircon grains from 29 samples from the Coconino Sandstone, Moenkopi Formation, and Chinle Formation. These samples were recovered from ∼ 520 m of drill core that was acquired during the Colorado Plateau Coring Project (CPCP), located in Petrified Forest National Park (Arizona). A sample from the lower Permian Coconino Sandstone yields a broad distribution of Proterozoic and Paleozoic ages that are consistent with derivation from the Appalachian and Ouachita orogens, with little input from local basement or Ancestral Rocky Mountain sources. Four samples from the Holbrook Member of the Moenkopi Formation yield a different set of Precambrian and Paleozoic age groups, indicating derivation from the Ouachita orogen, the East Mexico arc, and the Permo-Triassic arc built along the Cordilleran margin. A total of 23 samples from the Chinle Formation contain variable proportions of Proterozoic and Paleozoic zircon grains but are dominated by Late Triassic grains. LA-ICPMS ages of these grains belong to five main groups that correspond to the Mesa Redondo Member, Blue Mesa Member and lower part of the Sonsela Member, upper part of the Sonsela Member, middle part of the Petrified Forest Member, and upper part of the Petrified Forest Member. The ages of pre-Triassic grains also correspond to these chronostratigraphic units and are interpreted to reflect varying contributions from the Appalachian orogen to the east, Ouachita orogen to the southeast, Precambrian basement exposed in the ancestral Mogollon Highlands to the south, East Mexico arc, and Permian–Triassic arc built along the southern Cordilleran margin. Triassic grains in each chronostratigraphic unit also have distinct U and thorium (Th) concentrations, which are interpreted to reflect temporal changes in the chemistry of arc magmatism. Comparison of our LA-ICPMS ages with available chemical abrasion thermal ionization mass spectrometry (CA-TIMS) ages and new magnetostratigraphic data provides new insights into the depositional history of the Chinle Formation, as well as methods utilized to determine depositional ages of fluvial strata. For parts of the Chinle Formation that are dominated by fine-grained clastic strata (e.g., mudstone and siltstone), such as the Blue Mesa Member and Petrified Forest Member, all three chronometers agree (to within ∼ 1 Myr), and robust depositional chronologies have been determined. In contrast, for stratigraphic intervals dominated by coarse-grained clastic strata (e.g., sandstone), such as most of the Sonsela Member, the three chronologic records disagree due to recycling of older zircon grains and variable dilution of syn-depositional-age grains. This results in LA-ICPMS ages that significantly predate deposition and CA-TIMS ages that range between the other two chronometers. These complications challenge attempts to establish a well-defined chronostratigraphic age model for the Chinle Formation

U-Pb zircon geochronology and depositional age models for the Upper Triassic Chinle Formation (Petrified Forest National Park, Arizona, USA): implications for Late Triassic paleoecological and paleoenvironmental change

The Upper Triassic Chinle Formation is a critical non-marine archive of low-paleolatitude biotic and environmental change in southwestern North America. The well-studied and highly fossiliferous Chinle strata at Petrified Forest National Park (PFNP), Arizona, preserve a biotic turnover event recorded by vertebrate and palynomorph fossils, which has been alternatively hypothesized to coincide with tectonically driven climate change or with the Manicouagan impact event at ca. 215.5 Ma. Previous outcrop-based geochronologic age constraints are difficult to put in an accurate stratigraphic framework because lateral facies changes and discontinuous outcrops allow for multiple interpretations. A major goal of the Colorado Plateau Coring Project (CPCP) was to retrieve a continuous record in unambiguous superposition designed to remedy this situation. We sampled the 520-m-long core 1A of the CPCP to develop an accurate age model in unquestionable superposition by combining U-Pb zircon ages and magnetostratigraphy. From 13 horizons of volcanic detritus-rich siltstone and sandstone, we screened up to ∼300 zircon crystals per sample using laser ablation−inductively coupled plasma−mass spectrometry and subsequently analyzed up to 19 crystals of the youngest age population using the chemical abrasion−isotope dilution−thermal ionization mass (CA-ID-TIMS) spectrometry method. These data provide new maximum depositional ages for the top of the Moenkopi Formation (ca. 241 Ma), the lower Blue Mesa Member (ca. 222 Ma), and the lower (ca. 218 to 217 Ma) and upper (ca. 213.5 Ma) Sonsela Member. The maximum depositional ages obtained for the upper Chinle Formation fall well within previously proposed age constraints, whereas the maximum depositional ages for the lower Chinle Formation are relatively younger than previously proposed ages from outcrop; however, core to outcrop stratigraphic correlations remain uncertain. By correlating our new ages with the magnetostratigraphy of the core, two feasible age model solutions can be proposed. Model 1 assumes that the youngest, coherent U-Pb age clusters of each sample are representative of the maximum depositional ages and are close to (227 Ma) in age, and hence the biotic turnover event cannot be correlated to the Carnian−Norian boundary but is rather a mid-Norian event. Our age models demonstrate the powers, but also the challenges, of integrating detrital CA-ID-TIMS ages with magnetostratigraphic data to properly interpret complex sedimentary sequences

Adenylyl Cyclase Plays a Regulatory Role in Development, Stress Resistance and Secondary Metabolism in Fusarium fujikuroi

The ascomycete fungus Fusarium fujikuroi (Gibberella fujikuroi MP-C) produces secondary metabolites of biotechnological interest, such as gibberellins, bikaverin, and carotenoids. Production of these metabolites is regulated by nitrogen availability and, in a specific manner, by other environmental signals, such as light in the case of the carotenoid pathway. A complex regulatory network controlling these processes is recently emerging from the alterations of metabolite production found through the mutation of different regulatory genes. Here we show the effect of the targeted mutation of the acyA gene of F. fujikuroi, coding for adenylyl cyclase. Mutants lacking the catalytic domain of the AcyA protein showed different phenotypic alterations, including reduced growth, enhanced production of unidentified red pigments, reduced production of gibberellins and partially derepressed carotenoid biosynthesis in the dark. The phenotype differs in some aspects from that of similar mutants of the close relatives F. proliferatum and F. verticillioides: contrary to what was observed in these species, ΔacyA mutants of F. fujikuroi showed enhanced sensitivity to oxidative stress (H2O2), but no change in heavy metal resistance or in the ability to colonize tomato tissue, indicating a high versatility in the regulatory roles played by cAMP in this fungal group

Colorado Plateau Coring Project, Phase I (CPCP-I): a continuously cored, globally exportable chronology of Triassic continental environmental change from western North America

Phase 1 of the Colorado Plateau Coring Project (CPCP-I) recovered a total of over 850&thinsp;m of stratigraphically overlapping core from three coreholes at two sites in the Early to Middle and Late Triassic age largely fluvial Moenkopi and Chinle formations in Petrified Forest National Park (PFNP), northeastern Arizona, USA. Coring took place during November and December of 2013 and the project is now in its post-drilling science phase. The CPCP cores have abundant detrital zircon-producing layers (with survey LA-ICP-MS dates selectively resampled for CA-ID-TIMS U-Pb ages ranging in age from at least 210 to 241&thinsp;Ma), which together with their magnetic polarity stratigraphy demonstrate that a globally exportable timescale can be produced from these continental sequences and in the process show that a prominent gap in the calibrated Phanerozoic record can be filled. The portion of core CPCP-PFNP13-1A for which the polarity stratigraphy has been completed thus far spans  ∼ 215 to 209&thinsp;Ma of the Late Triassic age, and strongly validates the longer Newark-Hartford Astrochronostratigraphic-calibrated magnetic Polarity Time-Scale (APTS) based on cores recovered in the 1990s during the Newark Basin Coring Project (NBCP).Core recovery was  ∼ 100&thinsp;% in all holes (Table 1). The coreholes were inclined  ∼ 60–75° approximately to the south to ensure azimuthal orientation in the nearly flat-lying bedding, critical to the interpretation of paleomagentic polarity stratigraphy. The two longest of the cores (CPCP-PFNP13-1A and 2B) were CT-scanned in their entirety at the University of Texas High Resolution X-ray CT Facility in Austin, TX, and subsequently along with 2A, all cores were split and processed at the CSDCO/LacCore Facility, in Minneapolis, MN, where they were scanned for physical property logs and imaging. While remaining the property of the Federal Government, the archive half of each core is curated at the NSF-sponsored LacCore Core Repository and the working half is stored at the Rutgers University Core Repository in Piscataway, NJ, where the initial sampling party was held in 2015 with several additional sampling events following. Additional planned study will recover the rest of the polarity stratigraphy of the cores as additional zircon ages, sedimentary structure and paleosol facies analysis, stable isotope geochemistry, and calibrated XRF core scanning are accomplished. Together with strategic outcrop studies in Petrified Forest National Park and environs, these cores will allow the vast amount of surface paleontological and paleoenvironmental information recorded in the continental Triassic of western North America to be confidently placed in a secure context along with important events such as the giant Manicouagan impact at  ∼ 215.5&thinsp;Ma (Ramezani et al., 2005) and long wavelength astronomical cycles pacing global environmental change and trends in atmospheric gas composition during the dawn of the dinosaurs.</p

Relations between hinterland and foreland shortening: Sevier orogeny, central North American Cordillera

This is the published version. Copyright 2000 American Geophysical Union. All Rights Reserved.The tectonic relations between foreland and hinterland deformation in noncollisional orogens are critical to understanding the overall development of orogens. The classic central Cordilleran foreland fold-and-thrust belt in the United States (Late Jurassic to early Tertiary Sevier belt) and the more internal zones to the west (central Nevada thrust belt) provide data critical to understanding the development of internal and external parts of orogens. The Garden Valley thrust system, part of the central Nevada thrust belt, crops out in south-central Nevada within a region generally considered to be the hinterland of the Jurassic to Eocene Sevier thrust belt. The thrust system consists of at least four principal thrust plates composed of strata as young as Pennsylvanian in age that are unconformably overlain by rocks as old as Oligocene, suggesting that contraction occurred between those times. New U/Pb dates on intrusions that postdate contraction, combined with new paleomagnetic data showing significant tilting of one area prior to intrusion, suggest that regionally these thrusts were active before ∼85–100 Ma. The thrust faults are characterized by long, relatively steeply dipping ramps and associated folds that are broad and open to close, upright and overturned. Although now fragmented by Cenozoic crustal extension, individual thrusts can be correlated from range to range for tens to hundreds of kilometers along strike. We correlate the structurally lowest thrust of the Garden Valley thrust system, the Golden Gate-Mount Irish thrust, southward with the Gass Peak thrust of southern Nevada. This correlation carries the following regional implications. At least some of the slip across Jurassic to mid-Cretaceous foreland thrusts in southern Nevada continues northward along the central Nevada thrust belt rather than northeastward into Utah. This continuation is consistent with age relations, which indicate that thrusts in the type Sevier belt in central Utah are synchronous with or younger than the youngest thrusts in southern Nevada. This in turn implies that geometrically similar Sevier belt thrusts in Utah must die out southward before they reach Nevada, that slip along the southern Nevada thrusts is partitioned between central Nevada and Utah thrusts, or that the Utah thrusts persist into southeastern Nevada but are located east of the longitude of the central Nevada thrust belt. As a result of overall cratonward migration of thrusting, the central Nevada thrust belt probably formed the Cordilleran foreland fold-thrust belt early in the shortening event but later lay in the hinterland of the Sevier fold-thrust belt of Idaho-Wyoming-Utah

Evaluation in artificial intelligence: From task-oriented to ability-oriented measurement

The final publication is available at Springer via http://dx.doi.org/ 10.1007/s10462-016-9505-7.The evaluation of artificial intelligence systems and components is crucial for the progress of the discipline. In this paper we describe and critically assess the different ways AI systems are evaluated, and the role of components and techniques in these systems. We first focus on the traditional task-oriented evaluation approach. We identify three kinds of evaluation: human discrimination, problem benchmarks and peer confrontation. We describe some of the limitations of the many evaluation schemes and competitions in these three categories, and follow the progression of some of these tests. We then focus on a less customary (and challenging) ability-oriented evaluation approach, where a system is characterised by its (cognitive) abilities, rather than by the tasks it is designed to solve. We discuss several possibilities: the adaptation of cognitive tests used for humans and animals, the development of tests derived from algorithmic information theory or more integrated approaches under the perspective of universal psychometrics. We analyse some evaluation tests from AI that are better positioned for an ability-oriented evaluation and discuss how their problems and limitations can possibly be addressed with some of the tools and ideas that appear within the paper. Finally, we enumerate a series of lessons learnt and generic guidelines to be used when an AI evaluation scheme is under consideration.I thank the organisers of the AEPIA Summer School On Artificial Intelligence, held in September 2014, for giving me the opportunity to give a lecture on 'AI Evaluation'. This paper was born out of and evolved through that lecture. The information about many benchmarks and competitions discussed in this paper have been contrasted with information from and discussions with many people: M. Bedia, A. Cangelosi, C. Dimitrakakis, I. GarcIa-Varea, Katja Hofmann, W. Langdon, E. Messina, S. Mueller, M. Siebers and C. Soares. Figure 4 is courtesy of F. Martinez-Plumed. Finally, I thank the anonymous reviewers, whose comments have helped to significantly improve the balance and coverage of the paper. This work has been partially supported by the EU (FEDER) and the Spanish MINECO under Grants TIN 2013-45732-C4-1-P, TIN 2015-69175-C4-1-R and by Generalitat Valenciana PROMETEOII2015/013.José Hernández-Orallo (2016). Evaluation in artificial intelligence: From task-oriented to ability-oriented measurement. Artificial Intelligence Review. 1-51. https://doi.org/10.1007/s10462-016-9505-7S151Abel D, Agarwal A, Diaz F, Krishnamurthy A, Schapire RE (2016) Exploratory gradient boosting for reinforcement learning in complex domains. arXiv preprint arXiv:1603.04119Adams S, Arel I, Bach J, Coop R, Furlan R, Goertzel B, Hall JS, Samsonovich A, Scheutz M, Schlesinger M, Shapiro SC, Sowa J (2012) Mapping the landscape of human-level artificial general intelligence. AI Mag 33(1):25–42Adams SS, Banavar G, Campbell M (2016) I-athlon: towards a multi-dimensional Turing test. AI Mag 37(1):78–84Alcalá J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2010) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult Valued Logic Soft Comput 17:255–287Alexander JRM, Smales S (1997) Intelligence, learning and long-term memory. Personal Individ Differ 23(5):815–825Alpcan T, Everitt T, Hutter M (2014) Can we measure the difficulty of an optimization problem? In: IEEE information theory workshop (ITW)Alur R, Bodik R, Juniwal G, Martin MMK, Raghothaman M, Seshia SA, Singh R, Solar-Lezama A, Torlak E, Udupa A (2013) Syntax-guided synthesis. In: Formal methods in computer-aided design (FMCAD), 2013, IEEE, pp 1–17Alvarado N, Adams SS, Burbeck S, Latta C (2002) Beyond the Turing test: performance metrics for evaluating a computer simulation of the human mind. In: Proceedings of the 2nd international conference on development and learning, IEEE, pp 147–152Amigoni F, Bastianelli E, Berghofer J, Bonarini A, Fontana G, Hochgeschwender N, Iocchi L, Kraetzschmar G, Lima P, Matteucci M, Miraldo P, Nardi D, Schiaffonati V (2015) Competitions for benchmarking: task and functionality scoring complete performance assessment. IEEE Robot Autom Mag 22(3):53–61Anderson J, Lebiere C (2003) The Newell test for a theory of cognition. Behav Brain Sci 26(5):587–601Anderson J, Baltes J, Cheng CT (2011) Robotics competitions as benchmarks for AI research. Knowl Eng Rev 26(01):11–17Arel I, Rose DC, Karnowski TP (2010) Deep machine learning—a new frontier in artificial intelligence research. IEEE Comput Intell Mag 5(4):13–18Asada M, Hosoda K, Kuniyoshi Y, Ishiguro H, Inui T, Yoshikawa Y, Ogino M, Yoshida C (2009) Cognitive developmental robotics: a survey. IEEE Trans Auton Ment Dev 1(1):12–34Aziz H, Brill M, Fischer F, Harrenstein P, Lang J, Seedig HG (2015) Possible and necessary winners of partial tournaments. J Artif Intell Res 54:493–534Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/mlBagnall AJ, Zatuchna ZV (2005) On the classification of maze problems. In: Bull L, Kovacs T (eds) Foundations of learning classifier system. Studies in fuzziness and soft computing, vol. 183, Springer, pp 305–316. http://rd.springer.com/chapter/10.1007/11319122_12Baldwin D, Yadav SB (1995) The process of research investigations in artificial intelligence - a unified view. IEEE Trans Syst Man Cybern 25(5):852–861Bellemare MG, Naddaf Y, Veness J, Bowling M (2013) The arcade learning environment: an evaluation platform for general agents. J Artif Intell Res 47:253–279Besold TR (2014) A note on chances and limitations of psychometric ai. In: KI 2014: advances in artificial intelligence. Springer, pp 49–54Biever C (2011) Ultimate IQ: one test to rule them all. New Sci 211(2829, 10 September 2011):42–45Borg M, Johansen SS, Thomsen DL, Kraus M (2012) Practical implementation of a graphics Turing test. In: Advances in visual computing. Springer, pp 305–313Boring EG (1923) Intelligence as the tests test it. New Repub 35–37Bostrom N (2014) Superintelligence: paths, dangers, strategies. Oxford University Press, OxfordBrazdil P, Carrier CG, Soares C, Vilalta R (2008) Metalearning: applications to data mining. Springer, New YorkBringsjord S (2011) Psychometric artificial intelligence. J Exp Theor Artif Intell 23(3):271–277Bringsjord S, Schimanski B (2003) What is artificial intelligence? Psychometric AI as an answer. In: International joint conference on artificial intelligence, pp 887–893Brundage M (2016) Modeling progress in ai. AAAI 2016 Workshop on AI, Ethics, and SocietyBuchanan BG (1988) Artificial intelligence as an experimental science. Springer, New YorkBuhrmester M, Kwang T, Gosling SD (2011) Amazon’s mechanical turk a new source of inexpensive, yet high-quality, data? Perspect Psychol Sci 6(1):3–5Bursztein E, Aigrain J, Moscicki A, Mitchell JC (2014) The end is nigh: generic solving of text-based captchas. In: Proceedings of the 8th USENIX conference on Offensive Technologies, USENIX Association, p 3Campbell M, Hoane AJ, Hsu F (2002) Deep Blue. Artif Intell 134(1–2):57–83Cangelosi A, Schlesinger M, Smith LB (2015) Developmental robotics: from babies to robots. MIT Press, CambridgeCaputo B, Müller H, Martinez-Gomez J, Villegas M, Acar B, Patricia N, Marvasti N, Üsküdarlı S, Paredes R, Cazorla M et al (2014) Imageclef 2014: overview and analysis of the results. In: Information access evaluation. Multilinguality, multimodality, and interaction, Springer, pp 192–211Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka ER Jr, Mitchell TM (2010) Toward an architecture for never-ending language learning. In: AAAI, vol 5, p 3Carroll JB (1993) Human cognitive abilities: a survey of factor-analytic studies. Cambridge University Press, CambridgeCaruana R (1997) Multitask learning. Mach Learn 28(1):41–75Chaitin GJ (1982) Gödel’s theorem and information. Int J Theor Phys 21(12):941–954Chandrasekaran B (1990) What kind of information processing is intelligence? In: The foundation of artificial intelligence—a sourcebook. Cambridge University Press, pp 14–46Chater N (1999) The search for simplicity: a fundamental cognitive principle? Q J Exp Psychol Sect A 52(2):273–302Chater N, Vitányi P (2003) Simplicity: a unifying principle in cognitive science? Trends Cogn Sci 7(1):19–22Chu Z, Gianvecchio S, Wang H, Jajodia S (2010) Who is tweeting on twitter: human, bot, or cyborg? In: Proceedings of the 26th annual computer security applications conference, ACM, pp 21–30Cochran WG (2007) Sampling techniques. Wiley, New YorkCohen PR, Howe AE (1988) How evaluation guides AI research: the message still counts more than the medium. AI Mag 9(4):35Cohen Y (2013) Testing and cognitive enhancement. Technical repor, National Institute for Testing and Evaluation, Jerusalem, IsraelConrad JG, Zeleznikow J (2013) The significance of evaluation in AI and law: a case study re-examining ICAIL proceedings. In: Proceedings of the 14th international conference on artificial intelligence and law, ACM, pp 186–191Conrad JG, Zeleznikow J (2015) The role of evaluation in ai and law. In: Proceedings of the 15th international conference on artificial intelligence and law, pp 181–186Deary IJ, Der G, Ford G (2001) Reaction times and intelligence differences: a population-based cohort study. Intelligence 29(5):389–399Decker KS, Durfee EH, Lesser VR (1989) Evaluating research in cooperative distributed problem solving. Distrib Artif Intell 2:487–519Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30Detterman DK (2011) A challenge to Watson. Intelligence 39(2–3):77–78Dimitrakakis C (2016) Personal communicationDimitrakakis C, Li G, Tziortziotis N (2014) The reinforcement learning competition 2014. AI Mag 35(3):61–65Dowe DL (2013) Introduction to Ray Solomonoff 85th memorial conference. In: Dowe DL (ed) Algorithmic probability and friends. Bayesian prediction and artificial intelligence, lecture notes in computer science, vol 7070. Springer, Berlin, pp 1–36Dowe DL, Hajek AR (1997) A computational extension to the Turing Test. In: Proceedings of the 4th conference of the Australasian cognitive science society, University of Newcastle, NSW, AustraliaDowe DL, Hajek AR (1998) A non-behavioural, computational extension to the Turing test. In: International conference on computational intelligence and multimedia applications (ICCIMA’98), Gippsland, Australia, pp 101–106Dowe DL, Hernández-Orallo J (2012) IQ tests are not for machines, yet. Intelligence 40(2):77–81Dowe DL, Hernández-Orallo J (2014) How universal can an intelligence test be? Adapt Behav 22(1):51–69Drummond C (2009) Replicability is not reproducibility: nor is it good science. In: Proceedings of the evaluation methods for machine learning workshop at the 26th ICML, Montreal, CanadaDrummond C, Japkowicz N (2010) Warning: statistical benchmarking is addictive. Kicking the habit in machine learning. J Exp Theor Artif Intell 22(1):67–80Duan Y, Chen X, Houthooft R, Schulman J, Abbeel P (2016) Benchmarking deep reinforcement learning for continuous control. arXiv preprint arXiv:1604.06778Eden AH, Moor JH, Soraker JH, Steinhart E (2013) Singularity hypotheses: a scientific and philosophical assessment. Springer, New YorkEdmondson W (2012) The intelligence in ETI—what can we know? Acta Astronaut 78:37–42Elo AE (1978) The rating of chessplayers, past and present, vol 3. Batsford, LondonEmbretson SE, Reise SP (2000) Item response theory for psychologists. L. Erlbaum, HillsdaleEvans JM, Messina ER (2001) Performance metrics for intelligent systems. NIST Special Publication SP, pp 101–104Everitt T, Lattimore T, Hutter M (2014) Free lunch for optimisation under the universal distribution. In: 2014 IEEE Congress on evolutionary computation (CEC), IEEE, pp 167–174Falkenauer E (1998) On method overfitting. J Heuristics 4(3):281–287Feldman J (2003) Simplicity and complexity in human concept learning. Gen Psychol 38(1):9–15Ferrando PJ (2009) Difficulty, discrimination, and information indices in the linear factor analysis model for continuous item responses. Appl Psychol Meas 33(1):9–24Ferrando PJ (2012) Assessing the discriminating power of item and test scores in the linear factor-analysis model. Psicológica 33:111–139Ferri C, Hernández-Orallo J, Modroiu R (2009) An experimental comparison of performance measures for classification. Pattern Recogn Lett 30(1):27–38Ferrucci D, Brown E, Chu-Carroll J, Fan J, Gondek D, Kalyanpur AA, Lally A, Murdock J, Nyberg E, Prager J et al (2010) Building Watson: an overview of the DeepQA project. AI Mag 31(3):59–79Fogel DB (1991) The evolution of intelligent decision making in gaming. Cybern Syst 22(2):223–236Gaschnig J, Klahr P, Pople H, Shortliffe E, Terry A (1983) Evaluation of expert systems: issues and case studies. Build Exp Syst 1:241–278Geissman JR, Schultz RD (1988) Verification & validation. AI Exp 3(2):26–33Genesereth M, Love N, Pell B (2005) General game playing: overview of the AAAI competition. AI Mag 26(2):62Gerónimo D, López AM (2014) Datasets and benchmarking. In: Vision-based pedestrian protection systems for intelligent vehicles. Springer, pp 87–93Goertzel B, Pennachin C (eds) (2007) Artificial general intelligence. Springer, New YorkGoertzel B, Arel I, Scheutz M (2009) Toward a roadmap for human-level artificial general intelligence: embedding HLAI systems in broad, approachable, physical or virtual contexts. Artif Gen Intell Roadmap InitiatGoldreich O, Vadhan S (2007) Special issue on worst-case versus average-case complexity editors’ foreword. Comput complex 16(4):325–330Gordon BB (2007) Report on panel discussion on (re-)establishing or increasing collaborative links between artificial intelligence and intelligent systems. In: Messina ER, Madhavan R (eds) Proceedings of the 2007 workshop on performance metrics for intelligent systems, pp 302–303Gulwani S, Hernández-Orallo J, Kitzelmann E, Muggleton SH, Schmid U, Zorn B (2015) Inductive programming meets the real world. Commun ACM 58(11):90–99Hand DJ (2004) Measurement theory and practice. A Hodder Arnold Publication, LondonHernández-Orallo J (2000a) Beyond the Turing test. J Logic Lang Inf 9(4):447–466Hernández-Orallo J (2000b) On the computational measurement of intelligence factors. In: Meystel A (ed) Performance metrics for intelligent systems workshop. National Institute of Standards and Technology, Gaithersburg, pp 1–8Hernández-Orallo J (2000c) Thesis: computational measures of information gain and reinforcement in inference processes. AI Commun 13(1):49–50Hernández-Orallo J (2010) A (hopefully) non-biased universal environment class for measuring intelligence of biological and artificial systems. In: Artificial general intelligence, 3rd International Conference. Atlantis Press, Extended report at http://users.dsic.upv.es/proy/anynt/unbiased.pdf , pp 182–183Hernández-Orallo J (2014) On environment difficulty and discriminating power. Auton Agents Multi-Agent Syst. 29(3):402–454. doi: 10.1007/s10458-014-9257-1Hernández-Orallo J, Dowe DL (2010) Measuring universal intelligence: towards an anytime intelligence test. Artif Intell 174(18):1508–1539Hernández-Orallo J, Dowe DL (2013) On potential cognitive abilities in the machine kingdom. Minds Mach 23:179–210Hernández-Orallo J, Minaya-Collado N (1998) A formal definition of intelligence based on an intensional variant of Kolmogorov complexity. In: Proceedings of international symposium of engineering of intelligent systems (EIS’98), ICSC Press, pp 146–163Hernández-Orallo J, Dowe DL, España-Cubillo S, Hernández-Lloreda MV, Insa-Cabrera J (2011) On more realistic environment distributions for defining, evaluating and developing intelligence. In: Schmidhuber J, Thórisson K, Looks M (eds) Artificial general intelligence, LNAI, vol 6830. Springer, New York, pp 82–91Hernández-Orallo J, Flach P, Ferri C (2012a) A unified view of performance metrics: translating threshold choice into expected classification loss. J Mach Learn Res 13(1):2813–2869Hernández-Orallo J, Insa-Cabrera J, Dowe DL, Hibbard B (2012b) Turing Tests with Turing machines. In: Voronkov A (ed) Turing-100, EPiC Series, vol 10, pp 140–156Hernández-Orallo J, Dowe DL, Hernández-Lloreda MV (2014) Universal psychometrics: measuring cognitive abilities in the machine kingdom. Cogn Syst Res 27:50–74Hernández-Orallo J, Martínez-Plumed F, Schmid U, Siebers M, Dowe DL (2016) Computer models solving intelligence test problems: progress and implications. Artif Intell 230:74–107Herrmann E, Call J, Hernández-Lloreda MV, Hare B, Tomasello M (2007) Humans have evolved specialized skills of social cognition: the cultural intelligence hypothesis. Science 317(5843):1360–1366Hibbard B (2009) Bias and no free lunch in formal measures of intelligence. J Artif Gen Intell 1(1):54–61Hingston P (2010) A new design for a Turing Test for bots. In: 2010 IEEE symposium on computational intelligence and games (CIG), IEEE, pp 345–350Hingston P (2012) Believable bots: can computers play like people?. Springer, New YorkHo TK, Basu M (2002) Complexity measures of supervised classification problems. IEEE Trans Pattern Anal Mach Intell 24(3):289–300Hutter M (2007) Universal algorithmic intelligence: a mathematical top $$\rightarrow$$ → down approach. In: Goertzel B, Pennachin C (eds) Artificial general intelligence, cognitive technologies. Springer, Berlin, pp 227–290Igel C, Toussaint M (2005) A no-free-lunch theorem for non-uniform distributions of target functions. J Math Model Algorithms 3(4):313–322Insa-Cabrera J (2016) Towards a universal test of social intelligence. Ph.D. thesis, Departament de Sistemes Informátics i Computació, UPVInsa-Cabrera J, Dowe DL, España-Cubillo S, Hernández-Lloreda MV, Hernández-Orallo J (2011a) Comparing humans and ai agents. In: Schmidhuber J, Thórisson K, Looks M (eds) Artificial general intelligence, LNAI, vol 6830. Springer, New York, pp 122–132Insa-Cabrera J, Dowe DL, Hernández-Orallo J (2011) Evaluating a reinforcement learning algorithm with a general intelligence test. In: Lozano JA, Gamez JM (eds) Current topics in artificial intelligence. CAEPIA 2011, LNAI series 7023. Springer, New YorkInsa-Cabrera J, Benacloch-Ayuso JL, Hernández-Orallo J (2012) On measuring social intelligence: experiments on competition and cooperation. In: Bach J, Goertzel B, Iklé M (eds) AGI, lecture notes in computer science, vol 7716. Springer, New York, pp 126–135Jacoff A, Messina E, Weiss BA, Tadokoro S, Nakagawa Y (2003) Test arenas and performance metrics for urban search and rescue robots. In: Proceedings of 2003 IEEE/RSJ international conference on intelligent robots and systems, 2003 (IROS 2003), IEEE, vol 4, pp 3396–3403Japkowicz N, Shah M (2011) Evaluating learning algorithms. Cambridge University Press, CambridgeJiang J (2008) A literature survey on domain adaptation of statistical classifiers. http://sifaka.cs.uiuc.edu/jiang4/domain_adaptation/surveyJohnson M, Hofmann K, Hutton T, Bignell D (2016) The Malmo platform for artificial intelligence experimentation. In: International joint conference on artificial intelligence (IJCAI)Keith TZ, Reynolds MR (2010) Cattell–Horn–Carroll abilities and cognitive tests: what we’ve learned from 20 years of research. Psychol Schools 47(7):635–650Ketter W, Symeonidis A (2012) Competitive benchmarking: lessons learned from the trading agent competition. AI Mag 33(2):103Khreich W, Granger E, Miri A, Sabourin R (2012) A survey of techniques for incremental learning of HMM parameters. Inf Sci 197:105–130Kim JH (2004) Soccer robotics, vol 11. Springer, New YorkKitano H, Asada M, Kuniyoshi Y, Noda I, Osawa E (1997) Robocup: the robot world cup initiative. In: Proceedings of the first international conference on autonomous agents, ACM, pp 340–347Kleiner K (2011) Who are you calling bird-brained? An attempt is being made to devise a universal intelligence test. Economist 398(8723, 5 March 2011):82Knuth DE (1973) Sorting and searching, volume 3 of the art of computer programming. Addison-Wesley, ReadingKoza JR (2010) Human-competitive results produced by genetic programming. Genet Program Evolvable Mach 11(3–4):251–284Krueger J, Osherson D (1980) On the psychology of structural simplicity. In: Jusczyk PW, Klein RM (eds) The nature of thought: essays in honor of D. O. Hebb. Psychology Press, London, pp 187–205Langford J (2005) Clever methods of overfitting. Machine Learning (Theory). http://hunch.netLangley P (1987) Research papers in machine learning. Mach Learn 2(3):195–198Langley P (2011) The changing science of machine learning. Mach Learn 82(3):275–279Langley P (2012) The cognitive systems paradigm. Adv Cogn Syst 1:3–13Lattimore T, Hutter M (2013) No free lunch versus Occam’s razor in supervised learning. Algorithmic Probability and Friends. Springer, Bayesian Prediction and Artificial Intelligence, pp 223–235Leeuwenberg ELJ, Van Der Helm PA (2012) Structural information theory: the simplicity of visual form. Cambridge University Press, CambridgeLegg S, Hutter M (2007a) Tests of machine intelligence. In: Lungarella M, Iida F, Bongard J, Pfeifer R (eds) 50 Years of Artificial Intelligence, Lecture Notes in Computer Science, vol 4850, Springer Berlin Heidelberg, pp 232–242. doi: 10.1007/978-3-540-77296-5_22Legg S, Hutter M (2007b) Universal intelligence: a definition of machine intelligence. Minds Mach 17(4):391–444Legg S, Veness J (2013) An approximation of the universal intelligence measure. Algorithmic Probability and Friends. Springer, Bayesian Prediction and Artificial Intelligence, pp 236–249Levesque HJ (2014) On our best behaviour. Artif Intell 212:27–35Levesque HJ, Davis E, Morgenstern L (2012) The winog