The past decade has seen great progress in artificial intelligence (AI). Machines can now categorize and generate images, make complex physical and social inferences, and reach, or exceed, human performance in many games. However, the long-term goal of recreating human-like general intelligence remains out of reach and some argue that a radical change in approach is needed

Beyret, Benjamin

Crosby, Matthew

Halina, Marta

English

Apollo (Cambridge)

The Animal AI Olympics 1 Matthew Crosby, Benjamin Beyret and Marta Halina 2  3 A new competition presents AI agents with cognition challenges to test their 4 animal intelligence 5  6 The last decade has seen great progress in artificial intelligence (AI). Machines can now categorise 7 and generate images, make complex physical and social inferences, and reach, or exceed, human 8 performance in many games. However, the long-term goal of recreating human-like general 9 intelligence remains out of reach and some argue that a radical change in approach is needed [1].  10 Animal-level intelligence provides a natural stepping stone on the path towards human-level AI. 11 Tests in animal cognition research normally involve presenting an animal with a problem or 12 environment that it would not naturally encounter and seeing if it can ‘work out’ how to obtain a 13 reward—typically food. These tasks are carefully designed to probe for a particular cognitive 14 capacity, such that successful performance on the task provides evidence that the animal has the 15 capacity in question.  Researchers have used this approach to test for capacities such as episodic 16 memory, planning, spatial reasoning and social cognition in animals as varied as dogs, goats, 17 chimpanzees and spiders.  18 In contrast to most animals, modern AI systems cannot just be placed in new environments and be 19 expected to perform intelligently. Consider AlphaZero, a general algorithm that can be trained to 20 better-than-human levels at a wide range of perfect information games. Without extensive 21 retraining, however, it cannot adapt to never-seen-before games on the fly, and different games 22 may require different input spaces, fundamentally preventing transfer between them.  While 23 AlphaZero is an impressive feat of AI, this case illustrates the large difference between current 24 generalisation capabilities of state-of-the-art AI systems and animals.  25 In animal cognition research a wide range of species with different types of embodiment and 26 (biological) actuators have been tested using a variety of experimental paradigms. These paradigms 27 typically abstract away from interspecies differences by focusing on intelligent behaviour mediated 28 by the shared sensory modality of vision. At the same time, we have seen rapid progress in the 29 ability to train AI systems through visual inputs alone [2]. Thus, it is an ideal time for making direct 30 comparisons between animals and AI. This is the aim of the Animal-AI Olympics, a new AI 31 competition that translates vision-based animal cognition tasks into a testbed for cognitive AI. To 32 keep the comparison to the animal case as close as possible, the participants (like the animals) will 33 not know the exact tasks in advance. Participants will instead have to submit an agent that they 34 believe will display robust food retrieval behaviour in tasks unknown to the developer. 35 We will be releasing a ‘playground’, a simple simulation environment for intelligent agents based on 36 the Unity platform [3].  This environment has basic physics rules and a set of objects such as food, 37 walls, negative-reward zones, pushable blocks and more. The ‘playground’ can be configured by the 38 participants and they can spawn any combination of objects in preset or random positions, as 39 depicted in Figure 1. It will be important for the participants to design good environments for their 40 agents to learn in. Configuration files for the playground can also be exchanged between 41 participants should they wish to collaborate. The competition tasks will include ten cognitive 42 categories each with ten subtasks. This gives us one hundred distinct tasks, each of which will be run 43 multiple times with minor variations for testing purposes. The categories will range from basic food 44 retrieval—where only food is in the environment—to tasks that require capacities such as object 45 permanence, object manipulation and an understanding of the basic physics of the environment to 46 solve.  47 We expect this to be a hard challenge for modern AI systems, and want to give publicity to 48 interesting approaches that make even small advancements in this area. We also hope that this will 49 be a good testbed for approaches that use continual, transfer, and one-shot learning as well as non 50 goal-directed learning methods such as curiosity and intrinsic motivation and intuitive physics 51 modelling (see e.g. [4, 5]). Being able to solve all the tasks in a category would demonstrate real 52 cognitive capacities comparable to those found in animals.  53 We will release the playground at the end of April so that there is time for community feedback to 54 be incorporated before the full release of the competition at the end of June. The competition itself 55 will run from June to November with participants able to submit to a live leaderboard throughout. 56 The results will be announced at NeurIPS 2019 in December.  57 We hope this competition sparks further research in cognitive AI and that it becomes a useful 58 ongoing testbed. We expect it to help pinpoint the current challenges and limitations of AI for large-59 scale real-world application involving interaction with unknown environments. We have made great 60 progress on the hard problems, it is now time to tackle the easy ones.  61 Author Affiliations 62  Matthew Crosby and Benjamin Beyret, Leverhulme Centre for the Future of Intelligence, Imperial 63 College London. Marta Halina, Leverhulme Centre for the Future of Intelligence, University of 64 Cambridge; Department of History and Philosophy of Science, University of Cambridge. 65  66 Competing interests 67 The authors declare no competing interests. 68  69 References  70 1. Lake, B. M., Ullman, T. D., Tenenbaum, J. B. & Gershman, S. J. Building machines that learn and 71 think like people. Behavioral and Brain Sciences 40, (2017).  72 2. Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529 (2015). 73 3. Juliani, A. et al. Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627 74 (2018).  75 4. Haber, N., Mrowca, D., Fei-Fei, L. & Yamins, D. L. Emergence of structured behaviors from 76 curiosity-based intrinsic motivation. arXiv preprint arXiv:1802.07461 (2018).  77 5. Ullman, T. D., Spelke, E., Battaglia, P. & Tenenbaum, J. B. Mind games: Game engines as an 78 architecture for intuitive physics. Trends in Cognitive Sciences 21, 649–665 (2017). 79 

The Animal-AI Olympics

https://www.repository.cam.ac.uk/bitstream/handle/1810/291607/The%20Animal%20AI%20Olympics.pdf?sequence=1&isAllowed=y

The Animal-AI Olympics

Abstract

Similar works

Full text

Available Versions

Apollo (Cambridge)