RoboCup 2D Soccer Simulation League: Evaluation Challenges

We summarise the results of RoboCup 2D Soccer Simulation League in 2016 (Leipzig), including the main competition and the evaluation round. The evaluation round held in Leipzig confirmed the strength of RoboCup-2015 champion (WrightEagle, i.e. WE2015) in the League, with only eventual finalists of 2016 competition capable of defeating WE2015. An extended, post-Leipzig, round-robin tournament which included the top 8 teams of 2016, as well as WE2015, with over 1000 games played for each pair, placed WE2015 third behind the champion team (Gliders2016) and the runner-up (HELIOS2016). This establishes WE2015 as a stable benchmark for the 2D Simulation League. We then contrast two ranking methods and suggest two options for future evaluation challenges. The first one, "The Champions Simulation League", is proposed to include 6 previous champions, directly competing against each other in a round-robin tournament, with the view to systematically trace the advancements in the League. The second proposal, "The Global Challenge", is aimed to increase the realism of the environmental conditions during the simulated games, by simulating specific features of different participating countries.Comment: 12 pages, RoboCup-2017, Nagoya, Japan, July 201

AI Researchers, Video Games Are Your Friends!

If you are an artificial intelligence researcher, you should look to video games as ideal testbeds for the work you do. If you are a video game developer, you should look to AI for the technology that makes completely new types of games possible. This chapter lays out the case for both of these propositions. It asks the question "what can video games do for AI", and discusses how in particular general video game playing is the ideal testbed for artificial general intelligence research. It then asks the question "what can AI do for video games", and lays out a vision for what video games might look like if we had significantly more advanced AI at our disposal. The chapter is based on my keynote at IJCCI 2015, and is written in an attempt to be accessible to a broad audience.Comment: in Studies in Computational Intelligence Studies in Computational Intelligence, Volume 669 2017. Springe

Machine intelligence sports as research programs

Games and competitions have played a significant role throughout the history of artificial intelligence and robotics. Machine intelligence games are examined here from a distinctive methodological perspective, focusing on their use as generators of multidisciplinary research programs. In particular, Robocup is analyzed as an exemplary case of contemporary research program developing from machine intelligence games. These research programs arising are schematized in terms of framework building, subgoaling, and outcome appraisal processes. The latter process is found to involve a rather intricate system of rewards and penalties, which take into account the double allegiance of participating scientists, trading and sharing interchanges taking place in a multidisciplinary research environment, in addition to expected industrial payoffs and a variety of other fringe research benefits in the way of research outreach and results dissemination, recruitment of junior researchers and students enrollment

RoboCup@Home: Analysis and results of evolving competitions for domestic and service robots

Scientific competitions are becoming more common in many research areas of artificial intelligence and robotics, since they provide a shared testbed for comparing different solutions and enable the exchange of research results. Moreover, they are interesting for general audiences and industries. Currently, many major research areas in artificial intelligence and robotics are organizing multiple-year competitions that are typically associated with scientific conferences. One important aspect of such competitions is that they are organized for many years. This introduces a temporal evolution that is interesting to analyze. However, the problem of evaluating a competition over many years remains unaddressed. We believe that this issue is critical to properly fuel changes over the years and measure the results of these decisions. Therefore, this article focuses on the analysis and the results of evolving competitions. In this article, we present the RoboCup@Home competition, which is the largest worldwide competition for domestic service robots, and evaluate its progress over the past seven years. We show how the definition of a proper scoring system allows for desired functionalities to be related to tasks and how the resulting analysis fuels subsequent changes to achieve general and robust solutions implemented by the teams. Our results show not only the steadily increasing complexity of the tasks that RoboCup@Home robots can solve but also the increased performance for all of the functionalities addressed in the competition. We believe that the methodology used in RoboCup@Home for evaluating competition advances and for stimulating changes can be applied and extended to other robotic competitions as well as to multi-year research projects involving Artificial Intelligence and Robotics

Systematic mapping literature review of mobile robotics competitions

This paper presents a systematic mapping literature review about the mobile robotics competitions that took place over the last few decades in order to obtain an overview of the main objectives, target public, challenges, technologies used and final application area to show how these competitions have been contributing to education. In the review we found 673 papers from 5 different databases and at the end of the process, 75 papers were classified to extract all the relevant information using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method. More than 50 mobile robotics competitions were found and it was possible to analyze most of the competitions in detail in order to answer the research questions, finding the main goals, target public, challenges, technologies and application area, mainly in education.info:eu-repo/semantics/publishedVersio