Unmasking Clever Hans Predictors and Assessing What Machines Really Learn

Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly "intelligent" behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem solving behaviors. Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for. Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication

Modern Trends in the Automatic Generation of Content for Video Games

Attractive and realistic content has always played a crucial role in the penetration and popularity of digital games, virtual environments, and other multimedia applications. Procedural content generation enables the automatization of production of any type of game content including not only landscapes and narratives but also game mechanics and generation of whole games. The article offers a comparative analysis of the approaches to automatic generation of content for video games proposed in last five years. It suggests a new typology of the use of procedurally generated game content comprising of categories structured in three groups: content nature, generation process, and game dependence. Together with two other taxonomies – one of content type and the other of methods for content generation – this typology is used for comparing and discussing some specific approaches to procedural content generation in three promising research directions based on applying personalization and adaptation, descriptive languages, and semantic specifications