Unmasking Clever Hans Predictors and Assessing What Machines Really Learn

Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly "intelligent" behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem solving behaviors. Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for. Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication

Mapping and Developing Service Design Research in the UK.

This report is the outcome of the Service Design Research UK (SDR UK) Network with Lancaster University as primary investigator and London College of Communication, UAL as co-investigator. This project was funded as part of an Arts and Humanities Research Council Network grant. Service Design Research UK (SDR UK), funded by an AHRC Network Grant, aims to create a UK research network in an emerging field in Design that is Service Design. This field has a recent history and a growing, but still small and dispersed, research community that strongly needs support and visibility to consolidate its knowledge base and enhance its potential impact. Services represent a significant part of the UK economy and can have a transformational role in our society as they affect the way we organize, move, work, study or take care of our health and family. Design introduces a more human centred and creative approach to service innovation; this is critical to delivering more effective and novel solutions that have the potential to tackle contemporary challenges. Service Design Research UK reviewed and consolidated the emergence of Service Design within the estalished field of Design

Introduction to Microservice API Patterns (MAP)

The Microservice API Patterns (MAP) language and supporting website premiered under this name at Microservices 2019. MAP distills proven, platform- and technology-independent solutions to recurring (micro-)service design and interface specification problems such as finding well-fitting service granularities, rightsizing message representations, and managing the evolution of APIs and their implementations. In this paper, we motivate the need for such a pattern language, outline the language organization and present two exemplary patterns describing alternative options for representing nested data. We also identify future research and development directions

Sharing the vision:representing the matters of concern for design-led fledgling companies in Scotland

Design is being performed on an ever-increasing spectrum of complex practices arising in response to emerging markets and technologies, co-design, digital interaction, service design and cultures of innovation. This emerging notion of design has led to an expansive array of collaborative and facilitation skills to demonstrate and share how such methods can shape innovation. The meaning of these design things in practice can't be taken for granted as matters of fact, which raises a key challenge for design to represent its role through the contradictory nature of matters of concern. This paper explores an innovative, object-oriented approach within the field of design research, visually combining an actor-network theory framework with situational analysis, to report on the role of design for fledgling companies in Scotland, established and funded through the knowledge exchange hub Design in Action (DiA). Key findings and visual maps are presented from reflective discussions with actors from a selection of the businesses within DiA's portfolio. The suggestion is that any notions of strategic value, of engendering meaningful change, of sharing the vision of design, through design things, should be grounded in the reflexive interpretations of matters of concern that emerge

Principles and Concepts of Agent-Based Modelling for Developing Geospatial Simulations

The aim of this paper is to outline fundamental concepts and principles of the Agent-Based Modelling (ABM) paradigm, with particular reference to the development of geospatial simulations. The paper begins with a brief definition of modelling, followed by a classification of model types, and a comment regarding a shift (in certain circumstances) towards modelling systems at the individual-level. In particular, automata approaches (e.g. Cellular Automata, CA, and ABM) have been particularly popular, with ABM moving to the fore. A definition of agents and agent-based models is given; identifying their advantages and disadvantages, especially in relation to geospatial modelling. The potential use of agent-based models is discussed, and how-to instructions for developing an agent-based model are provided. Types of simulation / modelling systems available for ABM are defined, supplemented with criteria to consider before choosing a particular system for a modelling endeavour. Information pertaining to a selection of simulation / modelling systems (Swarm, MASON, Repast, StarLogo, NetLogo, OBEUS, AgentSheets and AnyLogic) is provided, categorised by their licensing policy (open source, shareware / freeware and proprietary systems). The evaluation (i.e. verification, calibration, validation and analysis) of agent-based models and their output is examined, and noteworthy applications are discussed.Geographical Information Systems (GIS) are a particularly useful medium for representing model input and output of a geospatial nature. However, GIS are not well suited to dynamic modelling (e.g. ABM). In particular, problems of representing time and change within GIS are highlighted. Consequently, this paper explores the opportunity of linking (through coupling or integration / embedding) a GIS with a simulation / modelling system purposely built, and therefore better suited to supporting the requirements of ABM. This paper concludes with a synthesis of the discussion that has proceeded. The aim of this paper is to outline fundamental concepts and principles of the Agent-Based Modelling (ABM) paradigm, with particular reference to the development of geospatial simulations. The paper begins with a brief definition of modelling, followed by a classification of model types, and a comment regarding a shift (in certain circumstances) towards modelling systems at the individual-level. In particular, automata approaches (e.g. Cellular Automata, CA, and ABM) have been particularly popular, with ABM moving to the fore. A definition of agents and agent-based models is given; identifying their advantages and disadvantages, especially in relation to geospatial modelling. The potential use of agent-based models is discussed, and how-to instructions for developing an agent-based model are provided. Types of simulation / modelling systems available for ABM are defined, supplemented with criteria to consider before choosing a particular system for a modelling endeavour. Information pertaining to a selection of simulation / modelling systems (Swarm, MASON, Repast, StarLogo, NetLogo, OBEUS, AgentSheets and AnyLogic) is provided, categorised by their licensing policy (open source, shareware / freeware and proprietary systems). The evaluation (i.e. verification, calibration, validation and analysis) of agent-based models and their output is examined, and noteworthy applications are discussed.Geographical Information Systems (GIS) are a particularly useful medium for representing model input and output of a geospatial nature. However, GIS are not well suited to dynamic modelling (e.g. ABM). In particular, problems of representing time and change within GIS are highlighted. Consequently, this paper explores the opportunity of linking (through coupling or integration / embedding) a GIS with a simulation / modelling system purposely built, and therefore better suited to supporting the requirements of ABM. This paper concludes with a synthesis of the discussion that has proceeded

OULDI-JISC Project Evaluation Report: the impact of new curriculum design tools and approaches on institutional process and design cultures

This report presents research and evaluation undertaken by the OULDI-JISC Project (Open University Learning Design Initiative JISC Project) between 2008 and 2012. In particular, it considers the impact of new curriculum design tools and approaches piloted by the project on institutional processes and design cultures. These tools and approaches include tools for sharing learning design expertise (Cloudworks), visualising designs (CompendiumLD, Module Map, Activity Profile) and for supporting design and reflection in workshops (Facilitation Cards, workshop activities, etc.). The project has adopted a learning design approach so as to help foreground pedagogy and learner experience. Nine pilots have been completed across six UK universities