Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence

An important challenge for safety in machine learning and artificial intelligence systems is a~set of related failures involving specification gaming, reward hacking, fragility to distributional shifts, and Goodhart's or Campbell's law. This paper presents additional failure modes for interactions within multi-agent systems that are closely related. These multi-agent failure modes are more complex, more problematic, and less well understood than the single-agent case, and are also already occurring, largely unnoticed. After motivating the discussion with examples from poker-playing artificial intelligence (AI), the paper explains why these failure modes are in some senses unavoidable. Following this, the paper categorizes failure modes, provides definitions, and cites examples for each of the modes: accidental steering, coordination failures, adversarial misalignment, input spoofing and filtering, and goal co-option or direct hacking. The paper then discusses how extant literature on multi-agent AI fails to address these failure modes, and identifies work which may be useful for the mitigation of these failure modes.Comment: 12 Pages, This version re-submitted to Big Data and Cognitive Computing, Special Issue "Artificial Superintelligence: Coordination & Strategy

Self-Evaluation Applied Mathematics 2003-2008 University of Twente

This report contains the self-study for the research assessment of the Department of Applied Mathematics (AM) of the Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) at the University of Twente (UT). The report provides the information for the Research Assessment Committee for Applied Mathematics, dealing with mathematical sciences at the three universities of technology in the Netherlands. It describes the state of affairs pertaining to the period 1 January 2003 to 31 December 2008

Competitive Benchmarking: An IS Research Approach to Address Wicked Problems with Big Data and Analytics

Wicked problems like sustainable energy and financial market stability are societal challenges that arise from complex socio-technical systems in which numerous social, economic, political, and technical factors interact. Understanding and mitigating them requires research methods that scale beyond the traditional areas of inquiry of Information Systems (IS) “individuals, organizations, and markets” and that deliver solutions in addition to insights. We describe an approach to address these challenges through Competitive Benchmarking (CB), a novel research method that helps interdisciplinary research communities to tackle complex challenges of societal scale by using different types of data from a variety of sources such as usage data from customers, production patterns from producers, public policy and regulatory constraints, etc. for a given instantiation. Further, the CB platform generates data that can be used to improve operational strategies and judge the effectiveness of regulatory regimes and policies. We describe our experience applying CB to the sustainable energy challenge in the Power Trading Agent Competition (Power TAC) in which more than a dozen research groups from around the world jointly devise, benchmark, and improve IS-based solutions

Challenges in Complex Systems Science

FuturICT foundations are social science, complex systems science, and ICT. The main concerns and challenges in the science of complex systems in the context of FuturICT are laid out in this paper with special emphasis on the Complex Systems route to Social Sciences. This include complex systems having: many heterogeneous interacting parts; multiple scales; complicated transition laws; unexpected or unpredicted emergence; sensitive dependence on initial conditions; path-dependent dynamics; networked hierarchical connectivities; interaction of autonomous agents; self-organisation; non-equilibrium dynamics; combinatorial explosion; adaptivity to changing environments; co-evolving subsystems; ill-defined boundaries; and multilevel dynamics. In this context, science is seen as the process of abstracting the dynamics of systems from data. This presents many challenges including: data gathering by large-scale experiment, participatory sensing and social computation, managing huge distributed dynamic and heterogeneous databases; moving from data to dynamical models, going beyond correlations to cause-effect relationships, understanding the relationship between simple and comprehensive models with appropriate choices of variables, ensemble modeling and data assimilation, modeling systems of systems of systems with many levels between micro and macro; and formulating new approaches to prediction, forecasting, and risk, especially in systems that can reflect on and change their behaviour in response to predictions, and systems whose apparently predictable behaviour is disrupted by apparently unpredictable rare or extreme events. These challenges are part of the FuturICT agenda

Analysis Of Aircraft Arrival Delay And Airport On-time Performance

While existing grid environments cater to specific needs of a particular user community, we need to go beyond them and consider general-purpose large-scale distributed systems consisting of large collections of heterogeneous computers and communication systems shared by a large user population with very diverse requirements. Coordination, matchmaking, and resource allocation are among the essential functions of large-scale distributed systems. Although deterministic approaches for coordination, matchmaking, and resource allocation have been well studied, they are not suitable for large-scale distributed systems due to the large-scale, the autonomy, and the dynamics of the systems. We have to seek for nondeterministic solutions for large-scale distributed systems. In this dissertation we describe our work on a coordination service, a matchmaking service, and a macro-economic resource allocation model for large-scale distributed systems. The coordination service coordinates the execution of complex tasks in a dynamic environment, the matchmaking service supports finding the appropriate resources for users, and the macro-economic resource allocation model allows a broker to mediate resource providers who want to maximize their revenues and resource consumers who want to get the best resources at the lowest possible price, with some global objectives, e.g., to maximize the resource utilization of the system

Smart Grid Technologies in Europe: An Overview

The old electricity network infrastructure has proven to be inadequate, with respect to modern challenges such as alternative energy sources, electricity demand and energy saving policies. Moreover, Information and Communication Technologies (ICT) seem to have reached an adequate level of reliability and flexibility in order to support a new concept of electricity network—the smart grid. In this work, we will analyse the state-of-the-art of smart grids, in their technical, management, security, and optimization aspects. We will also provide a brief overview of the regulatory aspects involved in the development of a smart grid, mainly from the viewpoint of the European Unio

Towards Decentralized Power Systems: Market & Regulatory Frameworks for Ontario

This MRP is about the sustainability transition of Ontario’s electricity system. A sustainability transition is understood as a type of purposive socio-technical transition, which is meant to address some normative goal rather than to exploit commercial opportunity. I use the analytical framework presented by the socio-technical transitions literature and multi-level perspective theory to assess the state of Ontario’s grid modernization as evidenced through primarily documentary evidence, most notably the 2017 Long-Term Energy Plan. I conclude that based on this evidence, Ontario is taking a transformation pathway, which is characterized as being driven from within the established regime that modifies its own trajectory in response to landscape pressures and an under-developed niche. This is represented by Ontario’s preferred approach to enable LDCs as the primary developers of DERs through regulatory changes. I then argue that in light of sustainability objectives that I identify in this paper, Ontario’s approach has some shortfalls, and instead I recommend a reconfiguration pathway that requires the strategic modification of 3 key areas to enable a competitive retail DERs market. The three key areas are: (1) adjustments to the grid architecture to address the operational and functional roles of grid actors; (2) establishment of a market structure known as a platform to enable the participation of distributed resources to compete with traditional resources on a level playing, which can be done either at the bulk or distribution levels; and (3) the regulation of a competitive retail DERs market in Ontario