Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence

An important challenge for safety in machine learning and artificial intelligence systems is a~set of related failures involving specification gaming, reward hacking, fragility to distributional shifts, and Goodhart's or Campbell's law. This paper presents additional failure modes for interactions within multi-agent systems that are closely related. These multi-agent failure modes are more complex, more problematic, and less well understood than the single-agent case, and are also already occurring, largely unnoticed. After motivating the discussion with examples from poker-playing artificial intelligence (AI), the paper explains why these failure modes are in some senses unavoidable. Following this, the paper categorizes failure modes, provides definitions, and cites examples for each of the modes: accidental steering, coordination failures, adversarial misalignment, input spoofing and filtering, and goal co-option or direct hacking. The paper then discusses how extant literature on multi-agent AI fails to address these failure modes, and identifies work which may be useful for the mitigation of these failure modes.Comment: 12 Pages, This version re-submitted to Big Data and Cognitive Computing, Special Issue "Artificial Superintelligence: Coordination & Strategy

A Cost-effective Shuffling Method against DDoS Attacks using Moving Target Defense

Moving Target Defense (MTD) has emerged as a newcomer into the asymmetric field of attack and defense, and shuffling-based MTD has been regarded as one of the most effective ways to mitigate DDoS attacks. However, previous work does not acknowledge that frequent shuffles would significantly intensify the overhead. MTD requires a quantitative measure to compare the cost and effectiveness of available adaptations and explore the best trade-off between them. In this paper, therefore, we propose a new cost-effective shuffling method against DDoS attacks using MTD. By exploiting Multi-Objective Markov Decision Processes to model the interaction between the attacker and the defender, and designing a cost-effective shuffling algorithm, we study the best trade-off between the effectiveness and cost of shuffling in a given shuffling scenario. Finally, simulation and experimentation on an experimental software defined network (SDN) indicate that our approach imposes an acceptable shuffling overload and is effective in mitigating DDoS attacks

THE IMPACT OF PROGRAMMING LANGUAGES ON THE SOFTWARE’S SECURITY

Security is usually defined as the ability of a system to protect itself against accidental or deliberate intrusion1. Ensuring integrity, confidentiality, availability, and accountability requirements even in the presence of a determined, malicious opponent is essential for computer security. Sensitive data has to be manipulated and consulted by authorized users only (integrity, confidentiality). Furthermore, the system should resist “denial of service” attacks that attempt to render it unusable (availability). Also the system has to ensure the inability to deny the ownership of prior actions (accountability).security

Rational Adversaries? Evidence from Randomized Trials in the Game of Cricket

In cricket, the right to make an important strategic decision is assigned via a coin toss. We utilize these "randomized trials" to examine (a) the consistency of choices made by teams with strictly opposed preferences, and (b) the treatment effects of chosen actions. We find significant evidence of inconsistency, with teams often agreeing on who is to bat first. Estimated treatment effects show that choices are often poorly made since they reduce the probability of the team winning.

EXPERIMENTAL MARKETS AND ENVIRONMENTAL POLICY

Experimental markets can be a useful tool to guide and evaluate environmental policy. This paper reviews four experiments to illustrate. Two institutional experiments are considered-Coasian bargaining with positive transaction costs, and a gaming experiment of dynamic choice in a conflict. Two valuation experiments are also discussed-the impact of sequential reduction mechanisms on the value of risk, and experimental auction markets to elicit the value of safer food.Environmental Economics and Policy,

A Graphical Adversarial Risk Analysis Model for Oil and Gas Drilling Cybersecurity

Oil and gas drilling is based, increasingly, on operational technology, whose cybersecurity is complicated by several challenges. We propose a graphical model for cybersecurity risk assessment based on Adversarial Risk Analysis to face those challenges. We also provide an example of the model in the context of an offshore drilling rig. The proposed model provides a more formal and comprehensive analysis of risks, still using the standard business language based on decisions, risks, and value.Comment: In Proceedings GraMSec 2014, arXiv:1404.163

Optimization incentive and relative riskiness in experimental coordination games

We compare the experimental results of three stag-hunt games. In contrast to Battalio et al. (2001), our design keeps the riskiness ratio of the payoff-dominant and the risk-dominant strategies at a constant level as the optimisation premium is increased. We define the riskiness ratio as the relative payoff range of the two strategies. We find that decreasing the riskiness ratio while keeping the optimization premium constant increases sharply the frequency of the risk-dominant strategy. On the other hand an increase of the optimization premium with a constant riskiness ratio has no effect on the choice frequencies. Finally, we confirm the dynamic properties found by Battalio et al. that increasing the optimization premium favours best-response and sensitivity to the history of play.

Negativity effect and the emergence of ideologies

``Negativity effect'' refers to the psychological phenomenon that people tend to attach greater weight to negative information than to equally extreme and equally likely positive information in a variety of information processing tasks. Numerous studies of impression formation have found that negative information is weighted more heavily than positive information as impressions of others are formed. There is empirical evidence in political science that shows the importance of the negativity effect in the information processing of the voters. This effect can explain the observed decrease of popularity for a president the longer he is in office. \\ We construct a dynamic model of political competition, incorporating the negativity effect in the decision rule of the voters and allowing their preferences to change over time, according to the past performance of the candidates while in office. Our model may explain the emergence of ideologies out of the competition for votes of myopic candidates freely choosing policy positions. This result gives rise to the formation of political parties, as infinitely--lived agents with a certain ideology. Furthermore, in this model some voters may start out by switching among parties associated with different policies, but find themselves supporting one of the parties from some point on. Thus, the model describes a process by which some voters become identified with a ``right'' or ``left'' bloc, while others ``swing'' between the two parties.Negativity effect, formation of ideologies

Application of Game Theory in Describing Efficacy of Decision Making in Sportsman\u27s Tactical Performance in Team Sports

A mathematical method of decision-making in which a competitive or cooperative situation is analyzed to determine the optimal course of action for an interested “player” is often called game theory. Game theory has very broad application in different sciences. Team sports tactical performance is considered from the aspects of data processing theory and the phenomenon of selective attention, as well as from the game theory. Team sports tactical performance is an asymmetric, sequential (of imperfect information), non-zero-sum game. In decision making, predictability in team sports is in fact bargaining, and the player has to use a mixed strategy for choosing option with highest expected utility. Player could choose a trembling hand equilibrium, to eliminate imperfect equilibrium. Strategic dominance concept can explain that a player could choose strategy which dominates between other possible strategies, and/or could be led by “team reasoning”, too. In this article, the level of predictability of the most frequent tactical performance of one player in a team sport game is considered, reflecting outcomes both for the same team’s tactical performance (co-players in one player’s team), as well as for the opponent team’s tactical performance. Four different possible situations during team sport competition could lead to considering utilities of one player’s specific decisions

Spartan Daily, May 15, 1990

Volume 94, Issue 69https://scholarworks.sjsu.edu/spartandaily/7999/thumbnail.jp