Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence

An important challenge for safety in machine learning and artificial intelligence systems is a~set of related failures involving specification gaming, reward hacking, fragility to distributional shifts, and Goodhart's or Campbell's law. This paper presents additional failure modes for interactions within multi-agent systems that are closely related. These multi-agent failure modes are more complex, more problematic, and less well understood than the single-agent case, and are also already occurring, largely unnoticed. After motivating the discussion with examples from poker-playing artificial intelligence (AI), the paper explains why these failure modes are in some senses unavoidable. Following this, the paper categorizes failure modes, provides definitions, and cites examples for each of the modes: accidental steering, coordination failures, adversarial misalignment, input spoofing and filtering, and goal co-option or direct hacking. The paper then discusses how extant literature on multi-agent AI fails to address these failure modes, and identifies work which may be useful for the mitigation of these failure modes.Comment: 12 Pages, This version re-submitted to Big Data and Cognitive Computing, Special Issue "Artificial Superintelligence: Coordination & Strategy

Principal agent problems under loss aversion: an application to executive stock options

Executive stock options reward success but do not penalise failure. In contrast, the standard principalagent model implies that pay is normally monotonically increasing in performance. This paper shows that, under loss aversion, the use of carrots but not sticks is a feature of an optimal compensation contract. Low risk aversion and high loss aversion is particularly propitious to the use of options. Moreover, loss aversion on the part of executives explains the award of at the money options rather than discounted stock or bonus related pay. Other features of stock option grants are also explained, such as resetting or reloading with an exercise price equal to the current stock price

Irreversible investment and the value of information gathering

This note develops a model in which a firm has to decide whether to undertake an irreversible investment. The firm has the option to delay it's decision in an effort to observe the actions of other firms. It is shown that a problem, akin to the herding phenomenon also applies, despite the endogenous time framework. In the context of an investment decision this manifests itself as the failure of a positive-payoff project to be undertaken. The most novel finding is that attempts to overcome this difficulty by further information gathering will, as a side effect, generate additional delay which may be enough to offset the gains of any new information

Irreversible investment and the value of information gathering

This note develops a model in which a firm has to decide whether to undertake an irreversible investment. The firm has the option to delay it's decision in an effort to observe the actions of other firms. It is shown that a problem, akin to the herding phenomenon also applies, despite the endogenous time framework. In the context of an investment decision this manifests itself as the failure of a positive-payoff project to be undertaken. The most novel finding is that attempts to overcome this difficulty by further information gathering will, as a side effect, generate additional delay which may be enough to offset the gains of any new information.herding

Comparative Tensile Strengths of Preceramic and Postceramic Solder Connectors Using High-Palladium Alloy

Purpose: To evaluate the tensile strength properties of Rx Naturelle Plus and Option (high-palladium) alloys on soldered connectors under simulated preceramic and postceramic soldering conditions. Materials and methods: Eighty cylindrical castings were fabricated (40 using Rx Naturelle Plus alloy and 40 using Option alloy). The 40 castings for each alloy were subdivided into 2 groups of 20 each. In the first group, castings for each alloy were randomly paired and soldered with SMG2 solder to produce 10 preceramic test connector specimens for each alloy. In the second group, castings were similarly paired and soldered with 490 fine solder to produce 10 postceramic test connector specimens for each alloy. Each solder connector was subjected to tensile force until failure using an Instron testing machine. Statistical analysis using a Wilcoxon Rank Sums Test was performed.Results: No significant difference was found between the mean connector strength for Rx Naturelle Plus preceramic solder (mean tensile failure stress, 50,000 Psi; STD, 11,250) compared to Option preceramic solder (mean tensile failure stress, 59,700.4 Psi; STD, 16,350) (p= 0.1202). However, the connector strength for Rx Naturelle Plus postceramic solder (mean tensile failure stress, 37,800 Psi; STD, 32,450) was significantly lower than the Option postceramic solder (mean tensile failure stress, 45,300 Psi; STD, 17,550) (p= 0.0407). Not only did Rx Naturelle Plus postceramic solder connectors show evidence of lower strength, but also an increased variation among specimens.Conclusions: Rx Naturelle Plus solder connectors provided better strength with the preceramic are opposed with the postceramic connectors. For postsolder connectors, Rx Naturelle Plus alloy had less acceptable strength and a wider variation, suggesting it is more technique sensitive than Option alloy connectors

Exceptional and Durable Responses to TDM-1 After Trastuzumab Failure for Breast Cancer Skin Metastases: Potential Implications of an Immunological Sanctuary

Breast Cancer (BC) skin metastases represent a challenging clinical scenario. Although they usually arise when other distant metastases are already present, they may also represent a form of locoregional recurrence (LRR). Systemic therapy in this setting may have a role both in case a radical locoregional approach is unfeasible in order to achieve disease control, and as adjuvant strategy after radical removal of cutaneous lesions, in order to prevent or delay subsequent disease spread. Systemic therapy for HER2+ metastatic BC (MBC) currently relies on anti-HER2 targeted agents. In this context TDM1 is an option in trastuzumab-resistant patients.Here we present 2 cases of isolated skin metastases in patients with HER2+ BC progressing during or early after trastuzumab-based therapy, showing impressive responses to TDM1. We hypothesize that the unique properties of skin immune microenvironment may explain the failure of trastuzumab, which exerts its action also through immunological mechanisms, and the subsequent outlier responses to TDM1, that relies on a partially different mechanism of action

Evidence and Consequences of the Central Role of the Kidneys in the Pathophysiology of Sympathetic Hyperactivity

Chronic elevation of the sympathetic nervous system has been identified as a major contributor to the complex pathophysiology of hypertension, states of volume overload – such as heart failure – and progressive kidney disease. It is also a strong determinant for clinical outcome. This review focuses on the central role of the kidneys in the pathogenesis of sympathetic hyperactivity. As a consequence, renal denervation may be an attractive option to treat sympathetic hyperactivity. The review will also focus on first results and the still remaining questions of this new treatment option

TOO MUCH INVESTMENT : A PROBLEM OF COORDINATION FAILURE

This paper shows that coordination failure and contractual incompleteness can lead to socially excessive investment. Firms and workers choose investment levels, then enter a stochastic matching process. If investment levels are discrete, and match frictions are low, high-investing workers (firms) impose a negative pecuniary externality on any worker (firm) who cuts investment. Specifically, an agent cutting investment subsequently bargains with a partner with a binding outside option due to the fact that it can easily match with another high investor. The deviant thus bears the full loss in revenue from its action. However, given enough complementarity in investments, when one agent cuts investment it is efficient that its partner also does so. So, only part of the cost saving accrues to the deviant, with the implication that the net private gain to cutting investment is less than the social gain. A similar argument establishes that over-investment can occur when agents are heterogenous i.e. differ in their cost of investing, even if investments are continuous. Then, over-investment occurs because low-cost investors have a private incentive to invest to shift rent away from high-cost investors. Our model can also explain some recent trends in graduate/non-graduate wage differentials.hold-up ; coordination failure ; matching ; over-investment

Three Essays on Default Risk in Capital Markets

This dissertation comprises three essays on default risk in capital markets exploring (a) failure risk of hedge funds, (b) pricing in equity option markets, and (c) relationship between option and credit default swap markets, respectively, with a particular focus on the recent financial crisis. The first essay “The role of Excess Leverage in Hedge Funds Failure” investigates the role of financial leverage, including the use of margins and derivatives, in the hedge funds failure during the 2008 financial crisis. Motivated by failure of the two Bear Sterns hedge funds, this paper examines why some hedge funds failed during and after the recent financial crisis, and why some also survived. Using a 15-year panel dataset of 17,202 hedge funds from the Lipper TASS Hedge Fund database, the empirical analysis shows that during the crisis period, financial leverage is more significant in increasing the probability of failure, whereas it becomes insignificant during non-crisis periods. Moreover, hedge funds following specific styles such as “Emerging Markets”, “Equity Market”, “Long/Short Equity Hedge”, and “Multi-strategy” are more likely to fail during the financial crisis. The second essay “Is Default Risk Priced in Equity Options?” explores the impact of default risk on equity option pricing. The impact is studied in detail by empirically examining to what extent the firm-specific default risk matters in pricing individual equity options. Since credit default swaps (CDS) are similar to put options in that both offer a low cost and effective protection against downside risk, we use CDS spread as credit risk proxy to investigate the effects of default risk on put pricing. By examining an exhaustive sample of US-listed firms with both CDS and put options data available over the period from 2002 to 2010, and studying the primary determinants of option implied volatility (IV) cross-sectionally and over time, the findings show that default risk is a significant factor in the prices of equity options. The results remain significant after controlling for firm-specific and macroeconomic factors, and endogeneity. The third essay “The Impact of Using CDS in Forecasting Option-Implied Volatility” addresses the issue of forecasting IV which is of interest to option market participants, who routinely formulate volatility and option price forecasts for trading and hedging purposes. Credit risk matters for option pricing since options are valued on firms with significant trading liquidity, yet subject to default risk, similar to liquidity risk. This essay particularly explores whether better out-of-sample forecasts for IV can be developed using lagged credit risk measures. Various time-series IV forecasts show that inclusion of default risk as measured by CDS can significantly improve out-of-sample performance, measured through decreased mean squared error (MSE) as well as smaller root mean squared error (RMSE)