Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback

Machine translation is a natural candidate problem for reinforcement learning from human feedback: users provide quick, dirty ratings on candidate translations to guide a system to improve. Yet, current neural machine translation training focuses on expensive human-generated reference translations. We describe a reinforcement learning algorithm that improves neural machine translation systems from simulated human feedback. Our algorithm combines the advantage actor-critic algorithm (Mnih et al., 2016) with the attention-based neural encoder-decoder architecture (Luong et al., 2015). This algorithm (a) is well-designed for problems with a large action space and delayed rewards, (b) effectively optimizes traditional corpus-level machine translation metrics, and (c) is robust to skewed, high-variance, granular feedback modeled after actual human behaviors.Comment: 11 pages, 5 figures, In Proceedings of Empirical Methods in Natural Language Processing (EMNLP) 201

The price of bank mergers in the 1990s

This article examines the primary motivations for the massive wave of bank mergers in the U.S. during the 1990s by analyzing the prices paid for target banks. The authors find that these prices reflect both general market and firm-specific characteristics. For example, the lifting of regulatory restrictions on geographic markets for bank mergers has a significant impact on the average price paid. Additionally, more profitable target banks tend to command a significantly higher market price.Bank mergers

Hallucination Detection for Grounded Instruction Generation

We investigate the problem of generating instructions to guide humans to navigate in simulated residential environments. A major issue with current models is hallucination: they generate references to actions or objects that are inconsistent with what a human follower would perform or encounter along the described path. We develop a model that detects these hallucinated references by adopting a model pre-trained on a large corpus of image-text pairs, and fine-tuning it with a contrastive loss that separates correct instructions from instructions containing synthesized hallucinations. Our final model outperforms several baselines, including using word probability estimated by the instruction-generation model, and supervised models based on LSTM and Transformer

Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Models

Recent work studies the cognitive capabilities of language models through psychological tests designed for humans. While these studies are helpful for understanding the general capabilities of these models, there is no guarantee that a model possessing sufficient capabilities to pass those tests would actually use those capabilities in performing real-life tasks. In this work, we formulate task-oriented cognitive capabilities, which are human-like cognitive capabilities that language models leverage to perform tasks. These capabilities are (i) the ability to quickly generate good candidate utterances (the search capability) (ii) the ability to predict how a listener interprets those utterances and choose the most appropriate one (the pragmatic capability). We design an evaluation scheme for comparing these capabilities of a language model with those of a human. Applying this scheme to examine various models in a navigation instruction generation problem, we find that their pragmatic capability is severely lacking. This insight leads us to augment them with better models of the listener and obtain a significant boost of 11% in success rate in guiding real humans. Our work advocates for having a principled procedure for aligning language models with humans that involves (i) formulating task-oriented capabilities, (ii) devising a method to quantify their deficiency, and (iii) iteratively improving them.Comment: Findings of ACL 202

Chevalier Jackson, M.D. (1865-1958): Il ne se repose jamais.

In the final year of the American Civil War, 1865, Chevalier Jackson was born on the 4th of November just outside Pittsburgh, Pennsylvania. The eldest of three sons of a poor, livestock-raising family, Jackson was raised in a period of social and political unrest. He was perhaps an even more unrestful boy. The description of his childhood days from his father’s father—Il ne se repose jamais, ‘‘He never rests’’—would ultimately reflect the man, doctor, and evangelist Jackson would later become.1 Indeed, he never did rest, Jackson would tirelessly pave the way for modern bronchoscopy and endoscopy as a whole; bringing international renown not only to himself, but also to his specialty

The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features

AI systems have been known to amplify biases in real world data. Explanations may help human-AI teams address these biases for fairer decision-making. Typically, explanations focus on salient input features. If a model is biased against some protected group, explanations may include features that demonstrate this bias, but when biases are realized through proxy features, the relationship between this proxy feature and the protected one may be less clear to a human. In this work, we study the effect of the presence of protected and proxy features on participants' perception of model fairness and their ability to improve demographic parity over an AI alone. Further, we examine how different treatments -- explanations, model bias disclosure and proxy correlation disclosure -- affect fairness perception and parity. We find that explanations help people detect direct biases but not indirect biases. Additionally, regardless of bias type, explanations tend to increase agreement with model biases. Disclosures can help mitigate this effect for indirect biases, improving both unfairness recognition and the decision-making fairness. We hope that our findings can help guide further research into advancing explanations in support of fair human-AI decision-making

Laparoscopic Assisted Fusion of the Lumbosacral Spine: A Biomechanical and Histologic Analysis of the Open Versus Laparoscopic Technique in an Animal Model

Study Design. An animal model for laparoscopic lumbosacral fusion. Objectives. To compare the biomechanical and histologic results of open to laparoscopic lumbosacral discectomy and fusion in an animal model. Background Data. Early clinical reports of laparoscopic lumbosacral fusions are encouraging, but animal experiments have not been reported. Methods. Ten pigs (50-80 kg) were divided into two groups. Group 1 underwent an open anterior lumbosacral discectomy and fusion at L7-S1 using autologous bone graft and a titanium MOSS (DePuy Motech) cage. Group 2 was identical to Group 1 except that a laparoscopic technique was used. The animals were killed at 3 months, and the lumbosacral spines were harvested for biomechanical and histologic testing. Results. Estimated blood loss and average length of operation, respectively, for the two groups were: Group 1, 50 mL, 2 hours 50 minutes; and Group 2, 40 mL, 3 hours 40 minutes. There were no perioperative or postoperative complications in either group. Motion analysis results showed less motion in lateral bending, flexion, and extension than in the intact specimen in both groups. Tensile testing showed that the stiffness was significantly greater in the open group than in the laparoscopic group (P \u3c 0.004). Histologic examination showed a less extensive discectomy and less bone growth in the implant in the laparoscopic group. Inadequate decortication of end-plates occurred in two animals who underwent laparoscopy. Conclusions. Although lumbosacral discectomy and implant insertion can be performed using the laparoscopic technique, the construct may not have the same biomechanical strength as that attained with the open procedure. Laparoscopic-assisted lumbosacral fusion surgery requires additional investigation before it is widely used in clinical situations