Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

A fundamental goal of scientific research is to learn about causal relationships. However, despite its critical role in the life and social sciences, causality has not had the same importance in Natural Language Processing (NLP), which has traditionally placed more emphasis on predictive tasks. This distinction is beginning to fade, with an emerging area of interdisciplinary research at the convergence of causal inference and language processing. Still, research on causality in NLP remains scattered across domains without unified definitions, benchmark datasets and clear articulations of the challenges and opportunities in the application of causal inference to the textual domain, with its unique properties. In this survey, we consolidate research across academic areas and situate it in the broader NLP landscape. We introduce the statistical challenge of estimating causal effects with text, encompassing settings where text is used as an outcome, treatment, or to address confounding. In addition, we explore potential uses of causal inference to improve the robustness, fairness, and interpretability of NLP models. We thus provide a unified overview of causal inference for the NLP community.Comment: Accepted to Transactions of the Association for Computational Linguistics (TACL

A Survey on Bayesian Deep Learning

A comprehensive artificial intelligence system needs to not only perceive the environment with different `senses' (e.g., seeing and hearing) but also infer the world's conditional (or even causal) relations and corresponding uncertainty. The past decade has seen major advances in many perception tasks such as visual object recognition and speech recognition using deep learning models. For higher-level inference, however, probabilistic graphical models with their Bayesian nature are still more powerful and flexible. In recent years, Bayesian deep learning has emerged as a unified probabilistic framework to tightly integrate deep learning and Bayesian models. In this general framework, the perception of text or images using deep learning can boost the performance of higher-level inference and in turn, the feedback from the inference process is able to enhance the perception of text or images. This survey provides a comprehensive introduction to Bayesian deep learning and reviews its recent applications on recommender systems, topic models, control, etc. Besides, we also discuss the relationship and differences between Bayesian deep learning and other related topics such as Bayesian treatment of neural networks.Comment: To appear in ACM Computing Surveys (CSUR) 202

Causality Is Logically Definable-Toward an Equilibrium-Based Computing Paradigm of Quantum Agents and Quantum Intelligence (QAQI)

A survey on agents, causality and intelligence is presented and an equilibrium-based computing paradigm of quantum agents and quantum intelligence (QAQI) is proposed. In the survey, Aristotle’s causality principle and its historical extensions by David Hume, Bertrand Russell, Lotfi Zadeh, Donald Rubin, Judea Pearl, Niels Bohr, Albert Einstein, David Bohm, and the causal set initiative are reviewed; bipolar dynamic logic (BDL) is introduced as a causal logic for bipolar inductive and deductive reasoning; bipolar quantum linear algebra (BQLA) is introduced as a causal algebra for quantum agent interaction and formation. Despite the widely held view that causality is undefinable with regularity, it is shown that equilibrium-based bipolar causality is logically definable using BDL and BQLA for causal inference in physical, social, biological, mental, and philosophical terms. This finding leads to the paradigm of QAQI where agents are modeled as quantum ensembles; intelligence is revealed as quantum intelligence. It is shown that the ensembles formation, mutation and interaction of agents can be described as direct or indirect results of quantum causality. Some fundamental laws of causation are presented for quantum agent entanglement and quantum intelligence. Applicability is illustrated; major challenges are identified in equilibrium based causal inference and quantum data mining

Causal Inference on Distribution Functions

Understanding causal relationships is one of the most important goals of modern science. So far, the causal inference literature has focused almost exclusively on outcomes coming from the Euclidean space $\mathbb{R}^p$. However, it is increasingly common that complex datasets collected through electronic sources, such as wearable devices, cannot be represented as data points from $\mathbb{R}^p$. In this paper, we present a novel framework of causal effects for outcomes from the Wasserstein space of cumulative distribution functions, which in contrast to the Euclidean space, is non-linear. We develop doubly robust estimators and associated asymptotic theory for these causal effects. As an illustration, we use our framework to quantify the causal effect of marriage on physical activity patterns using wearable device data collected through the National Health and Nutrition Examination Survey

Kingmakers or Cheerleaders? Party Power and the Causal Effects of Endorsements

When parties make endorsements in primary elections, does the favored candidate receive a real boost in his or her vote share, or do parties simply pick the favorites who are already destined to win? To answer this question, we draw on two research designs aimed at isolating the causal effect of Democratic Party endorsements in California’s 2012 primary election. First, we conduct a survey experiment in which we randomly assign a party endorsement, holding all other aspects of a candidate’s background and policy positions constant. Second, we use a unique dataset to implement a regression discontinuity analysis of electoral trends by comparing the vote shares captured by candidates who barely won or barely lost the internal party endorsement contest. We find a constellation of evidence suggesting that endorsements do indeed matter, although this effect appears to be contingent upon the type of candidate and voter: endorsements matter most for candidates in their party’s mainstream, and for voters who identify with that party and for independents. The magnitude of their impact is dramatically smaller than might be estimated from research designs less attuned to recent advances in causal inference

Opium for the Masses: How Foreign Free Media Can Stabilize Authoritarian Regimes

A common claim in the democratization literature is that foreign free media undermine authoritarian rule. No reliable micro-level evidence on this topic exists, however, since independent survey research is rarely possible in authoritarian regimes and self-selection into media consumption complicates causal inferences. In this case study of the impact of West German television on political attitudes in communist East Germany, we address these problems by making use of previously secret survey data and a natural experiment. While most East Germans were able to tune in to West German broadcasts, some of them were cut off from West German television due to East Germany's topography. We exploit this plausibly exogenous variation to estimate the impact of West German television on East Germans' political attitudes using instrumental variable estimators. Contrary to conventional wisdom, East Germans who watched West German television were more satisfied with life in East Germany and the communist regime. To explain this surprising finding, we demonstrate that West German television's role in transmitting political information not available in the state-controlled communist media was insignificant and that television primarily served as a means of entertainment for East Germans. Archival material on the reaction of the East German regime to the availability of West German television corroborates our argument.instrumental variables; causal inference; local average response function; media effects; East Germany; democratization