CausaLM: Causal Model Explanation Through Counterfactual Language Models

Understanding predictions made by deep neural networks is notoriously difficult, but also crucial to their dissemination. As all ML-based methods, they are as good as their training data, and can also capture unwanted biases. While there are tools that can help understand whether such biases exist, they do not distinguish between correlation and causation, and might be ill-suited for text-based models and for reasoning about high level language concepts. A key problem of estimating the causal effect of a concept of interest on a given model is that this estimation requires the generation of counterfactual examples, which is challenging with existing generation technology. To bridge that gap, we propose CausaLM, a framework for producing causal model explanations using counterfactual language representation models. Our approach is based on fine-tuning of deep contextualized embedding models with auxiliary adversarial tasks derived from the causal graph of the problem. Concretely, we show that by carefully choosing auxiliary adversarial pre-training tasks, language representation models such as BERT can effectively learn a counterfactual representation for a given concept of interest, and be used to estimate its true causal effect on model performance. A byproduct of our method is a language representation model that is unaffected by the tested concept, which can be useful in mitigating unwanted bias ingrained in the data.Comment: Our code and data are available at: https://amirfeder.github.io/CausaLM/ Under review for the Computational Linguistics journa

Setting Allocations and Prices in Auctions

This disclosure describes techniques to rank advertisers in an auction, such as an auction for advertisements delivered over the Internet. Advertisers that participate in the auction are classified into two types. The first type includes advertisers that make use of remarketing lists, or other valuable information. The second type includes advertisers that do not use remarketing lists. Bids from advertisers of the first type are adjusted to a virtual bid prior to conducting the ad auction. Bidders are ranked on the basis of their bids. Prices that the winning bidders pay, e.g., price per click, are adjusted by a scaling factor. Incorporating such techniques in ad auctions can enable greater revenue for advertising networks and content owners

Environmental variability and modularity of bacterial metabolic networks

<p>Abstract</p> <p>Background</p> <p>Biological systems are often modular: they can be decomposed into nearly-independent structural units that perform specific functions. The evolutionary origin of modularity is a subject of much current interest. Recent theory suggests that modularity can be enhanced when the environment changes over time. However, this theory has not yet been tested using biological data.</p> <p>Results</p> <p>To address this, we studied the relation between environmental variability and modularity in a natural and well-studied system, the metabolic networks of bacteria. We classified 117 bacterial species according to the degree of variability in their natural habitat. We find that metabolic networks of organisms in variable environments are significantly more modular than networks of organisms that evolved under more constant conditions.</p> <p>Conclusion</p> <p>This study supports the view that variability in the natural habitat of an organism promotes modularity in its metabolic network and perhaps in other biological systems.</p

Bid Optimization in Broad-Match Ad auctions

Ad auctions in sponsored search support ``broad match'' that allows an advertiser to target a large number of queries while bidding only on a limited number. While giving more expressiveness to advertisers, this feature makes it challenging to optimize bids to maximize their returns: choosing to bid on a query as a broad match because it provides high profit results in one bidding for related queries which may yield low or even negative profits. We abstract and study the complexity of the {\em bid optimization problem} which is to determine an advertiser's bids on a subset of keywords (possibly using broad match) so that her profit is maximized. In the query language model when the advertiser is allowed to bid on all queries as broad match, we present an linear programming (LP)-based polynomial-time algorithm that gets the optimal profit. In the model in which an advertiser can only bid on keywords, ie., a subset of keywords as an exact or broad match, we show that this problem is not approximable within any reasonable approximation factor unless P=NP. To deal with this hardness result, we present a constant-factor approximation when the optimal profit significantly exceeds the cost. This algorithm is based on rounding a natural LP formulation of the problem. Finally, we study a budgeted variant of the problem, and show that in the query language model, one can find two budget constrained ad campaigns in polynomial time that implement the optimal bidding strategy. Our results are the first to address bid optimization under the broad match feature which is common in ad auctions.Comment: World Wide Web Conference (WWW09), 10 pages, 2 figure

The environmental footprint of Holocene societies: a multi-temporal study of trails in the Judean Desert, Israel

The global distribution of footpaths and their inferred antiquity implies that they are widespread spatial and temporal anthropogenic landscape units. Arid environments are of special interest for investigating historically used footpaths, as older routes may preserve better due to minimal modern impact and slower pedogenic processes. Here we examine footpaths in the Judean Desert of the southern Levant, a human hotspot throughout the Holocene. We studied one modern and two archaeological footpaths (one attributed to the Early Bronze Age and one to the Roman period) using micromorphology, bulk samples laboratory analysis, and remote sensing. Field observations and color analysis indicate that footpaths in the studied arid limestone environment can result in brighter surface color than their non-path surroundings. Similar color changes are reflected using both laboratory analysis and high-resolution remote sensing, where the difference is also significant. Microscopically, the footpaths studied tend to be less porous and with fewer biogenic activities when compared to their non-path controls. However, the two ancient footpaths studied do exhibit minimal indicators of biogenic activities that are not detectable in the modern footpath sample. Our study shows that high-resolution remote sensing coupled with micromorphology, while using appropriate local modern analogies, can help to locate and assess both the environmental effect and the antiquity of footpaths

Nanoscale imaging of equilibrium quantum Hall edge currents and of the magnetic monopole response in graphene

The recently predicted topological magnetoelectric effect and the response to an electric charge that mimics an induced mirror magnetic monopole are fundamental attributes of topological states of matter with broken time reversal symmetry. Using a SQUID-on-tip, acting simultaneously as a tunable scanning electric charge and as ultrasensitive nanoscale magnetometer, we induce and directly image the microscopic currents generating the magnetic monopole response in a graphene quantum Hall electron system. We find a rich and complex nonlinear behavior governed by coexistence of topological and nontopological equilibrium currents that is not captured by the monopole models. Furthermore, by utilizing a tuning fork that induces nanoscale vibrations of the SQUID-on-tip, we directly image the equilibrium currents of individual quantum Hall edge states for the first time. We reveal that the edge states that are commonly assumed to carry only a chiral downstream current, in fact carry a pair of counterpropagating currents, in which the topological downstream current in the incompressible region is always counterbalanced by heretofore unobserved nontopological upstream current flowing in the adjacent compressible region. The intricate patterns of the counterpropagating equilibrium-state orbital currents provide new insights into the microscopic origins of the topological and nontopological charge and energy flow in quantum Hall systems

Two Biexciton Types Coexisting in Coupled Quantum Dot Molecules

Coupled colloidal quantum dot molecules (CQDMs) are an emerging class of nanomaterials, manifesting two coupled emission centers and thus introducing additional degrees of freedom for designing quantum-dot-based technologies. The properties of multiply excited states in these CQDMs are crucial to their performance as quantum light emitters, but they cannot be fully resolved by existing spectroscopic techniques. Here we study the characteristics of biexcitonic species, which represent a rich landscape of different configurations essentially categorized as either segregated or localized biexciton states. To this end, we introduce an extension of Heralded Spectroscopy to resolve the different biexciton species in the prototypical CdSe/CdS CQDM system. By comparing CQDMs with single quantum dots and with nonfused quantum dot pairs, we uncover the coexistence and interplay of two distinct biexciton species: A fast-decaying, strongly interacting biexciton species, analogous to biexcitons in single quantum dots, and a long-lived, weakly interacting species corresponding to two nearly independent excitons. The two biexciton types are consistent with numerical simulations, assigning the strongly interacting species to two excitons localized at one side of the quantum dot molecule and the weakly interacting species to excitons segregated to the two quantum dot molecule sides. This deeper understanding of multiply excited states in coupled quantum dot molecules can support the rational design of tunable single- or multiple-photon quantum emitters.U.B. and D.O. acknowledge the support of the Israel Science Foundation (ISF) and the Directorate for Defense Research and Development (DDR&D), grant No. 3415/21. J.I.C. and J.P. acknowledge support from UJI project B-2021-06. E.S., A.L., Y.E.P., and Y.O. acknowledge support from the Hebrew University Center for Nanoscience and Nanotechnology

The environmental footprint of Holocene societies: a multi-temporal study of trails in the Judean Desert, Israel

The global distribution of footpaths and their inferred antiquity implies that they are widespread spatial and temporal anthropogenic landscape units. Arid environments are of special interest for investigating historically used footpaths, as older routes may preserve better due to minimal modern impact and slower pedogenic processes. Here we examine footpaths in the Judean Desert of the southern Levant, a human hotspot throughout the Holocene. We studied one modern and two archaeological footpaths (one attributed to the Early Bronze Age and one to the Roman period) using micromorphology, bulk samples laboratory analysis, and remote sensing. Field observations and color analysis indicate that footpaths in the studied arid limestone environment can result in brighter surface color than their non-path surroundings. Similar color changes are reflected using both laboratory analysis and high-resolution remote sensing, where the difference is also significant. Microscopically, the footpaths studied tend to be less porous and with fewer biogenic activities when compared to their non-path controls. However, the two ancient footpaths studied do exhibit minimal indicators of biogenic activities that are not detectable in the modern footpath sample. Our study shows that high-resolution remote sensing coupled with micromorphology, while using appropriate local modern analogies, can help to locate and assess both the environmental effect and the antiquity of footpaths

An Automated Machine Learning-based Model Predicts Postoperative Mortality Using Readily-Extractable Preoperative Electronic Health Record Data

Background Rapid, preoperative identification of patients with the highest risk for medical complications is necessary to ensure that limited infrastructure and human resources are directed towards those most likely to benefit. Existing risk scores either lack specificity at the patient level or utilise the American Society of Anesthesiologists (ASA) physical status classification, which requires a clinician to review the chart. Methods We report on the use of machine learning algorithms, specifically random forests, to create a fully automated score that predicts postoperative in-hospital mortality based solely on structured data available at the time of surgery. Electronic health record data from 53 097 surgical patients (2.01% mortality rate) who underwent general anaesthesia between April 1, 2013 and December 10, 2018 in a large US academic medical centre were used to extract 58 preoperative features. Results Using a random forest classifier we found that automatically obtained preoperative features (area under the curve [AUC] of 0.932, 95% confidence interval [CI] 0.910–0.951) outperforms Preoperative Score to Predict Postoperative Mortality (POSPOM) scores (AUC of 0.660, 95% CI 0.598–0.722), Charlson comorbidity scores (AUC of 0.742, 95% CI 0.658–0.812), and ASA physical status (AUC of 0.866, 95% CI 0.829–0.897). Including the ASA physical status with the preoperative features achieves an AUC of 0.936 (95% CI 0.917–0.955). Conclusions This automated score outperforms the ASA physical status score, the Charlson comorbidity score, and the POSPOM score for predicting in-hospital mortality. Additionally, we integrate this score with a previously published postoperative score to demonstrate the extent to which patient risk changes during the perioperative period

Coarse-Graining and Self-Dissimilarity of Complex Networks

Can complex engineered and biological networks be coarse-grained into smaller and more understandable versions in which each node represents an entire pattern in the original network? To address this, we define coarse-graining units (CGU) as connectivity patterns which can serve as the nodes of a coarse-grained network, and present algorithms to detect them. We use this approach to systematically reverse-engineer electronic circuits, forming understandable high-level maps from incomprehensible transistor wiring: first, a coarse-grained version in which each node is a gate made of several transistors is established. Then, the coarse-grained network is itself coarse-grained, resulting in a high-level blueprint in which each node is a circuit-module made of multiple gates. We apply our approach also to a mammalian protein-signaling network, to find a simplified coarse-grained network with three main signaling channels that correspond to cross-interacting MAP-kinase cascades. We find that both biological and electronic networks are 'self-dissimilar', with different network motifs found at each level. The present approach can be used to simplify a wide variety of directed and nondirected, natural and designed networks.Comment: 11 pages, 11 figure