Causally Denoise Word Embeddings Using Half-Sibling Regression

Distributional representations of words, also known as word vectors, have become crucial for modern natural language processing tasks due to their wide applications. Recently, a growing body of word vector postprocessing algorithm has emerged, aiming to render off-the-shelf word vectors even stronger. In line with these investigations, we introduce a novel word vector postprocessing scheme under a causal inference framework. Concretely, the postprocessing pipeline is realized by Half-Sibling Regression (HSR), which allows us to identify and remove confounding noise contained in word vectors. Compared to previous work, our proposed method has the advantages of interpretability and transparency due to its causal inference grounding. Evaluated on a battery of standard lexical-level evaluation tasks and downstream sentiment analysis tasks, our method reaches state-of-the-art performance.Comment: Accepted by AAAI 202

A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations

Word embedding has become essential for natural language processing as it boosts empirical performances of various tasks. However, recent research discovers that gender bias is incorporated in neural word embeddings, and downstream tasks that rely on these biased word vectors also produce gender-biased results. While some word-embedding gender-debiasing methods have been developed, these methods mainly focus on reducing gender bias associated with gender direction and fail to reduce the gender bias presented in word embedding relations. In this paper, we design a causal and simple approach for mitigating gender bias in word vector relation by utilizing the statistical dependency between gender-definition word embeddings and gender-biased word embeddings. Our method attains state-of-the-art results on gender-debiasing tasks, lexical- and sentence-level evaluation tasks, and downstream coreference resolution tasks.Comment: Accepted by AAAI 202

A Hybrid SIE-PDE Formulation Without Boundary Condition Requirement for Transverse Magnetic Electromagnetic Analysis

A hybrid surface integral equation partial differential equation (SIE-PDE) formulation without the boundary condition requirement is proposed to solve the transverse magnetic (TM) electromagnetic problems. In the proposed formulation, the computational domain is decomposed into two overlapping domains: the SIE and PDE domains. In the SIE domain, complex structures with piecewise homogeneous media, e.g., highly conductive media, are included. An equivalent model for those structures is constructed by replacing them with the background medium and introducing a surface equivalent electric current density on an enclosed boundary to represent their electromagnetic effects. The remaining computational domain and homogeneous background medium replaced domain consist of the PDE domain, in which inhomogeneous or non-isotropic media are included. Through combining the surface equivalent electric current density and the inhomogeneous Helmholtz equation, a hybrid SIE-PDE formulation is derived. It requires no boundary conditions, and is mathematically equivalent to the original physical model. Through careful construction of basis functions to expand electric fields and the equivalent current density, the discretized formulation is made compatible with the SIE and PDE domain interface. The accuracy and efficiency are validated through two numerical examples. Results show that the proposed SIE-PDE formulation can obtain accurate results, and significant performance improvements in terms of CPU time and memory consumption compared with the FEM are achieved