Transfer and Multi-Task Learning for Noun-Noun Compound Interpretation

In this paper, we empirically evaluate the utility of transfer and multi-task learning on a challenging semantic classification task: semantic interpretation of noun--noun compounds. Through a comprehensive series of experiments and in-depth error analysis, we show that transfer learning via parameter initialization and multi-task learning via parameter sharing can help a neural classification model generalize over a highly skewed distribution of relations. Further, we demonstrate how dual annotation with two distinct sets of relations over the same set of compounds can be exploited to improve the overall accuracy of a neural classifier and its F1 scores on the less frequent, but more difficult relations.Comment: EMNLP 2018: Conference on Empirical Methods in Natural Language Processing (EMNLP

What's in a compound? Review article on Lieber and Štekauer (eds) 2009. 'The Oxford Handbook of Compounding'

The Oxford Handbook of Compounding surveys a variety of theoretical and descriptive issues, presenting overviews of compounding in a number of frameworks and sketches of compounding in a number of languages. Much of the book deals with Germanic noun–noun compounding. I take up some of the theoretical questions raised surrounding such constructions, in particular, the notion of attributive modification in noun-headed compounds. I focus on two issues. The first is the semantic relation between the head noun and its nominal modifier. Several authors repeat the argument that there is a small(-ish) fixed number of general semantic relations in noun–noun compounds (‘Lees's solution’), but I argue that the correct way to look at such compounds is what I call ‘Downing's solution’, in which we assume that the relation is specified pragmatically, and hence could be any relation at all. The second issue is the way that adjectives modify nouns inside compounds. Although there are languages in which compounded adjectives modify just as they do in phrases (Chukchee, Arleplog Swedish), in general the adjective has a classifier role and not that of a compositional attributive modifier. Thus, even if an English (or German) adjective–noun compound looks compositional, it isn't

Decorrelation and shallow semantic patterns for distributional clustering of nouns and verbs

Distributional approximations to lexical semantics are very useful not only in helping the creation of lexical semantic resources (Kilgariff et al., 2004; Snow et al., 2006), but also when directly applied in tasks that can benefit from large-coverage semantic knowledge such as coreference resolution (Poesio et al., 1998; Gasperin and Vieira, 2004; Versley, 2007), word sense disambiguation (Mc- Carthy et al., 2004) or semantical role labeling (Gordon and Swanson, 2007). We present a model that is built from Webbased corpora using both shallow patterns for grammatical and semantic relations and a window-based approach, using singular value decomposition to decorrelate the feature space which is otherwise too heavily influenced by the skewed topic distribution of Web corpora

Integrative priming occurs rapidly and uncontrollably during lexical processing

Lexical priming, whereby a prime word facilitates recognition of a related target word (e.g., nurse ? doctor), is typically attributed to association strength, semantic similarity, or compound familiarity. Here, the authors demonstrate a novel type of lexical priming that occurs among unassociated, dissimilar, and unfamiliar concepts (e.g., horse ? doctor). Specifically, integrative priming occurs when a prime word can be easily integrated with a target word to create a unitary representation. Across several manipulations of timing (stimulus onset asynchrony) and list context (relatedness proportion), lexical decisions for the target word were facilitated when it could be integrated with the prime word. Moreover, integrative priming was dissociated from both associative priming and semantic priming but was comparable in terms of both prevalence (across participants) and magnitude (within participants). This observation of integrative priming challenges present models of lexical priming, such as spreading activation, distributed representation, expectancy, episodic retrieval, and compound cue models. The authors suggest that integrative priming may be explained by a role activation model of relational integration

Distributional composition using higher-order dependency vectors

This paper concerns how to apply compositional methods to vectors based on grammatical dependency relation vectors. We demonstrate the potential of a novel approach which uses higher-order grammatical dependency relations as features. We apply the approach to adjective-noun compounds with promising results in the prediction of the vectors for (held-out) observed phrases