5 research outputs found
Focusing Knowledge-based Graph Argument Mining via Topic Modeling
Decision-making usually takes five steps: identifying the problem, collecting
data, extracting evidence, identifying pro and con arguments, and making
decisions. Focusing on extracting evidence, this paper presents a hybrid model
that combines latent Dirichlet allocation and word embeddings to obtain
external knowledge from structured and unstructured data. We study the task of
sentence-level argument mining, as arguments mostly require some degree of
world knowledge to be identified and understood. Given a topic and a sentence,
the goal is to classify whether a sentence represents an argument in regard to
the topic. We use a topic model to extract topic- and sentence-specific
evidence from the structured knowledge base Wikidata, building a graph based on
the cosine similarity between the entity word vectors of Wikidata and the
vector of the given sentence. Also, we build a second graph based on
topic-specific articles found via Google to tackle the general incompleteness
of structured knowledge bases. Combining these graphs, we obtain a graph-based
model which, as our evaluation shows, successfully capitalizes on both
structured and unstructured data
Informing unsupervised pretraining with external linguistic knowledge
Unsupervised pretraining models have been shown to facilitate a wide range of downstream applications. These models, however, still encode only the distributional knowledge, incorporated through language modeling objectives. In this work, we complement the encoded distributional knowledge with external lexical knowledge. We generalize the recently proposed (state-of-the-art) unsupervised pretraining model BERT to a multi-task learning setting: we couple BERT's masked language modeling and next sentence prediction objectives with the auxiliary binary word relation classification, through which we inject clean linguistic knowledge into the model. Our initial experiments suggest that our "linguistically-informed" BERT (LIBERT) yields performance gains over the linguistically-blind "vanilla" BERT on several language understanding tasks
Informing unsupervised pretraining with external linguistic knowledge
Unsupervised pretraining models have been shown to facilitate a wide range of downstream applications. These models, however, still encode only the distributional knowledge, incorporated through language modeling objectives. In this work, we complement the encoded distributional knowledge with external lexical knowledge. We generalize the recently proposed (state-of-the-art) unsupervised pretraining model BERT to a multi-task learning setting: we couple BERT's masked language modeling and next sentence prediction objectives with the auxiliary binary word relation classification, through which we inject clean linguistic knowledge into the model. Our initial experiments suggest that our "linguistically-informed" BERT (LIBERT) yields performance gains over the linguistically-blind "vanilla" BERT on several language understanding tasks