9 research outputs found
Graph clustering for natural language processing
Graph-based representations are proven to be an effective approach for a variety of Natural Language Processing (NLP) tasks. Graph clustering makes it possible to extract useful knowledge by exploiting the implicit structure of the data. In this tutorial, we will present several efficient graph clustering algorithms, show their strengths and weaknesses as well as their implementations and applications. Then, the evaluation methodology in unsupervised NLP tasks will be discussed
Combined distributional and logical semantics
Understanding natural language sentences requires interpreting words, and combining
the meanings of words into the meanings of sentences. Despite much work on lexical
and compositional semantics individually, existing approaches are unlikely to offer a
complete solution. This thesis introduces a new approach, which combines the benefits
of distributional lexical semantics and logical compositional semantics.
Linguistic theories of compositional semantics have shown how logical forms can
be built for sentences, and how to represent semantic operators such as negatives,
quantifiers and modals. However, computational implementations of such theories
have shown poor performance on applications, mainly due to a reliance on incomplete
hand-built ontologies for the meanings of content words. Conversely, distributional semantics
has been shown to be effective in learning the representations of content words
based on collocations in large unlabelled corpora, but there are major outstanding challenges
in representing function words and building representations for sentences.
I introduce a new model which captures the main advantages of logical and distributional
approaches. The proposal closely follows formal semantics, except for changing
the definitions of content words. In traditional formal semantics, each word would
express a different symbol. Instead, I allow multiple words to express the same symbol,
corresponding to underlying concepts. For example, both the verb write and the noun
author can be made to express the same relation. These symbols can be learnt by clustering
symbols based on distributional statisticsâfor example, write and author will
share many similar arguments. Crucially, the clustering means that the representations
are symbolic, so can easily be incorporated into standard logical approaches.
The simple model proves insufficient, and I develop several extensions. I develop
an unsupervised probabilistic model of ambiguity, and show how this model can be
built into compositional derivations to produce a distribution over logical forms. The
flat clustering approach does not model relations between concepts, for example that
buying implies owning. Instead, I show how to build graph structures over the clusters,
which allows such inferences. I also explore if the abstract concepts can be generalized
cross-lingually, for example mapping French verb ecrire to the same cluster as
the English verb write. The systems developed show good performance on question
answering and entailment tasks, and are capable of both sophisticated multi-sentence
inferences involving quantifiers, and subtle reasoning about lexical semantics.
These results show that distributional and formal logical semantics are not mutually
exclusive, and that a combined model can be built that captures the advantages of each