39 research outputs found
Lambek vs. Lambek: Functorial Vector Space Semantics and String Diagrams for Lambek Calculus
The Distributional Compositional Categorical (DisCoCat) model is a
mathematical framework that provides compositional semantics for meanings of
natural language sentences. It consists of a computational procedure for
constructing meanings of sentences, given their grammatical structure in terms
of compositional type-logic, and given the empirically derived meanings of
their words. For the particular case that the meaning of words is modelled
within a distributional vector space model, its experimental predictions,
derived from real large scale data, have outperformed other empirically
validated methods that could build vectors for a full sentence. This success
can be attributed to a conceptually motivated mathematical underpinning, by
integrating qualitative compositional type-logic and quantitative modelling of
meaning within a category-theoretic mathematical framework.
The type-logic used in the DisCoCat model is Lambek's pregroup grammar.
Pregroup types form a posetal compact closed category, which can be passed, in
a functorial manner, on to the compact closed structure of vector spaces,
linear maps and tensor product. The diagrammatic versions of the equational
reasoning in compact closed categories can be interpreted as the flow of word
meanings within sentences. Pregroups simplify Lambek's previous type-logic, the
Lambek calculus, which has been extensively used to formalise and reason about
various linguistic phenomena. The apparent reliance of the DisCoCat on
pregroups has been seen as a shortcoming. This paper addresses this concern, by
pointing out that one may as well realise a functorial passage from the
original type-logic of Lambek, a monoidal bi-closed category, to vector spaces,
or to any other model of meaning organised within a monoidal bi-closed
category. The corresponding string diagram calculus, due to Baez and Stay, now
depicts the flow of word meanings.Comment: 29 pages, pending publication in Annals of Pure and Applied Logi
Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics
This thesis is about the problem of compositionality in distributional
semantics. Distributional semantics presupposes that the meanings of words are
a function of their occurrences in textual contexts. It models words as
distributions over these contexts and represents them as vectors in high
dimensional spaces. The problem of compositionality for such models concerns
itself with how to produce representations for larger units of text by
composing the representations of smaller units of text.
This thesis focuses on a particular approach to this compositionality
problem, namely using the categorical framework developed by Coecke, Sadrzadeh,
and Clark, which combines syntactic analysis formalisms with distributional
semantic representations of meaning to produce syntactically motivated
composition operations. This thesis shows how this approach can be
theoretically extended and practically implemented to produce concrete
compositional distributional models of natural language semantics. It
furthermore demonstrates that such models can perform on par with, or better
than, other competing approaches in the field of natural language processing.
There are three principal contributions to computational linguistics in this
thesis. The first is to extend the DisCoCat framework on the syntactic front
and semantic front, incorporating a number of syntactic analysis formalisms and
providing learning procedures allowing for the generation of concrete
compositional distributional models. The second contribution is to evaluate the
models developed from the procedures presented here, showing that they
outperform other compositional distributional models present in the literature.
The third contribution is to show how using category theory to solve linguistic
problems forms a sound basis for research, illustrated by examples of work on
this topic, that also suggest directions for future research.Comment: DPhil Thesis, University of Oxford, Submitted and accepted in 201
Towards Functorial Language-Games
In categorical compositional semantics of natural language one studies
functors from a category of grammatical derivations (such as a Lambek pregroup)
to a semantic category (such as real vector spaces). We compositionally build
game-theoretic semantics of sentences by taking the semantic category to be the
category whose morphisms are open games. This requires some modifications to
the grammar category to compensate for the failure of open games to form a
compact closed category. We illustrate the theory using simple examples of
Wittgenstein's language-games.Comment: In Proceedings CAPNS 2018, arXiv:1811.0270
A Context-theoretic Framework for Compositionality in Distributional Semantics
Techniques in which words are represented as vectors have proved useful in
many applications in computational linguistics, however there is currently no
general semantic formalism for representing meaning in terms of vectors. We
present a framework for natural language semantics in which words, phrases and
sentences are all represented as vectors, based on a theoretical analysis which
assumes that meaning is determined by context.
In the theoretical analysis, we define a corpus model as a mathematical
abstraction of a text corpus. The meaning of a string of words is assumed to be
a vector representing the contexts in which it occurs in the corpus model.
Based on this assumption, we can show that the vector representations of words
can be considered as elements of an algebra over a field. We note that in
applications of vector spaces to representing meanings of words there is an
underlying lattice structure; we interpret the partial ordering of the lattice
as describing entailment between meanings. We also define the context-theoretic
probability of a string, and, based on this and the lattice structure, a degree
of entailment between strings.
We relate the framework to existing methods of composing vector-based
representations of meaning, and show that our approach generalises many of
these, including vector addition, component-wise multiplication, and the tensor
product.Comment: Submitted to Computational Linguistics on 20th January 2010 for
revie
Self-move and Other-move: Quantum Categorical Foundations of Japanese
The purpose of this work is to contribute toward the larger goal of creating
a Quantum Natural Language Processing (QNLP) translator program. This work
contributes original diagrammatic representations of the Japanese language
based on prior work that accomplished on the English language based on category
theory. The germane differences between the English and Japanese languages are
emphasized to help address English language bias in the current body of
research. Additionally, topological principles of these diagrams and many
potential avenues for further research are proposed. Why is this endeavor
important? Hundreds of languages have developed over the course of millennia
coinciding with the evolution of human interaction across time and geographic
location. These languages are foundational to human survival, experience,
flourishing, and living the good life. They are also, however, the strongest
barrier between people groups. Over the last several decades, advancements in
Natural Language Processing (NLP) have made it easier to bridge the gap between
individuals who do not share a common language or culture. Tools like Google
Translate and DeepL make it easier than ever before to share our experiences
with people globally. Nevertheless, these tools are still inadequate as they
fail to convey our ideas across the language barrier fluently, leaving people
feeling anxious and embarrassed. This is particularly true of languages born
out of substantially different cultures, such as English and Japanese. Quantum
computers offer the best chance to achieve translation fluency in that they are
better suited to simulating the natural world and natural phenomenon such as
natural speech.
Keywords: category theory, DisCoCat, DisCoCirc, Japanese grammar, English
grammar, translation, topology, Quantum Natural Language Processing, Natural
Language ProcessingComment: 104 pages; 31 figures; 9 table