"Not not bad" is not "bad": A distributional account of negation

With the increasing empirical success of distributional models of compositional semantics, it is timely to consider the types of textual logic that such models are capable of capturing. In this paper, we address shortcomings in the ability of current models to capture logical operations such as negation. As a solution we propose a tripartite formulation for a continuous vector space representation of semantics and subsequently use this representation to develop a formal compositional notion of negation within such models.Comment: 9 pages, to appear in Proceedings of the 2013 Workshop on Continuous Vector Space Models and their Compositionalit

Vector-based Approach to Verbal Cognition

Human verbal thinking is an object of many multidisciplinary studies Verbal cognition is often an integration of complex mental activities such as neurocognitive and psychological processes In neuro-cognitive study of language neural architecture and neuropsychological mechanism of verbal cognition are basis of a vector based modeling Human mental states as constituents of mental continuum represent an infinite set of meanings Number of meanings is not limited but numbers of words and rules that are used for building complex verbal structures are limited Verbal perception and interpretation of the multiple meanings and propositions in mental continuum can be modeled by applying tensor methods A comparison of human mental space to a vector space is an effective way of analyzing of human semantic vocabulary mental representations and rules of clustering and mapping As such Euclidean and non-Euclidean spaces can be applied for a description of human semantic vocabulary and high order Additionally changes in semantics and structures can be analyzed in 3D and other dimensional spaces It is suggested that different forms of verbal representation should be analyzed in a light of vector tensor transformations Vector dot and cross product covariance and contra variance have been applied to analysis of semantic transformations and pragmatic change in high order syntax structures These ideas are supported by empirical data from typologically different languages such as Mongolian English and Russian Moreover the author argues that the vectorbased approach to cognitive linguistics offers new opportunities to develop an alternative version of quantitative semantics and thus to extend theory of Universal grammar in new dimension

Multilingual Models for Compositional Distributed Semantics

We present a novel technique for learning semantic representations, which extends the distributional hypothesis to multilingual data and joint-space embeddings. Our models leverage parallel data and learn to strongly align the embeddings of semantically equivalent sentences, while maintaining sufficient distance between those of dissimilar sentences. The models do not rely on word alignments or any syntactic information and are successfully applied to a number of diverse languages. We extend our approach to learn semantic representations at the document level, too. We evaluate these models on two cross-lingual document classification tasks, outperforming the prior state of the art. Through qualitative analysis and the study of pivoting effects we demonstrate that our representations are semantically plausible and can capture semantic relationships across languages without parallel data.Comment: Proceedings of ACL 2014 (Long papers

Experimental Support for a Categorical Compositional Distributional Model of Meaning

Modelling compositional meaning for sentences using empirical distributional methods has been a challenge for computational linguists. We implement the abstract categorical model of Coecke et al. (arXiv:1003.4394v1 [cs.CL]) using data from the BNC and evaluate it. The implementation is based on unsupervised learning of matrices for relational words and applying them to the vectors of their arguments. The evaluation is based on the word disambiguation task developed by Mitchell and Lapata (2008) for intransitive sentences, and on a similar new experiment designed for transitive sentences. Our model matches the results of its competitors in the first experiment, and betters them in the second. The general improvement in results with increase in syntactic complexity showcases the compositional power of our model.Comment: 11 pages, to be presented at EMNLP 2011, to be published in Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processin

A Convolutional Neural Network for Modelling Sentences

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline

Analysing Lexical Semantic Change with Contextualised Word Representations

This paper presents the first unsupervised approach to lexical semantic change that makes use of contextualised word representations. We propose a novel method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics. We create a new evaluation dataset and show that the model representations and the detected semantic shifts are positively correlated with human judgements. Our extensive qualitative analysis demonstrates that our method captures a variety of synchronic and diachronic linguistic phenomena. We expect our work to inspire further research in this direction.Comment: To appear in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL-2020

Sistem Temu Kembali Informasi dengan Pemeringkatan Metode Vector Space Model

The objective of designing information retrieval system (IRS) with Vector Space Model (VSM) Method is to facilitate users to search Indonesian documents. IRS Software is designed to provide search results with the optimum number of documents (low recall) and accuracy (high precision) with VSM method that users may get fast and accurate results. VSM method provides a different credit for each document stored in a database which in turns to determine the document most similar to the query, where the documents with the highest credits are placed on the top of the search results. The evaluation of search results with IRS is conducted under recall and precision tests. This study fascinatingly creates a system which can preprocess (tokenizing, filtering, and stemming) within computation time of four minutes forty-one seconds

Don’t Invite BERT to Drink a Bottle: Modeling the Interpretation of Metonymies Using BERT and Distributional Representations

In this work, we carry out two experiments in order to assess the ability of BERT to capture themeaning shift associated with metonymic expressions. We test the model on a new dataset that isrepresentative of the most common types of metonymy. We compare BERT with the StructuredDistributional Model (SDM), a model for the representation of words in context which is basedon the notion of Generalized Event Knowledge. The results reveal that, while BERT abilityto deal with metonymy is quite limited, SDM is good at predicting the meaning of metonymicexpressions, providing support for an account of metonymy based on event knowledge