3,194 research outputs found
Learning Character-level Compositionality with Visual Features
Previous work has modeled the compositionality of words by creating
character-level models of meaning, reducing problems of sparsity for rare
words. However, in many writing systems compositionality has an effect even on
the character-level: the meaning of a character is derived by the sum of its
parts. In this paper, we model this effect by creating embeddings for
characters based on their visual characteristics, creating an image for the
character and running it through a convolutional neural network to produce a
visual character embedding. Experiments on a text classification task
demonstrate that such model allows for better processing of instances with rare
characters in languages such as Chinese, Japanese, and Korean. Additionally,
qualitative analyses demonstrate that our proposed model learns to focus on the
parts of characters that carry semantic content, resulting in embeddings that
are coherent in visual space.Comment: Accepted to ACL 201
Multi Sense Embeddings from Topic Models
Distributed word embeddings have yielded state-of-the-art performance in many NLP tasks, mainly due to their success in capturing useful semantic information. These representations assign only a single vector to each word whereas a large number of words are polysemous (i.e., have multiple meanings). In this work, we approach this critical problem in lexical semantics, namely that of representing various senses of polysemous words in vector spaces. We propose a topic modeling based skip-gram approach for learning multi-prototype word embeddings. We also introduce a method to prune the embeddings determined by the probabilistic representation of the word in each topic. We use our embeddings to show that they can capture the context and word similarity strongly and outperform various state-of-the-art implementations
Can humain association norm evaluate latent semantic analysis?
This paper presents the comparison of word association norm created by a psycholinguistic experiment to association lists generated by algorithms operating on text corpora. We compare lists generated by Church and Hanks algorithm and lists generated by LSA algorithm. An argument is presented on how those automatically generated lists reflect real semantic relations
Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models
In this perspective paper, we first comprehensively review existing
evaluations of Large Language Models (LLMs) using both standardized tests and
ability-oriented benchmarks. We pinpoint several problems with current
evaluation methods that tend to overstate the capabilities of LLMs. We then
articulate what artificial general intelligence should encompass beyond the
capabilities of LLMs. We propose four characteristics of generally intelligent
agents: 1) they can perform unlimited tasks; 2) they can generate new tasks
within a context; 3) they operate based on a value system that underpins task
generation; and 4) they have a world model reflecting reality, which shapes
their interaction with the world. Building on this viewpoint, we highlight the
missing pieces in artificial general intelligence, that is, the unity of
knowing and acting. We argue that active engagement with objects in the real
world delivers more robust signals for forming conceptual representations.
Additionally, knowledge acquisition isn't solely reliant on passive input but
requires repeated trials and errors. We conclude by outlining promising future
research directions in the field of artificial general intelligence
- …