50 research outputs found
Text segmentation with character-level text embeddings
Learning word representations has recently seen much success in computational
linguistics. However, assuming sequences of word tokens as input to linguistic
analysis is often unjustified. For many languages word segmentation is a
non-trivial task and naturally occurring text is sometimes a mixture of natural
language strings and other character data. We propose to learn text
representations directly from raw character sequences by training a Simple
recurrent Network to predict the next character in text. The network uses its
hidden layer to evolve abstract representations of the character sequences it
sees. To demonstrate the usefulness of the learned text embeddings, we use them
as features in a supervised character level text segmentation and labeling
task: recognizing spans of text containing programming language code. By using
the embeddings as features we are able to substantially improve over a baseline
which uses only surface character n-grams.Comment: Workshop on Deep Learning for Audio, Speech and Language Processing,
ICML 201
The Expressive Power of Word Embeddings
We seek to better understand the difference in quality of the several
publicly released embeddings. We propose several tasks that help to distinguish
the characteristics of different embeddings. Our evaluation of sentiment
polarity and synonym/antonym relations shows that embeddings are able to
capture surprisingly nuanced semantics even in the absence of sentence
structure. Moreover, benchmarking the embeddings shows great variance in
quality and characteristics of the semantics captured by the tested embeddings.
Finally, we show the impact of varying the number of dimensions and the
resolution of each dimension on the effective useful features captured by the
embedding space. Our contributions highlight the importance of embeddings for
NLP tasks and the effect of their quality on the final results.Comment: submitted to ICML 2013, Deep Learning for Audio, Speech and Language
Processing Workshop. 8 pages, 8 figure
Boosting Named Entity Recognition with Neural Character Embeddings
Most state-of-the-art named entity recognition (NER) systems rely on
handcrafted features and on the output of other NLP tasks such as
part-of-speech (POS) tagging and text chunking. In this work we propose a
language-independent NER system that uses automatically learned features only.
Our approach is based on the CharWNN deep neural network, which uses word-level
and character-level representations (embeddings) to perform sequential
classification. We perform an extensive number of experiments using two
annotated corpora in two different languages: HAREM I corpus, which contains
texts in Portuguese; and the SPA CoNLL-2002 corpus, which contains texts in
Spanish. Our experimental results shade light on the contribution of neural
character embeddings for NER. Moreover, we demonstrate that the same neural
network which has been successfully applied to POS tagging can also achieve
state-of-the-art results for language-independet NER, using the same
hyperparameters, and without any handcrafted features. For the HAREM I corpus,
CharWNN outperforms the state-of-the-art system by 7.9 points in the F1-score
for the total scenario (ten NE classes), and by 7.2 points in the F1 for the
selective scenario (five NE classes).Comment: 9 page
NILC_USP: aspect extraction using semantic labels
This paper details the system NILC USP that participated in the Semeval 2014: Aspect Based Sentiment Analysis task. This system uses a Conditional Random Field (CRF) algorithm for extracting the aspects mentioned in the text. Our work added semantic labels into a basic feature set for measuring the efficiency of those for aspect extraction. We used the semantic roles and the highest verb frame as features for the machine learning. Overall, our results demonstrated that the system could not improve with the use of this semantic information, but its precision was increased.FAPES
Polyglot: Distributed Word Representations for Multilingual NLP
Distributed word representations (word embeddings) have recently contributed
to competitive performance in language modeling and several NLP tasks. In this
work, we train word embeddings for more than 100 languages using their
corresponding Wikipedias. We quantitatively demonstrate the utility of our word
embeddings by using them as the sole features for training a part of speech
tagger for a subset of these languages. We find their performance to be
competitive with near state-of-art methods in English, Danish and Swedish.
Moreover, we investigate the semantic features captured by these embeddings
through the proximity of word groupings. We will release these embeddings
publicly to help researchers in the development and enhancement of multilingual
applications.Comment: 10 pages, 2 figures, Proceedings of Conference on Computational
Natural Language Learning CoNLL'201