1,482 research outputs found
Unsupervised Bilingual POS Tagging with Markov Random Fields
In this paper, we give a treatment to the problem of bilingual part-of-speech induction with parallel data. We demonstrate that naïve optimization of log-likelihood with joint MRFs suffers from a severe problem of local maxima, and suggest an alternative – using contrastive estimation for estimation of the parameters. Our experiments show that estimating the parameters this way, using overlapping features with joint MRFs performs better than previous work on the 1984 dataset.
Density Matching for Bilingual Word Embedding
Recent approaches to cross-lingual word embedding have generally been based
on linear transformations between the sets of embedding vectors in the two
languages. In this paper, we propose an approach that instead expresses the two
monolingual embedding spaces as probability densities defined by a Gaussian
mixture model, and matches the two densities using a method called normalizing
flow. The method requires no explicit supervision, and can be learned with only
a seed dictionary of words that have identical strings. We argue that this
formulation has several intuitively attractive properties, particularly with
the respect to improving robustness and generalization to mappings between
difficult language pairs or word pairs. On a benchmark data set of bilingual
lexicon induction and cross-lingual word similarity, our approach can achieve
competitive or superior performance compared to state-of-the-art published
results, with particularly strong results being found on etymologically distant
and/or morphologically rich languages.Comment: Accepted by NAACL-HLT 201
Why is unsupervised alignment of English embeddings from different algorithms so hard?
This paper presents a challenge to the community: Generative adversarial
networks (GANs) can perfectly align independent English word embeddings induced
using the same algorithm, based on distributional information alone; but fails
to do so, for two different embeddings algorithms. Why is that? We believe
understanding why, is key to understand both modern word embedding algorithms
and the limitations and instability dynamics of GANs. This paper shows that (a)
in all these cases, where alignment fails, there exists a linear transform
between the two embeddings (so algorithm biases do not lead to non-linear
differences), and (b) similar effects can not easily be obtained by varying
hyper-parameters. One plausible suggestion based on our initial experiments is
that the differences in the inductive biases of the embedding algorithms lead
to an optimization landscape that is riddled with local optima, leading to a
very small basin of convergence, but we present this more as a challenge paper
than a technical contribution.Comment: Accepted at EMNLP 201
Plan Optimization to Bilingual Dictionary Induction for Low-Resource Language Families
Creating bilingual dictionary is the first crucial step in enriching
low-resource languages. Especially for the closely-related ones, it has been
shown that the constraint-based approach is useful for inducing bilingual
lexicons from two bilingual dictionaries via the pivot language. However, if
there are no available machine-readable dictionaries as input, we need to
consider manual creation by bilingual native speakers. To reach a goal of
comprehensively create multiple bilingual dictionaries, even if we already have
several existing machine-readable bilingual dictionaries, it is still difficult
to determine the execution order of the constraint-based approach to reducing
the total cost. Plan optimization is crucial in composing the order of
bilingual dictionaries creation with the consideration of the methods and their
costs. We formalize the plan optimization for creating bilingual dictionaries
by utilizing Markov Decision Process (MDP) with the goal to get a more accurate
estimation of the most feasible optimal plan with the least total cost before
fully implementing the constraint-based bilingual lexicon induction. We model a
prior beta distribution of bilingual lexicon induction precision with language
similarity and polysemy of the topology as and parameters. It
is further used to model cost function and state transition probability. We
estimated the cost of all investment plan as a baseline for evaluating the
proposed MDP-based approach with total cost as an evaluation metric. After
utilizing the posterior beta distribution in the first batch of experiments to
construct the prior beta distribution in the second batch of experiments, the
result shows 61.5\% of cost reduction compared to the estimated all investment
plan and 39.4\% of cost reduction compared to the estimated MDP optimal plan.
The MDP-based proposal outperformed the baseline on the total cost.Comment: 29 pages, 16 figures, 9 tables, accepted for publication in ACM
TALLI
Plan Optimization for Creating Bilingual Dictionaries of Low-Resource Languages
The constraint-based approach has been proven useful for inducing bilingual lexicons for closely-related low- resource languages. When we want to create multiple bilingual dictionaries linking several languages, we need to consider manual creation by bilingual language experts if there are no available machine-readable dictionaries are available as input. To overcome the difficulty in planning the creation of bilingual dictionaries, the consideration of various methods and costs, plan optimization is essential. We adopt the Markov Decision Process (MDP) in formalizing plan optimization for creating bilingual dictionaries; the goal is to better predict the most feasible optimal plan with the least total cost before fully implementing the constraint-based bilingual dictionary induction framework. We define heuristics based on input language characteristics to devise a baseline plan for evaluating our MDP-based approach with total cost as an evaluation metric. The MDP-based proposal outperformed heuristic planning on the total cost for all datasets examined
- …