Duality Regularization for Unsupervised Bilingual Lexicon Induction

Unsupervised bilingual lexicon induction naturally exhibits duality, which results from symmetry in back-translation. For example, EN-IT and IT-EN induction can be mutually primal and dual problems. Current state-of-the-art methods, however, consider the two tasks independently. In this paper, we propose to train primal and dual models jointly, using regularizers to encourage consistency in back translation cycles. Experiments across 6 language pairs show that the proposed method significantly outperforms competitive baselines, obtaining the best-published results on a standard benchmark

A survey of cross-lingual word embedding models

Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages. In this survey, we provide a comprehensive typology of cross-lingual word embedding models. We compare their data requirements and objective functions. The recurring theme of the survey is that many of the models presented in the literature optimize for the same objectives, and that seemingly different models are often equivalent, modulo optimization strategies, hyper-parameters, and such. We also discuss the different ways cross-lingual word embeddings are evaluated, as well as future challenges and research horizons.</jats:p

Empirical studies on word representations

One of the most fundamental tasks in natural language processing is representing words with mathematical objects (such as vectors). The word representations, which are most often estimated from data, allow capturing the meaning of words. They enable comparing words according to their semantic similarity, and have been shown to work extremely well when included in complex real-world applications. A large part of our work deals with ways of estimating word representations directly from large quantities of text. Our methods exploit the idea that words which occur in similar contexts have a similar meaning. How we define the context is an important focus of our thesis. The context can consist of a number of words to the left and to the right of the word in question, but, as we show, obtaining context words via syntactic links (such as the link between the verb and its subject) often works better. We furthermore investigate word representations that accurately capture multiple meanings of a single word. We show that translation of a word in context contains information that can be used to disambiguate the meaning of that word

Data fusion and matching by maximizing statistical dependencies

The core aim of machine learning is to make a computer program learn from the experience. Learning from data is usually defined as a task of learning regularities or patterns in data in order to extract useful information, or to learn the underlying concept. An important sub-field of machine learning is called multi-view learning where the task is to learn from multiple data sets or views describing the same underlying concept. A typical example of such scenario would be to study a biological concept using several biological measurements like gene expression, protein expression and metabolic profiles, or to classify web pages based on their content and the contents of their hyperlinks. In this thesis, novel problem formulations and methods for multi-view learning are presented. The contributions include a linear data fusion approach during exploratory data analysis, a new measure to evaluate different kinds of representations for textual data, and an extension of multi-view learning for novel scenarios where the correspondence of samples in the different views or data sets is not known in advance. In order to infer the one-to-one correspondence of samples between two views, a novel concept of multi-view matching is proposed. The matching algorithm is completely data-driven and is demonstrated in several applications such as matching of metabolites between humans and mice, and matching of sentences between documents in two languages.Koneoppimisessa pyritään luomaan tietokoneohjelmia, jotka oppivat kokemuksen kautta. Tehtävänä on usein oppia tietoaineistoista säännönmukaisuuksia joiden avulla saadaan uutta tietoa aineiston taustalla olevasta ilmiöstä ja voidaan ymmärtää ilmiötä paremmin. Eräs keskeinen koneoppimisen alahaara käsittelee oppimista useita samaa ilmiötä käsitteleviä tietoaineistoja yhdistelemällä. Tavoitteena voi olla esimerkiksi solutason biologisen ilmiön ymmärtäminen tarkastelemalla geenien aktiivisuusmittauksia, proteiinien konsentraatioita ja metabolista aktiivisuutta samanaikaisesti. Toisena esimerkkinä verkkosivuja voidaan luokitella samanaikaisesti sekä niiden tekstisisällön että hyperlinkkirakenteen perusteella. Tässä väitöskirjassa esitellään uusia periaatteita ja menetelmiä useiden tietolähteiden yhdistelemiseen. Työn päätuloksina esitellään lineaarinen tietoaineistojen yhdistelemismenetelmä tutkivaan analysiin, uusi menetelmä tekstiaineistojen erilaisten esitystapojen vertailuun sekä uusi yhdistelemisperiaate tilanteisiin joissa aineistojen näytteiden vastaavuutta toisiinsa ei tunneta ennalta. Työssä esitetään kuinka vastaavuus voidaan oppia tietoaineistoista itsestään, ilman ulkopuolista ohjausta. Uutta menetelmää sovelletaan työssä esimerkiksi hakemaan vastaavuuksia ihmisten ja hiirten metaboliamittauksista sekä etsimään samaa merkitseviä lauseita kahdella eri kielellä kirjoitetuista teksteistä

Unsupervised Graph-Based Similarity Learning Using Heterogeneous Features.

Relational data refers to data that contains explicit relations among objects. Nowadays, relational data are universal and have a broad appeal in many different application domains. The problem of estimating similarity between objects is a core requirement for many standard Machine Learning (ML), Natural Language Processing (NLP) and Information Retrieval (IR) problems such as clustering, classiffication, word sense disambiguation, etc. Traditional machine learning approaches represent the data using simple, concise representations such as feature vectors. While this works very well for homogeneous data, i.e, data with a single feature type such as text, it does not exploit the availability of dfferent feature types fully. For example, scientic publications have text, citations, authorship information, venue information. Each of the features can be used for estimating similarity. Representing such objects has been a key issue in efficient mining (Getoor and Taskar, 2007). In this thesis, we propose natural representations for relational data using multiple, connected layers of graphs; one for each feature type. Also, we propose novel algorithms for estimating similarity using multiple heterogeneous features. Also, we present novel algorithms for tasks like topic detection and music recommendation using the estimated similarity measure. We demonstrate superior performance of the proposed algorithms (root mean squared error of 24.81 on the Yahoo! KDD Music recommendation data set and classiffication accuracy of 88% on the ACL Anthology Network data set) over many of the state of the art algorithms, such as Latent Semantic Analysis (LSA), Multiple Kernel Learning (MKL) and spectral clustering and baselines on large, standard data sets.Ph.D.Computer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/89824/1/mpradeep_1.pd

Speaking while listening: Language processing in speech shadowing and translation

Contains fulltext : 233349.pdf (Publisher’s version ) (Open Access)Radboud University, 25 mei 2021Promotores : Meyer, A.S., Roelofs, A.P.A.199 p