Unsupervised and Lightly Supervised Part-of-Speech Tagging Using Recurrent Neural Networks

International audienceIn this paper, we propose a novel approach to induce automatically a Part-Of-Speech (POS) tagger for resource-poor languages (languages that have no labeled training data). This approach is based on cross-language projection of linguistic annotations from parallel corpora without the use of word alignment information. Our approach does not assume any knowledge about foreign languages, making it applicable to a wide range of resource-poor languages. We use Recurrent Neural Networks (RNNs) as multilingual analysis tool. Our approach combined with a basic cross-lingual projection method (using word alignment information) achieves comparable results to the state-of-the-art. We also use our approach in a weakly supervised context, and it shows an excellent potential for very low-resource settings (less than 1k training utterances)

Inducing Multilingual Text Analysis Tools Using Bidirectional Recurrent Neural Networks

International audienceThis work focuses on the rapid development of linguistic annotation tools for resource-poor languages. We experiment several cross-lingual annotation projection methods using Recurrent Neural Networks (RNN) models. The distinctive feature of our approach is that our multilingual word representation requires only a parallel corpus between the source and target language. More precisely, our method has the following characteristics: (a) it does not use word alignment information, (b) it does not assume any knowledge about foreign languages, which makes it applicable to a wide range of resource-poor languages, (c) it provides truly multilingual taggers. We investigate both uni-and bi-directional RNN models and propose a method to include external information (for instance low level information from POS) in the RNN to train higher level taggers (for instance, super sense taggers). We demonstrate the validity and genericity of our model by using parallel corpora (obtained by manual or automatic translation). Our experiments are conducted to induce cross-lingual POS and super sense taggers

Utilisation des réseaux de neurones récurrents pour la projection interlingue d'étiquettes morpho-syntaxiques à partir d'un corpus parallèle

International audienceIn this paper, we propose a method to automatically induce linguistic analysis tools for languages that have no labeled training data. This method is based on cross-language projection of linguistic annotations from parallel corpora. Our method does not assume any knowledge about foreign languages, making it applicable to a wide range of resource-poor languages. No word alignment information is needed in our approach. We use Recurrent Neural Networks (RNNs) as cross-lingual analysis tool. To illustrate the potential of our approach, we firstly investigate Part-Of-Speech (POS) tagging. Combined with a simple projection method (using word alignment information), it achieves performance comparable to the one of recently published approaches for cross-lingual projection. Mots-clés : Multilinguisme, transfert crosslingue, étiquetage morpho-syntaxique, réseaux de neurones récurrents

Projection Interlingue d'Étiquettes pour l'Annotation Sémantique Non Supervisée

International audienceCross-lingual Annotation Projection for Unsupervised Semantic Tagging. This work focuses on the development of linguistic analysis tools for resource-poor languages. In a previous study, we proposed a method based on cross-language projection of linguistic annotations from parallel corpora to automatically induce a morpho-syntactic analyzer. Our approach was based on Recurrent Neural Networks (RNNs). In this paper, we present an improvement of our neural model. We investigate the inclusion of external information (POS tags) in the neural network to train a multilingual SuperSenses Tagger. We demonstrate the validity and genericity of our method by using parallel corpora (obtained by manual or automatic translation). Our experiments are conducted for cross-lingual annotation projection from English to French and Italian

Improving the Performance of an Example-Based Machine Translation System Using a Domain-specific Bilingual Lexicon

Conference of 29th Pacific Asia Conference on Language, Information and Computation, PACLIC 2015 ; Conference Date: 30 October 2015 Through 1 November 2015; Conference Code:119467International audienceIn this paper, we study the impact of using a domain-specific bilingual lexicon on the performance of an Example-Based Machine Translation system. We conducted experiments for the English-French language pair on in-domain texts from Europarl (European Parliament Proceedings) and out-of-domain texts from Emea (European Medicines Agency Documents), and we compared the results of the Example-Based Machine Translation system against those of the Statistical Machine Translation system Moses. The obtained results revealed that adding a domain-specific bilingual lexicon (extracted from a parallel domain-specific corpus) to the general-purpose bilingual lexicon of the Example-Based Machine Translation system improves translation quality for both in-domain as well as outof-domain texts, and the Example-Based Machine Translation system outperforms Moses when texts to translate are related to the specific domain

Automatic creation of linguistic tools and resources from parallel corpora

Cette thèse porte sur la construction automatique d’outils et de ressources pour l’analyse linguistique de textes des langues peu dotées. Nous proposons une approche utilisant des réseaux de neurones récurrents (RNN - Recurrent Neural Networks) et n'ayant besoin que d'un corpus parallèle ou mutli-parallele entre une langue source bien dotée et une ou plusieurs langues cibles moins bien ou peu dotées. Ce corpus parallèle ou mutli-parallele est utilisé pour la construction d'une représentation multilingue des mots des langues source et cible. Nous avons utilisé cette représentation multilingue pour l’apprentissage de nos modèles neuronaux et nous avons exploré deux architectures neuronales : les RNN simples et les RNN bidirectionnels. Nous avons aussi proposé plusieurs variantes des RNN pour la prise en compte d'informations linguistiques de bas niveau (informations morpho-syntaxiques) durant le processus de construction d'annotateurs linguistiques de niveau supérieur (SuperSenses et dépendances syntaxiques). Nous avons démontré la généricité de notre approche sur plusieurs langues ainsi que sur plusieurs tâches d'annotation linguistique. Nous avons construit trois types d'annotateurs linguistiques multilingues: annotateurs morpho-syntaxiques, annotateurs en SuperSenses et annotateurs en dépendances syntaxiques, avec des performances très satisfaisantes. Notre approche a les avantages suivants : (a) elle n'utilise aucune information d'alignement des mots, (b) aucune connaissance concernant les langues cibles traitées n'est requise au préalable (notre seule supposition est que, les langues source et cible n'ont pas une grande divergence syntaxique), ce qui rend notre approche applicable pour le traitement d'un très grand éventail de langues peu dotées, (c) elle permet la construction d'annotateurs multilingues authentiques (un annotateur pour N langages).This thesis focuses on the automatic construction of linguistic tools and resources for analyzing texts of low-resource languages. We propose an approach using Recurrent Neural Networks (RNN) and requiring only a parallel or multi-parallel corpus between a well-resourced language and one or more low-resource languages. This parallel or multi-parallel corpus is used to construct a multilingual representation of words of the source and target languages. We used this multilingual representation to train our neural models and we investigated both uni and bidirectional RNN models. We also proposed a method to include external information (for instance, low-level information from Part-Of-Speech tags) in the RNN to train higher level taggers (for instance, SuperSenses taggers and Syntactic dependency parsers). We demonstrated the validity and genericity of our approach on several languages and we conducted experiments on various NLP tasks: Part-Of-Speech tagging, SuperSenses tagging and Dependency parsing. The obtained results are very satisfactory. Our approach has the following characteristics and advantages: (a) it does not use word alignment information, (b) it does not assume any knowledge about target languages (one requirement is that the two languages (source and target) are not too syntactically divergent), which makes it applicable to a wide range of low-resource languages, (c) it provides authentic multilingual taggers (one tagger for N languages)

Accélération matérielle de convolutions éparses appliquées à la détection 3D

RÉSUMÉ: Ce mémoire se concentre sur l’optimisation et l’accélération matérielle des convolutions 3D sur des données éparses. La validation de l’algorithme est réalisée sur le composant de convolution de l’architecture Voxel-RCNN, utilisée pour la détection d’objets en 3D, afin de faciliter son déploiement. L’objectif principal est d’améliorer les performances et l’efficacité de l’implémentation de ce composant en explorant diverses techniques d’optimisation et d’accélération matérielle. Les techniques expérimentales déployées dans cette étude incluent la quantification, la binarisation, et le recours à un module FPGA spécialisé. Ces méthodes ont été appliquées spécifiquement sur le composant de convolution de l’architecture Voxel-RCNN. Des méthodes d’optimisation préliminaires, à savoir la quantification et la binarisation, sont mises en avant. Ces approches ont permis de réaliser des améliorations substantielles. Ce projet aborde également la modélisation logicielle de l’implémentation matérielle, en sous divisant l’espace en cuves auxquelles les voxels sont répartis. Ce processus est analysé, soulignant l’efficacité de la modélisation logicielle. Cette recherche aborde la question de l’accélération matérielle, en exposant en détail l’architecture du module VHDL développé, la gestion des entrées et sorties, ainsi que la mise en place de la machine à état pour le calcul de convolution. Elle se focalise également sur l’utilisation des ressources matérielles par les modules et leurs temps de traitement respectifs. Un point d’attention particulier a été porté sur l’impact de la parallélisation sur le calcul de la convolution, en explorant diverses configurations de modules parallèles. Cet aspect a révélé le potentiel de l’approche parallèle, ainsi que les défis liés à la gestion des accès au bus. En considérant les contraintes matérielles de la carte utilisée, il a été possible d’instancier jusqu’à 35 modules dans le design proposé testé sur le dataset KITTI. L’efficacité de la parallélisation a été confirmée par une diminution significative du temps de calcul à mesure que le nombre de modules de convolution augmentait. Cependant, malgré ces avancées, la compétitivité de cette approche reste limitée par rapport aux solutions optimisées existantes utilisant des GPU. ABSTRACT: This thesis focuses on the optimization and hardware acceleration of 3D convolution operations within the Voxel-RCNN architecture for LiDAR-based 3D object detection. The key objective is to enhance the architecture’s efficiency and performance by leveraging various optimization techniques and FPGA-based hardware acceleration. Efficient 3D data manipulation is achieved by voxelizing LiDAR data, with the added benefit of harnessing FPGA’s parallel processing capabilities. Experimental methods include quantization, binarization and the use of specialized FPGA modules. Modifications to the Voxel-RCNN architecture are also presented. Initial optimization methods such as quantization and binarization are analyzed This project also focuses on the development of a software model to emulate the hardware implementation. In this model, the 3D space is subdivided into distinct segments, termed ’vats’, to which voxels are allocated. An in-depth analysis of this method underscores the efficacy of such an approach in software modeling. The study then turns to hardware acceleration, involving the development of a specialized VHDL module and the implementation of a state machine for convolution computation. This process considers the use of hardware resources and their processing times. The study explores the potential benefits and challenges of parallelization in convolution computation. The research also probes the possibility of incorporating multiple convolution modules, with the limit being 35 modules per design due to the constraints of the DE-10 board. This design facilitates the processing of the KITTI dataset. The efficient use of available resources is demonstrated through the reduced computation time observed when increasing the number of convolution modules in parallel, independent of the voxel count. The current implementation of our FPGA-based convolution module is not competitive with existing GPU models in terms of overall performance

Construction automatique d'outils et de ressources linguistiques à partir de corpus parallèles

This thesis focuses on the automatic construction of linguistic tools and resources for analyzing texts of low-resource languages. We propose an approach using Recurrent Neural Networks (RNN) and requiring only a parallel or multi-parallel corpus between a well-resourced language and one or more low-resource languages. This parallel or multi-parallel corpus is used to construct a multilingual representation of words of the source and target languages. We used this multilingual representation to train our neural models and we investigated both uni and bidirectional RNN models. We also proposed a method to include external information (for instance, low-level information from Part-Of-Speech tags) in the RNN to train higher level taggers (for instance, SuperSenses taggers and Syntactic dependency parsers). We demonstrated the validity and genericity of our approach on several languages and we conducted experiments on various NLP tasks: Part-Of-Speech tagging, SuperSenses tagging and Dependency parsing. The obtained results are very satisfactory. Our approach has the following characteristics and advantages: (a) it does not use word alignment information, (b) it does not assume any knowledge about target languages (one requirement is that the two languages (source and target) are not too syntactically divergent), which makes it applicable to a wide range of low-resource languages, (c) it provides authentic multilingual taggers (one tagger for N languages).Cette thèse porte sur la construction automatique d’outils et de ressources pour l’analyse linguistique de textes des langues peu dotées. Nous proposons une approche utilisant des réseaux de neurones récurrents (RNN - Recurrent Neural Networks) et n'ayant besoin que d'un corpus parallèle ou mutli-parallele entre une langue source bien dotée et une ou plusieurs langues cibles moins bien ou peu dotées. Ce corpus parallèle ou mutli-parallele est utilisé pour la construction d'une représentation multilingue des mots des langues source et cible. Nous avons utilisé cette représentation multilingue pour l’apprentissage de nos modèles neuronaux et nous avons exploré deux architectures neuronales : les RNN simples et les RNN bidirectionnels. Nous avons aussi proposé plusieurs variantes des RNN pour la prise en compte d'informations linguistiques de bas niveau (informations morpho-syntaxiques) durant le processus de construction d'annotateurs linguistiques de niveau supérieur (SuperSenses et dépendances syntaxiques). Nous avons démontré la généricité de notre approche sur plusieurs langues ainsi que sur plusieurs tâches d'annotation linguistique. Nous avons construit trois types d'annotateurs linguistiques multilingues: annotateurs morpho-syntaxiques, annotateurs en SuperSenses et annotateurs en dépendances syntaxiques, avec des performances très satisfaisantes. Notre approche a les avantages suivants : (a) elle n'utilise aucune information d'alignement des mots, (b) aucune connaissance concernant les langues cibles traitées n'est requise au préalable (notre seule supposition est que, les langues source et cible n'ont pas une grande divergence syntaxique), ce qui rend notre approche applicable pour le traitement d'un très grand éventail de langues peu dotées, (c) elle permet la construction d'annotateurs multilingues authentiques (un annotateur pour N langages)

Evaluating the impact of using a domain-specific bilingual lexicon on the performance of a hybrid machine translation approach

Conference of 10th International Conference on Recent Advances in Natural Language Processing, RANLP 2015 ; Conference Date: 7 September 2015 Through 9 September 2015; Conference Code:116983International audienceThis paper describes an Example-Based Machine Translation prototype and presents an evaluation of the impact of using a domain specific vocabulary on its performance. This prototype is based on a hybrid approach which needs only monolingual texts in the target language and consists to combine translation candidates returned by a cross-language search engine with translation hypotheses provided by a finite-state transducer. The results of this combination are evaluated against a statistical language model of the target language in order to obtain the n-best translations. To measure the performance of this hybrid approach, we achieved several experiments using corpora on two domains from the European Parliament proceedings (Europarl) and the European Medicines Agency documents (Emea). The obtained results show that the proposed approach out performs the state-of-the-art Statistical Machine Translation system Moses when texts to translate are related to the specialized domain