Improving transition-based dependency parsing with buffer transitions

In this paper, we show that significant improvements in the accuracy of well-known transition-based parsers can be obtained, without sacrificing efficiency, by enriching the parsers with simple transitions that act on buffer nodes. First, we show how adding a specific transition to create either a left or right arc of length one between the first two buffer nodes produces improvements in the accuracy of Nivre's arc-eager projective parser on a number of datasets from the CoNLL-X shared task. Then, we show that accuracy can also be improved by adding transitions involving the topmost stack node and the second buffer node (allowing a limited form of non-projectivity). None of these transitions has a negative impact on the computational complexity of the algorithm. Although the experiments in this paper use the arc-eager parser, the approach is generic enough to be applicable to any stack-based dependency parser.Ministerio de Ciencia e Innovación | Ref. TIN2010-18552-C03-01Ministerio de Ciencia e Innovación | Ref. TIN2010-18552-C03-02Xunta de Galici

Arc-swift: A Novel Transition System for Dependency Parsing

Transition-based dependency parsers often need sequences of local shift and reduce operations to produce certain attachments. Correct individual decisions hence require global information about the sentence context and mistakes cause error propagation. This paper proposes a novel transition system, arc-swift, that enables direct attachments between tokens farther apart with a single transition. This allows the parser to leverage lexical information more directly in transition decisions. Hence, arc-swift can achieve significantly better performance with a very small beam size. Our parsers reduce error by 3.7--7.6% relative to those using existing transition systems on the Penn Treebank dependency parsing task and English Universal Dependencies.Comment: Accepted at ACL 201

Left-to-Right Dependency Parsing with Pointer Networks

We propose a novel transition-based algorithm that straightforwardly parses sentences from left to right by building $n$ attachments, with $n$ being the length of the input sentence. Similarly to the recent stack-pointer parser by Ma et al. (2018), we use the pointer network framework that, given a word, can directly point to a position from the sentence. However, our left-to-right approach is simpler than the original top-down stack-pointer parser (not requiring a stack) and reduces transition sequence length in half, from 2$n$-1 actions to $n$. This results in a quadratic non-projective parser that runs twice as fast as the original while achieving the best accuracy to date on the English PTB dataset (96.04% UAS, 94.43% LAS) among fully-supervised single-model dependency parsers, and improves over the former top-down transition system in the majority of languages tested.Comment: Proceedings of NAACL 2019. 7 page

Improvements to the performance and applicability of dependency parsing

[Resumen]Los analizadores de dependencias han generado un gran interés en las últimas décadas debido a su utilidad en un amplio rango de tareas de procesamiento de lenguaje natural. Estos utilizan grafos de dependencias para definir la estructura sintáctica de una oración dada. En particular, los algoritmos basados en transiciones proveen un análisis sintáctico de dependencias eficiente y preciso. Sin embargo, su principal inconveniente es que tienden a sufrir propagación de errores. Así, una decisión temprana tomada erróneamente podría posicionar el analizador en un estado incorrecto, causando más errores en futuras decisiones. Esta tesis se centra en mejorar la precisión de los analizadores basados en transiciones mediante la reducción del efecto de la propagación de errores, mientras mantienen su velocidad y eficiencia. Concretamente, proponemos cinco enfoques diferentes que han demostrado ser beneficiosos para su rendimiento, al aliviar la propagación de errores e incrementar su precisión. Además, hemos ampliado la utilidad de los analizadores de dependencias más allá de la construcción de grafos de dependencias. Presentamos una novedosa técnica que permite que estos sean capaces de construir representaciones de constituyentes. Esto cubriría la necesidad de la comunidad de procesamiento de lenguaje natural de disponer de un analizador eficiente capaz de proveer un árbol de constituyentes para representar la estructura sintáctica de las oraciones.[Abstract]Dependency parsers have attracted a remarkable interest in the last two decades due to their usefulness in a wide range of natural language processing tasks. They employ a dependency graph to define the syntactic structure of a given sentence. In particular, transition-based algorithms provide accurate and efficient dependency syntactic analyses. However, the main drawback of these techniques is that they tend to suffer from error propagation. So, an early erroneous decision may place the parser into an incorrect state, causing more errors in future decisions. This thesis focuses on improving the accuracy of transition-based parsers by reducing the effect of error propagation, while preserving their speed and efficiency. Concretely, we propose five different approaches that proved to be beneficial for their performance, mitigating the presence of error propagation and boosting its accuracy. We also extend the usefulness of dependency parsers beyond building dependency graphs.We present a novel technique that allows these to build constituent representations. This meets the necessity of the natural language processing community to have an efficient parser able to provide constituent trees to represent the syntactic structure of sentences.[Resumo]Os analizadores de dependencias xeraron gran interese nas últimas décadas debido á súa utilidade nun amplo rango de tarefas de procesamento da linguaxe natural. Estes utilizan grafos de dependencias para definir a estrutura sintáctica dunha oración dada. En particular, os algoritmos baseados en transicións provén un análise sintáctico de dependencias eficiente e preciso. Sen embargo, o seu principal inconveniente é que tenden a sufrir propagación de erros. Así, unha decisión temprana tomada erroneamente podería posicionar o analizador nun estado incorrecto, causando máis erros en futuras decisións. Esta tese centrase en mellorar a precisión dos analizadores baseados en transicións mediante a redución do efecto da propagación de erros, mentres manteñen a súa velocidade e eficiencia. Concretamente, propomos cinco diferentes enfoques que demostraron ser beneficiosos para o seu rendemento, ó aliviar a propagación de erros e incrementar a súa precisión. Ademais, ampliámo-la utilidade dos analizadores de dependencias máis alá da construción de grafos de dependencias. Presentamos unha novidosa técnica que permite que estes sexan capaces de construir representacións de constituíntes. Isto cubriría a necesidade da comunidade de procesamento da linguaxe natural de dispor dun analizador eficiente capaz de prover unha árbore de constituíntes para representar a estrutura sintáctica das oracións