Italian Lemmatization by Rules with GETARUNS

We present an approach to lemmatization based on exhaustive morphological analysis and use of external knowledge sources to help disambiguation which is the most relevant issue to cope with. Our system GETARUNS was not concerned with lemmatization directly and used morphological analysis only as backoff solution in case the word was not retrieved in the wordform dictionaries available. We found out that both the rules and the root dictionary needed amending. This was started during development and before testset was distributed, but not completed for lack of time. Thus the task final results only depict an incomplete system, which has now eventually come to a complete version with rather different outcome. We moved from 98.42 to 99.82 in the testset and from 99.82 to 99.91 in the devset. As said above, this is produced by rules and is not subject to statistical evaluation which may change according to different training sets. In this version of the paper we perform additional experiments with WordForm dictionaries of Italian freely available online

Can humain association norm evaluate latent semantic analysis?

This paper presents the comparison of word association norm created by a psycholinguistic experiment to association lists generated by algorithms operating on text corpora. We compare lists generated by Church and Hanks algorithm and lists generated by LSA algorithm. An argument is presented on how those automatically generated lists reflect real semantic relations

Implementación de un lematizador para una lengua de escasos recursos: caso shipibo-konibo

Desde que el Ministerio de Educación oficializó el alfabeto shipibo-konibo, existe la necesidad de generar una gran cantidad de documentos educativos y oficiales para los hablantes de esta lengua, los cuales solo se realizan actualmente mediante el apoyo de traductores o personas bilingües. Sin embargo, en el campo de la lingüística computacional existen herramientas que permiten facilitar estas labores, como es el caso de un lematizador, el cual se encarga de obtener el lema o forma base de una palabra a partir de su forma flexionada. Su realización se da comúnmente mediante dos métodos: el uso de reglas morfológicas y el uso de diccionarios. Debido a esto, este proyecto tiene como objetivo principal desarrollar una herramienta de lematización para el shipibo-konibo usando un corpus de palabras, la cual se base en los estándares de anotación utilizados en otras lenguas, y que sea fácil de utilizar mediante una librería de funciones y un servicio web. Esta herramienta final se realizó utilizando principalmente el método de clasificación de los k-vecinos más cercanos, el cual permite estimar la clase de un nuevo caso mediante la comparación de sus características con las de casos previamente clasificados y dando como resultado la clase más frecuente para valores similares. Finalmente, la herramienta de lematización desarrollada logró alcanzar una precisión de 0.736 y de esta manera superar a herramientas utilizadas en otros idiomas.Tesi

Italian Lemmatization by Rules with Getaruns

Abstract. We present an approach to lemmatization based on exhaustive morphological analysis and use of external knowledge sources to help disambiguation which is the most relevant issue to cope with. Our system GETARUNS was not concerned with lemmatization directly and used morphological analysis only as backoff solution in case the word was not retrieved in the wordform dictionaries available. We found out that both the rules and the root dictionary needed amending. This was started during development and before testset was distributed, but not completed for lack of time. Thus the task final results only depict an incomplete system, which has now eventually come to a complete version with rather different outcome. We moved from 98.42 to 99.82 in the testset and from 99.85 to 99.91 in the devset. As said above, this is produced by rules and is not subject to statistical evaluation which may change according to different training sets