15 research outputs found
NLP for writing: What has changed?
Proceedings of the Workshop on NLP for Reading and Writing – Resources,
Algorithms and Tools (SLTC 2008).
Editors: Rickard Domeij, Sofie Johansson Kokkinakis, Ola Knutsson and
Sylvana Sofkova Hashemi.
NEALT Proceedings Series, Vol. 3 (2009), 1-11.
© 2009 The editors and contributors.
Published by
Northern European Association for Language
Technology (NEALT)
http://omilia.uio.no/nealt .
Electronically published at
Tartu University Library (Estonia)
http://hdl.handle.net/10062/4116
Part of Speech Tagging for Text Clustering in Swedish
Proceedings of the 17th Nordic Conference of Computational Linguistics
NODALIDA 2009.
Editors: Kristiina Jokinen and Eckhard Bick.
NEALT Proceedings Series, Vol. 4 (2009), 150-157.
© 2009 The editors and contributors.
Published by
Northern European Association for Language
Technology (NEALT)
http://omilia.uio.no/nealt .
Electronically published at
Tartu University Library (Estonia)
http://hdl.handle.net/10062/9206
Evaluation of really good grammatical error correction
Although rarely stated, in practice, Grammatical Error Correction (GEC)
encompasses various models with distinct objectives, ranging from grammatical
error detection to improving fluency. Traditional evaluation methods fail to
fully capture the full range of system capabilities and objectives.
Reference-based evaluations suffer from limitations in capturing the wide
variety of possible correction and the biases introduced during reference
creation and is prone to favor fixing local errors over overall text
improvement. The emergence of large language models (LLMs) has further
highlighted the shortcomings of these evaluation strategies, emphasizing the
need for a paradigm shift in evaluation methodology. In the current study, we
perform a comprehensive evaluation of various GEC systems using a recently
published dataset of Swedish learner texts. The evaluation is performed using
established evaluation metrics as well as human judges. We find that GPT-3 in a
few-shot setting by far outperforms previous grammatical error correction
systems for Swedish, a language comprising only 0.11% of its training data. We
also found that current evaluation methods contain undesirable biases that a
human evaluation is able to reveal. We suggest using human post-editing of GEC
system outputs to analyze the amount of change required to reach native-level
human performance on the task, and provide a dataset annotated with human
post-edits and assessments of grammaticality, fluency and meaning preservation
of GEC system outputs
Linguistically Fuelled Text Similarity
Proceedings of the 16th Nordic Conference
of Computational Linguistics NODALIDA-2007.
Editors: Joakim Nivre, Heiki-Jaan Kaalep, Kadri Muischnek and Mare Koit.
University of Tartu, Tartu, 2007.
ISBN 978-9985-4-0513-0 (online)
ISBN 978-9985-4-0514-7 (CD-ROM)
pp. 207-211
Recreating Humorous Split Compound Errors in Swedish by Using Grammaticality
Proceedings of the 16th Nordic Conference
of Computational Linguistics NODALIDA-2007.
Editors: Joakim Nivre, Heiki-Jaan Kaalep, Kadri Muischnek and Mare Koit.
University of Tartu, Tartu, 2007.
ISBN 978-9985-4-0513-0 (online)
ISBN 978-9985-4-0514-7 (CD-ROM)
pp. 389-393
Developing and Evaluating a Searchable Swedish-Thai Lexicon
Proceedings of the 16th Nordic Conference
of Computational Linguistics NODALIDA-2007.
Editors: Joakim Nivre, Heiki-Jaan Kaalep, Kadri Muischnek and Mare Koit.
University of Tartu, Tartu, 2007.
ISBN 978-9985-4-0513-0 (online)
ISBN 978-9985-4-0514-7 (CD-ROM)
pp. 324-328
Erroreak automatikoki detektatzeko tekniken azterlana eta euskararentzako aplikazioak
In this article, we study the techniques used for detecting errors in Natural Language Processing (NLP). We classify the techniques according to their approach (symbolic or empirical), and then we describe them in depth. Following that, we describe the systems we have developed for detecting syntactic errors in Basque, by using that technique as a criterion for the classification of those systems, and enhancing it with examples
Proceedings (all articles)
Proceedings of the Workshop on NLP for Reading and Writing – Resources,
Algorithms and Tools (SLTC 2008).
Editors: Rickard Domeij, Sofie Johansson Kokkinakis, Ola Knutsson and
Sylvana Sofkova Hashemi.
NEALT Proceedings Series, Vol. 3 (2009), v+23 pp.
© 2009 The editors and contributors.
Published by
Northern European Association for Language Technology (NEALT)
http://omilia.uio.no/nealt .
Electronically published at
Tartu University Library (Estonia)
http://hdl.handle.net/10062/4116