Article thumbnail
Location of Repository

Definitional, personal and mechanical constraints on part of speech annotation performance

By Anna Babarczy, John Carroll and Geoffrey Sampson

Abstract

For one aspect of grammatical annotation, part-of-speech tagging, we investigate experimentally whether the ceiling on accuracy stems from limits to the precision of tag definition or limits to analysts¿ ability to apply precise definitions, and we examine how analysts¿ performance is affected by alternative types of semi-automatic support. We find that, even for analysts very well-versed in a part-of-speech tagging scheme, human ability to conform to the scheme is a more serious constraint than precision of scheme definition. We also find that although semi-automatic techniques can greatly increase speed relative to manual tagging, they have little effect on accuracy, either positively (by suggesting valid candidate tags) or negatively (by lending an appearance of authority to incorrect tag assignments). On the other hand, it emerges that there are large differences between individual analysts with respect to usability of particular types of semi-automatic support

Year: 2006
OAI identifier: oai:sro.sussex.ac.uk:29715
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • http://dx.doi.org/10.1017/S135... (external link)
  • Suggested articles


    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.