File(s) not publicly available
Definitional, personal and mechanical constraints on part of speech annotation performance
journal contribution
posted on 2023-06-08, 08:56 authored by Anna Babarczy, John Carroll, Geoffrey SampsonFor one aspect of grammatical annotation, part-of-speech tagging, we investigate experimentally whether the ceiling on accuracy stems from limits to the precision of tag definition or limits to analysts¿ ability to apply precise definitions, and we examine how analysts¿ performance is affected by alternative types of semi-automatic support. We find that, even for analysts very well-versed in a part-of-speech tagging scheme, human ability to conform to the scheme is a more serious constraint than precision of scheme definition. We also find that although semi-automatic techniques can greatly increase speed relative to manual tagging, they have little effect on accuracy, either positively (by suggesting valid candidate tags) or negatively (by lending an appearance of authority to incorrect tag assignments). On the other hand, it emerges that there are large differences between individual analysts with respect to usability of particular types of semi-automatic support.
History
Publication status
- Published
Journal
Journal of Natural Language EngineeringISSN
1351-3249Publisher URL
External DOI
Issue
1Volume
12Page range
77-90Pages
14.0Department affiliated with
- Informatics Publications
Full text available
- No
Peer reviewed?
- Yes
Legacy Posted Date
2012-02-06Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC