SemEval-2007 Task 10: English Lexical Substitution Task

McCarthy, Diana and Navigli, Roberto (2007) SemEval-2007 Task 10: English Lexical Substitution Task. In: Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), Prague, Czech Republic.

Full text not available from this repository.
Official URL:


In this paper we describe the English Lexical Substitution task for SemEval. In the task, annotators and systems find an alternative substitute word or phrase for a target word in context. The task involves both finding the synonyms and disambiguating the context. Participating systems are free to use any lexical resource. There is a subtask which requires identifying cases where the word is functioning as part of a multiword in the sentence and detecting what that multiword is.

Item Type: Conference or Workshop Item (Paper)
Additional Information: Originality: This paper describes a new approach for evaluating word sense disambiguation algorithms and the lexical resources that these algorithms rely on by evaluating the ability of a system to find a good substitute for a word in context. In contrast to previous work, the method does not require a man-made inventory. Rigour: We collected annotations from 5 annotators for over 2000 sentences and evaluated systems using our new methodology in an international competition (SemEval 2007). Significance: This paper makes a major contribution because the task makes no requirement on the inventory of word meaning and the task reflects capabilities that will benefit NLP applications such as question answering. Impact: The competition was run in 2007 and we had 8 different international teams participating, with a total of 10 systems. This is a high number of systems for a brand new task. We anticipate that this research will spawn new directions in the evaluation of lexical sense inventories and lexical disambiguation. Outlet: SemEval is the fourth of a successful series of international evaluation workshops, initially called SENSEVAL. This workshop was held at ACL 2007. Full papers were reviewed.
Schools and Departments: School of Engineering and Informatics > Informatics
Depositing User: Diana Frances McCarthy
Date Deposited: 06 Feb 2012 18:34
Last Modified: 12 Apr 2012 09:52
📧 Request an update