Lexical Substitution as a Task for WSD Evaluation

The paper is intended to promote discussion on potential application oriented methodologies for the next SENSEVAL, and to suggest one possibility for an application-oriented task. Whilst the traditional gold-standard sense-tagging methodology has proved useful in the last two SENSEVALS, the problem of coming up with a satisfactory sense inventory remains, as the choice of the inventory typically creates biases in favour of particular systems. Coupled with the problems that these biases impose is the issue that the inventory, and level of granularity, should reflect the purpose of the application for which the WSD component is intended. In the last SENSEVAL, the Japanese Translation task was a step in this direction. In this paper we will outline some possibilities for a lexical substitution task, and argue that such a task is relevant to several applications to which a WSD system might be applied and would permit participants to select their own inventory.