Improving sparse word representations with distributional inference for semantic composition

Kober, Thomas, Weeds, Julie, Reffin, Jeremy and Weir, David (2016) Improving sparse word representations with distributional inference for semantic composition. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TX, 1-5 November 2016. Published in: Unset. 1691-1702. Association for Computational Linguistics ISBN 9781945626258

[img] PDF - Published Version
Available under License Creative Commons Attribution.

Download (237kB)


Distributional models are derived from co- occurrences in a corpus, where only a small proportion of all possible plausible co-occurrences will be observed. This results in a very sparse vector space, requiring a mechanism for inferring missing knowledge. Most methods face this challenge in ways that render the resulting word representations uninterpretable, with the consequence that semantic composition becomes hard to model. In this paper we explore an alternative which involves explicitly inferring unobserved co-occurrences using the distributional neighbourhood. We show that distributional inference improves sparse word repre- sentations on several word similarity benchmarks and demonstrate that our model is competitive with the state-of-the-art for adjective- noun, noun-noun and verb-object compositions while being fully interpretable.

Item Type: Conference Proceedings
Schools and Departments: School of Engineering and Informatics > Informatics
Research Centres and Groups: Data Science Research Group
Subjects: Q Science > Q Science (General)
Q Science > QA Mathematics
Depositing User: Thomas Kober
Date Deposited: 20 Feb 2017 12:58
Last Modified: 05 Aug 2020 08:25

View download statistics for this item

📧 Request an update