Comparing Similarity Measures for Distributional Thesauri

Muntsa Padró, Marco Idiart, Aline Villavicencio, Carlos Ramisch


Abstract
Distributional thesauri have been applied for a variety of tasks involving semantic relatedness. In this paper, we investigate the impact of three parameters: similarity measures, frequency thresholds and association scores. We focus on the robustness and stability of the resulting thesauri, measuring inter-thesaurus agreement when testing different parameter values. The results obtained show that low-frequency thresholds affect thesaurus quality more than similarity measures, with more agreement found for increasing thresholds. These results indicate the sensitivity of distributional thesauri to frequency. Nonetheless, the observed differences do not transpose over extrinsic evaluation using TOEFL-like questions. While this may be specific to the task, we argue that a careful examination of the stability of distributional resources prior to application is needed.
Anthology ID:
L14-1496
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2964–2971
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/619_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Muntsa Padró, Marco Idiart, Aline Villavicencio, and Carlos Ramisch. 2014. Comparing Similarity Measures for Distributional Thesauri. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 2964–2971, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
Comparing Similarity Measures for Distributional Thesauri (Padró et al., LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/619_Paper.pdf