Evaluating Compositionality of Sentence Representation Models

Hanoz Bhathena, Angelica Willis, Nathan Dass


Abstract
We evaluate the compositionality of general-purpose sentence encoders by proposing two different metrics to quantify compositional understanding capability of sentence encoders. We introduce a novel metric, Polarity Sensitivity Scoring (PSS), which utilizes sentiment perturbations as a proxy for measuring compositionality. We then compare results from PSS with those obtained via our proposed extension of a metric called Tree Reconstruction Error (TRE) (CITATION) where compositionality is evaluated by measuring how well a true representation producing model can be approximated by a model that explicitly combines representations of its primitives.
Anthology ID:
2020.repl4nlp-1.22
Volume:
Proceedings of the 5th Workshop on Representation Learning for NLP
Month:
July
Year:
2020
Address:
Online
Editors:
Spandana Gella, Johannes Welbl, Marek Rei, Fabio Petroni, Patrick Lewis, Emma Strubell, Minjoon Seo, Hannaneh Hajishirzi
Venue:
RepL4NLP
SIG:
SIGREP
Publisher:
Association for Computational Linguistics
Note:
Pages:
185–193
Language:
URL:
https://aclanthology.org/2020.repl4nlp-1.22
DOI:
10.18653/v1/2020.repl4nlp-1.22
Bibkey:
Cite (ACL):
Hanoz Bhathena, Angelica Willis, and Nathan Dass. 2020. Evaluating Compositionality of Sentence Representation Models. In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 185–193, Online. Association for Computational Linguistics.
Cite (Informal):
Evaluating Compositionality of Sentence Representation Models (Bhathena et al., RepL4NLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.repl4nlp-1.22.pdf
Software:
 2020.repl4nlp-1.22.Software.zip
Video:
 http://slideslive.com/38929788