CVAE-based Re-anchoring for Implicit Discourse Relation Classification

Zujun Dou, Yu Hong, Yu Sun, Guodong Zhou


Abstract
Training implicit discourse relation classifiers suffers from data sparsity. Variational AutoEncoder (VAE) appears to be the proper solution. It is because ideally VAE is capable of generating inexhaustible varying samples, and this facilitates selective data augmentation. However, our experiments show that coupling VAE with the RoBERTa-based classifier results in severe performance degradation. We ascribe the unusual phenomenon to erroneous sampling that would happen when VAE pursued variations. To overcome the problem, we develop a re-anchoring strategy, where Conditional VAE (CVAE) is used for estimating the risk of erroneous sampling, and meanwhile migrating the anchor to reduce the risk. The test results on PDTB v2.0 illustrate that, compared to the RoBERTa-based baseline, re-anchoring yields substantial improvements. Besides, we observe that re-anchoring can cooperate with other auxiliary strategies (transfer learning and interactive attention mechanism) to further improve the baseline, obtaining the F-scores of about 55%, 63%, 80% and 44% for the four main relation types (Comparison, Contingency, Expansion, Temporality) in the binary classification (Yes/No) scenario.
Anthology ID:
2021.findings-emnlp.110
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1275–1283
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.110
DOI:
10.18653/v1/2021.findings-emnlp.110
Bibkey:
Cite (ACL):
Zujun Dou, Yu Hong, Yu Sun, and Guodong Zhou. 2021. CVAE-based Re-anchoring for Implicit Discourse Relation Classification. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1275–1283, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
CVAE-based Re-anchoring for Implicit Discourse Relation Classification (Dou et al., Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.110.pdf
Software:
 2021.findings-emnlp.110.Software.zip
Video:
 https://aclanthology.org/2021.findings-emnlp.110.mp4