Which Sentence Representation is More Informative: An Analysis on Text Classification

Necva Bölücü, Burcu Can


Abstract
Text classification is a popular and well-studied problem in Natural Language Processing. Most previous work on text classification has focused on deep neural networks such as LSTMs and CNNs. However, text classification studies using syntactic and semantic information are very limited in the literature. In this study, we propose a model using Graph Attention Network (GAT) that incorporates semantic and syntactic information as input for the text classification task. The semantic representations of UCCA and AMR are used as semantic information and the dependency tree is used as syntactic information. Extensive experimental results and in-depth analysis show that UCCA-GAT model, which is a semantic-aware model outperforms the AMR-GAT and DEP-GAT, which are semantic and syntax-aware models respectively. We also provide a comprehensive analysis of the proposed model to understand the limitations of the representations for the problem.
Anthology ID:
2023.depling-1.2
Volume:
Proceedings of the Seventh International Conference on Dependency Linguistics (Depling, GURT/SyntaxFest 2023)
Month:
March
Year:
2023
Address:
Washington, D.C.
Editors:
Owen Rambow, François Lareau
Venues:
DepLing | SyntaxFest
SIG:
SIGPARSE
Publisher:
Association for Computational Linguistics
Note:
Pages:
9–21
Language:
URL:
https://aclanthology.org/2023.depling-1.2
DOI:
Bibkey:
Cite (ACL):
Necva Bölücü and Burcu Can. 2023. Which Sentence Representation is More Informative: An Analysis on Text Classification. In Proceedings of the Seventh International Conference on Dependency Linguistics (Depling, GURT/SyntaxFest 2023), pages 9–21, Washington, D.C.. Association for Computational Linguistics.
Cite (Informal):
Which Sentence Representation is More Informative: An Analysis on Text Classification (Bölücü & Can, DepLing-SyntaxFest 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.depling-1.2.pdf
Video:
 https://aclanthology.org/2023.depling-1.2.mp4