scaLAR SemEval-2024 Task 1: Semantic Textual Relatednes for English

Anand Kumar, Hemanth Kumar


Abstract
This study investigates Semantic TextualRelated- ness (STR) within Natural LanguageProcessing (NLP) through experiments conducted on a dataset from the SemEval-2024STR task. The dataset comprises train instances with three features (PairID, Text, andScore) and test instances with two features(PairID and Text), where sentence pairs areseparated by '/n’ in the Text column. UsingBERT(sentence transformers pipeline), we explore two approaches: one with fine-tuning(Track A: Supervised) and another without finetuning (Track B: UnSupervised). Fine-tuningthe BERT pipeline yielded a Spearman correlation coefficient of 0.803, while without finetuning, a coefficient of 0.693 was attained usingcosine similarity. The study concludes by emphasizing the significance of STR in NLP tasks,highlighting the role of pre-trained languagemodels like BERT and Sentence Transformersin enhancing semantic relatedness assessments.
Anthology ID:
2024.semeval-1.129
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
902–906
Language:
URL:
https://aclanthology.org/2024.semeval-1.129
DOI:
10.18653/v1/2024.semeval-1.129
Bibkey:
Cite (ACL):
Anand Kumar and Hemanth Kumar. 2024. scaLAR SemEval-2024 Task 1: Semantic Textual Relatednes for English. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 902–906, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
scaLAR SemEval-2024 Task 1: Semantic Textual Relatednes for English (Kumar & Kumar, SemEval 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.semeval-1.129.pdf
Supplementary material:
 2024.semeval-1.129.SupplementaryMaterial.txt