TLDR at SemEval-2024 Task 2: T5-generated clinical-Language summaries for DeBERTa Report Analysis

Spandan Das, Vinay Samuel, Shahriar Noroozizadeh


Abstract
This paper introduces novel methodologies for the Natural Language Inference for Clinical Trials (NLI4CT) task. We present TLDR (T5-generated clinical-Language summaries for DeBERTa Report Analysis) which incorporates T5-model generated premise summaries for improved entailment and contradiction analysis in clinical NLI tasks. This approach overcomes the challenges posed by small context windows and lengthy premises, leading to a substantial improvement in Macro F1 scores: a 0.184 increase over truncated premises. Our comprehensive experimental evaluation, including detailed error analysis and ablations, confirms the superiority of TLDR in achieving consistency and faithfulness in predictions against semantically altered inputs.
Anthology ID:
2024.semeval-1.79
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
520–529
Language:
URL:
https://aclanthology.org/2024.semeval-1.79
DOI:
Bibkey:
Cite (ACL):
Spandan Das, Vinay Samuel, and Shahriar Noroozizadeh. 2024. TLDR at SemEval-2024 Task 2: T5-generated clinical-Language summaries for DeBERTa Report Analysis. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 520–529, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
TLDR at SemEval-2024 Task 2: T5-generated clinical-Language summaries for DeBERTa Report Analysis (Das et al., SemEval 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.semeval-1.79.pdf
Supplementary material:
 2024.semeval-1.79.SupplementaryMaterial.txt
Supplementary material:
 2024.semeval-1.79.SupplementaryMaterial.zip