Hinoki Crum
2024
hinoki at SemEval-2024 Task 7: Numeral-Aware Headline Generation (English)
Hinoki Crum
|
Steven Bethard
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Numerical reasoning is challenging even for large pre-trained language models. We show that while T5 models are capable of generating relevant headlines with proper numerical values, they can also make mistakes in reading comprehension and miscalculate numerical values. To overcome these issues, we propose a two-step training process: first train models to read text and generate formal representations of calculations, then train models to read calculations and generate numerical values. On the SemEval 2024 Task 7 headline fill-in-the-blank task, our two-stage Flan-T5-based approach achieved 88% accuracy. On the headline generation task, our T5-based approach achieved RougeL of 0.390, BERT F1 Score of 0.453, and MoverScore of 0.587.
Search