Jeongwon Kwak


2023

pdf bib
Context and Literacy Aware Learnable Metric for Text Simplification
Jeongwon Kwak | Hyeryun Park | Kyungmo Kim | Jinwook Choi
Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)

Automatic evaluation of text simplification is important; but assessing its transformation into simpler sentences can be challenging for various reasons. However, the most commonly used metric in text simplification, SARI, fails to capture the difficulty of generating words that are not present in the references, regardless of their meaning. We propose a new learnable evaluation metric that decomposes and reconstructs sentences to simultaneously measure the similarity and difficulty of sentences within a single system. Through experiments, we confirm that it exhibited the highest similarity in correlation with the human evaluation.