GOT: Testing for Originality in Natural Language Generation

Jennifer Brooks, Abdou Youssef


Abstract
We propose an approach to automatically test for originality in generation tasks where no standard automatic measures exist. Our proposal addresses original uses of language, not necessarily original ideas. We provide an algorithm for our approach and a run-time analysis. The algorithm, which finds all of the original fragments in a ground-truth corpus and can reveal whether a generated fragment copies an original without attribution, has a run-time complexity of theta(nlogn) where n is the number of sentences in the ground truth.
Anthology ID:
2021.gem-1.7
Volume:
Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Antoine Bosselut, Esin Durmus, Varun Prashant Gangal, Sebastian Gehrmann, Yacine Jernite, Laura Perez-Beltrachini, Samira Shaikh, Wei Xu
Venue:
GEM
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
68–72
Language:
URL:
https://aclanthology.org/2021.gem-1.7
DOI:
10.18653/v1/2021.gem-1.7
Bibkey:
Cite (ACL):
Jennifer Brooks and Abdou Youssef. 2021. GOT: Testing for Originality in Natural Language Generation. In Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021), pages 68–72, Online. Association for Computational Linguistics.
Cite (Informal):
GOT: Testing for Originality in Natural Language Generation (Brooks & Youssef, GEM 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.gem-1.7.pdf