Learning Bill Similarity with Annotated and Augmented Corpora of Bills

Jiseon Kim, Elden Griggs, In Song Kim, Alice Oh


Abstract
Bill writing is a critical element of representative democracy. However, it is often overlooked that most legislative bills are derived, or even directly copied, from other bills. Despite the significance of bill-to-bill linkages for understanding the legislative process, existing approaches fail to address semantic similarities across bills, let alone reordering or paraphrasing which are prevalent in legal document writing. In this paper, we overcome these limitations by proposing a 5-class classification task that closely reflects the nature of the bill generation process. In doing so, we construct a human-labeled dataset of 4,721 bill-to-bill relationships at the subsection-level and release this annotated dataset to the research community. To augment the dataset, we generate synthetic data with varying degrees of similarity, mimicking the complex bill writing process. We use BERT variants and apply multi-stage training, sequentially fine-tuning our models with synthetic and human-labeled datasets. We find that the predictive performance significantly improves when training with both human-labeled and synthetic data. Finally, we apply our trained model to infer section- and bill-level similarities. Our analysis shows that the proposed methodology successfully captures the similarities across legal documents at various levels of aggregation.
Anthology ID:
2021.emnlp-main.787
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10048–10064
Language:
URL:
https://aclanthology.org/2021.emnlp-main.787
DOI:
10.18653/v1/2021.emnlp-main.787
Bibkey:
Cite (ACL):
Jiseon Kim, Elden Griggs, In Song Kim, and Alice Oh. 2021. Learning Bill Similarity with Annotated and Augmented Corpora of Bills. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10048–10064, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Learning Bill Similarity with Annotated and Augmented Corpora of Bills (Kim et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.787.pdf
Software:
 2021.emnlp-main.787.Software.zip
Video:
 https://aclanthology.org/2021.emnlp-main.787.mp4
Code
 hikoseon12/learning-bill-similarity