Timeline Extraction from Decision Letters Using ChatGPT

Femke Bakker, Ruben Van Heusden, Maarten Marx


Abstract
Freedom of Information Act (FOIA) legislation grants citizens the right to request information from various levels of the government, and aims to promote the transparency of governmental agencies. However, the processing of these requests is often met with delays, due to the inherent complexity of gathering the required documents. To obtain accurate estimates of the processing times of requests, and to identify bottlenecks in the process, this research proposes a pipeline to automatically extract these timelines from decision letters of Dutch FOIA requests. These decision letters are responses to requests, and contain an overview of the process, including when the request was received, and possible communication between the requester and the relevant agency. The proposed pipeline can extract dates with an accuracy of .94, extract event phrases with a mean ROUGE- L F1 score of .80 and can classify events with a macro F1 score of .79.Out of the 50 decision letters used for testing (each letter containing one timeline), the model correctly classified 10 of the timelines completely correct, with an average of 3.1 mistakes per decision letter.
Anthology ID:
2024.case-1.3
Volume:
Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024)
Month:
March
Year:
2024
Address:
St. Julians, Malta
Editors:
Ali Hürriyetoğlu, Hristo Tanev, Surendrabikram Thapa, Gökçe Uludoğan
Venues:
CASE | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
24–31
Language:
URL:
https://aclanthology.org/2024.case-1.3
DOI:
Bibkey:
Cite (ACL):
Femke Bakker, Ruben Van Heusden, and Maarten Marx. 2024. Timeline Extraction from Decision Letters Using ChatGPT. In Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024), pages 24–31, St. Julians, Malta. Association for Computational Linguistics.
Cite (Informal):
Timeline Extraction from Decision Letters Using ChatGPT (Bakker et al., CASE-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.case-1.3.pdf
Supplementary material:
 2024.case-1.3.SupplementaryMaterial.txt