Solving Historical Dictionary Codes with a Neural Language Model

Christopher Chu, Raphael Valenti, Kevin Knight


Abstract
We solve difficult word-based substitution codes by constructing a decoding lattice and searching that lattice with a neural language model. We apply our method to a set of enciphered letters exchanged between US Army General James Wilkinson and agents of the Spanish Crown in the late 1700s and early 1800s, obtained from the US Library of Congress. We are able to decipher 75.1% of the cipher-word tokens correctly.
Anthology ID:
2020.emnlp-main.471
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5845–5854
Language:
URL:
https://aclanthology.org/2020.emnlp-main.471
DOI:
10.18653/v1/2020.emnlp-main.471
Bibkey:
Cite (ACL):
Christopher Chu, Raphael Valenti, and Kevin Knight. 2020. Solving Historical Dictionary Codes with a Neural Language Model. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5845–5854, Online. Association for Computational Linguistics.
Cite (Informal):
Solving Historical Dictionary Codes with a Neural Language Model (Chu et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.471.pdf
Video:
 https://slideslive.com/38939072