Driving Context into Text-to-Text Privatization

Stefan Arnold, Dilara Yesilbas, Sven Weinzierl


Abstract
Metric Differential Privacy enables text-to-text privatization by adding calibrated noise to the vector of a word derived from an embedding space and projecting this noisy vector back to a discrete vocabulary using a nearest neighbor search. Since words are substituted without context, this mechanism is expected to fall short at finding substitutes for words with ambiguous meanings, such as ‘bank’. To account for these ambiguous words, we leverage a sense embedding and incorporate a sense disambiguation step prior to noise injection. We encompass our modification to the privatization mechanism with an estimation of privacy and utility. For word sense disambiguation on the Words in Context dataset, we demonstrate a substantial increase in classification accuracy by 6.05%.
Anthology ID:
2023.trustnlp-1.2
Volume:
Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anaelia Ovalle, Kai-Wei Chang, Ninareh Mehrabi, Yada Pruksachatkun, Aram Galystan, Jwala Dhamala, Apurv Verma, Trista Cao, Anoop Kumar, Rahul Gupta
Venue:
TrustNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
15–25
Language:
URL:
https://aclanthology.org/2023.trustnlp-1.2
DOI:
10.18653/v1/2023.trustnlp-1.2
Bibkey:
Cite (ACL):
Stefan Arnold, Dilara Yesilbas, and Sven Weinzierl. 2023. Driving Context into Text-to-Text Privatization. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 15–25, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Driving Context into Text-to-Text Privatization (Arnold et al., TrustNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.trustnlp-1.2.pdf
Supplementary material:
 2023.trustnlp-1.2.SupplementaryMaterial.zip
Video:
 https://aclanthology.org/2023.trustnlp-1.2.mp4