Hybrid PIR: Balancing Token Compression and Structural Invariance via Pāṇinian Role Assignment

Kriti Bharadwaj, Gowri Srinivasa


Abstract
Neural text representations often lack structural guarantees for syntactic variations, relying instead on statistical interpolation to recognize paraphrases. We introduce the Pāṇinian Intermediate Representation (PIR), a deterministic compiler that maps English sentences to role-tagged canonical sequences using Universal Dependency parses. Unlike heuristic methods such as stopword removal, the PIR compiler is deterministic: given a correct dependency parse, syntactically equivalent paraphrases compile to identical strings. On a 200-pair diagnostic suite of templated syntactic alternations spanning active/passive voice, dative shift, and topicalization, the compiler produces identical strings for 196 pairs; all four failures are structurally typed and traced to known parser limitations. Stopword removal produces identical strings for 0 of 200 pairs. To bridge the performance gap in downstream tasks, we propose Hybrid PIR, which selectively restores lexical content through a deduplication mechanism. Hybrid PIR achieves similarity performance comparable to stopword removal (0.8130 vs. 0.8154 average ρ; bootstrap 95% CI on the difference includes zero) at 43.8% token compression, while providing structural guarantees that surface-level methods cannot. On retrieval benchmarks, Hybrid PIR reaches 80% of the English baseline nDCG@10 on BEIR SciFact and 86% of MRR@10 on MS MARCO QA at 39.0% compression. A three-variant representation ablation validates our deduplication design: removing this mechanism degrades retrieval performance by 3.5 points and turns the compression benefit negative (-3.0%). In a QQP-validation duplicate-detection use case (40,430 pairs), Hybrid PIR reaches precision 0.861 with no tuning, matching a tuned embedding threshold and validating the intended deployment as an offline canonicalization tool for deduplication and index construction.
Anthology ID:
2026.findings-emnlp.1647
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
35057–35070
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1647/
DOI:
Bibkey:
Cite (ACL):
Kriti Bharadwaj and Gowri Srinivasa. 2026. Hybrid PIR: Balancing Token Compression and Structural Invariance via Pāṇinian Role Assignment. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 35057–35070, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Hybrid PIR: Balancing Token Compression and Structural Invariance via Pāṇinian Role Assignment (Bharadwaj & Srinivasa, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1647.pdf