Preserving Memory Function in Hybrid Language Models: A Survey of State, Evidence, Access, and Granularity

Zhuangfei Hu, Runqing Xu, Yuchen Li, Yongqi Zhang


Abstract
Hybrid language models combine Transformer attention with recurrent, state-space, or online-updated components to lower long-context serving cost while preserving memory fidelity. Existing surveys mostly organize this space by architecture family, chronology, or efficiency technique, leaving workload-specific memory functions underspecified. We propose State-Evidence-Access (SEA) for this constraint: bounded State may lose distinctions needed by future queries not fixed in advance. Architectures therefore allocate active State, addressable Evidence, and a per-step Access budget to jointly support reusable structure, query-conditioned recovery, and effective use. This view helps explain why attention ratios or average perplexity can be misleading: a hybrid can save memory while losing the retrieval, state-update, or access function needed by the task. We survey hybrids by allocation locus—from key-value (K-V) and head-level mechanisms through layer, block, external-memory, and system-cache policies—and characterize their *function-granularity mismatch* risks. For workloads such as long-document QA, SEA attributes failure to missing evidence, unstable state updates, access bottlenecks, or poor integration.
Anthology ID:
2026.findings-emnlp.1194
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
25206–25231
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1194/
DOI:
Bibkey:
Cite (ACL):
Zhuangfei Hu, Runqing Xu, Yuchen Li, and Yongqi Zhang. 2026. Preserving Memory Function in Hybrid Language Models: A Survey of State, Evidence, Access, and Granularity. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 25206–25231, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Preserving Memory Function in Hybrid Language Models: A Survey of State, Evidence, Access, and Granularity (Hu et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1194.pdf