Listwise Explanation of Embedding-Based Rankings via Semantic Chunk Grouping

Hyunkyu Kim, Yeeun Yoo, Youngjun Kwak


Abstract
Dense embedding rankers score documents through contextual sentence- and passage-level representations, yet listwise explanation methods often attribute rankings to isolated words. We study this mismatch and introduce ChunkGroupSHAP, a listwise Shapley method that clusters semantically related chunks across documents into shared features, preserving contextual evidence while bounding the KernelSHAP regression dimension by the group count. Across MS MARCO, FinanceBench, AILACaseDocs, and FinQA with E5-family rankers and BM25, raw chunks improve rank-reconstruction Fidelity over RankSHAP’s word features in all 11 dense-ranker settings. The best chunk-group configuration further improves on raw chunks in eight of these settings, with the incremental benefit depending on grouping scope; word features remain strongest in three of four BM25 settings. These results show that explanation units should match the ranking model: contextual chunks better suit dense bi-encoders, whereas words remain effective for BM25. ChunkGroupSHAP supports listwise attribution over contextual evidence through a bounded feature space shared across documents.
Anthology ID:
2026.findings-emnlp.2230
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
47743–47761
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2230/
DOI:
Bibkey:
Cite (ACL):
Hyunkyu Kim, Yeeun Yoo, and Youngjun Kwak. 2026. Listwise Explanation of Embedding-Based Rankings via Semantic Chunk Grouping. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 47743–47761, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Listwise Explanation of Embedding-Based Rankings via Semantic Chunk Grouping (Kim et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2230.pdf