Exemplars Primarily Shift Retrieval, Verification Instructions Can Shift Rejection: An Anatomy of Inference-Time Interventions for Closed-Book Knowledge Base Construction

Takashi Sasaki


Abstract
We describe our AKBC 2026 closed-book knowledge base construction system: Gemma-3-27B-pt, raw text completion, 20 candidates per query, and relation-specific aggregation without an arbiter. It uses 27.4B parameters and scores 0.6670 row-averaged Macro F1 on the official test, placing sixth. Experiments distinguish this final pipeline from an earlier Mistral system with an 8B arbiter. Backbone and access format outweighed prompt-level interventions: under matched prompt formats, the pretrained checkpoint exceeds its instruction-tuned sibling by 0.11 on validation. Numeric example configurations primarily shift retrieval; procedural wording with five added JSON examples changes the selected answers by only one row. Verification instructions can shift rejection. A biography-first city prompt format affects both, raising validation F1 from 0.55 to 0.61 at a fixed vote gate. Gating, information outside the model, and seed stability help interpret transfer. These findings have counterexamples; leaderboard comparisons describe development outcomes, not an untouched final evaluation.
Anthology ID:
2026.akbc-1.36
Volume:
Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026)
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Jan-Christoph Kalo, Ndapa Nakashole, Simon Razniewski, Fabian M. Suchanek, Andreas Vlachos, Andrew McCallum
Venues:
AKBC | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
473–479
Language:
URL:
https://aclanthology.org/2026.akbc-1.36/
DOI:
Bibkey:
Cite (ACL):
Takashi Sasaki. 2026. Exemplars Primarily Shift Retrieval, Verification Instructions Can Shift Rejection: An Anatomy of Inference-Time Interventions for Closed-Book Knowledge Base Construction. In Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026), pages 473–479, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Exemplars Primarily Shift Retrieval, Verification Instructions Can Shift Rejection: An Anatomy of Inference-Time Interventions for Closed-Book Knowledge Base Construction (Sasaki, AKBC 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.akbc-1.36.pdf