Mikołaj Tym
Author directory2026
Pre-training Aware Directed Reasoning for Closed-Book Knowledge Base Construction
Mikołaj Tym
Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026)
Mikołaj Tym
Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026)
Extracting facts from the parametric memory of Large Language Models (LLMs) for Automated Knowledge Base Completion (AKBC) under closed-book constraints remains a significant challenge. We describe our system for the 2026 AKBC Shared Task, where retrieval, external knowledge sources and fine-tuning are not permitted. We first evaluate 18 open-weight models on the training set to check how much the choice of the model matters for this task. The results differ significantly, from 0.186 to 0.499 macro F1, and the best model is not the largest one. Moreover, the models that score well on coding benchmarks tend to score worse on knowledge base construction. We then introduce Pre-training Aware Directed Reasoning (PADR), a set of relation-specific reasoning pipelines optimized for the best evaluated model, Mistral Small 3.2 24B. PADR improves the macro F1 score on the training set from 0.499 to 0.614. On the shared task’s test set, PADR reaches 0.629 macro F1, more than double the official baseline’s 0.296. The code is available at https://github.com/MikolajTym/akbc-padr.