Mikołaj Tym

Author directory

2026

Extracting facts from the parametric memory of Large Language Models (LLMs) for Automated Knowledge Base Completion (AKBC) under closed-book constraints remains a significant challenge. We describe our system for the 2026 AKBC Shared Task, where retrieval, external knowledge sources and fine-tuning are not permitted. We first evaluate 18 open-weight models on the training set to check how much the choice of the model matters for this task. The results differ significantly, from 0.186 to 0.499 macro F1, and the best model is not the largest one. Moreover, the models that score well on coding benchmarks tend to score worse on knowledge base construction. We then introduce Pre-training Aware Directed Reasoning (PADR), a set of relation-specific reasoning pipelines optimized for the best evaluated model, Mistral Small 3.2 24B. PADR improves the macro F1 score on the training set from 0.499 to 0.614. On the shared task’s test set, PADR reaches 0.629 macro F1, more than double the official baseline’s 0.296. The code is available at https://github.com/MikolajTym/akbc-padr.
Search
Co-authors
    Venues
    Fix author