Pretrained language models (LMs) have significantly advanced a variety of knowledge engineering tasks and have shown promise as sources of factual knowledge. While prior work has studied factual knowledge through probing and prompting, reliably constructing complete knowledge bases from the parametric knowledge of LMs remains challenging. The AKBC 2026 Shared Task invited participants to construct knowledge bases directly from LMs, given specific subjects and relations. Unlike existing probing benchmarks, the challenge imposed no simplifying assumptions on relation cardinality, allowing a subject entity to be associated with zero, one, or multiple objects, including long answer sets. To ensure accessibility, the challenge featured a single track using open-weight LMs with a total neural parameter budget of up to 32B. Nineteen submissions were received, of which twelve were ranked, exploring a variety of approaches including relation-specific prompting, evidence aggregation, and model selection.

Schemas play a vital role in ensuring data quality and supporting usability in the Semantic Web and natural language processing. Traditionally, their creation demands substantial involvement from knowledge engineers and domain experts. Leveraging the impressive capabilities of large language models (LLMs) in tasks like ontology engineering, we explore schema generation using LLMs. To bridge the resource gap, we introduce two datasets: YAGO Schema and Wikidata EntitySchema, along with novel evaluation metrics. The LLM-based pipelines utilize local and global information from knowledge graphs (KGs) to generate schemas in Shape Expressions (ShEx). Experiments demonstrate LLMs’ strong potential in producing high-quality ShEx schemas, paving the way for scalable, automated schema generation for large KGs. Furthermore, our benchmark introduces a new challenge for structured generation, pushing the limits of LLMs on syntactically rich formalisms.