Despite their generalization capabilities, large language models (LLMs) have counterintuitively underperformed smaller pre-trained language models (PLMs) on few-shot relation extraction (RE), the task of identifying semantic relationships between entities in text. This gap is primarily due to a systematic over-prediction bias in LLMs. We propose Distill-RE, a two-stage knowledge distillation framework in which a teacher LLM generates chain-of-thought reasoning over existing training examples to supervise two student models: a classification model that predicts relations and a confirmation model that filters false positives. Across student scales from 3B to 70B, Distill-RE establishes a new state-of-the-art among RE models, with its top-performing configuration outperforming the strongest baseline by over 31.5 micro-F1 points averaged across TACRED, TACREV, and Re-TACRED at 8-, 16-, and 32-shot settings. Notably, without synthesizing new relation instances, it outperforms most data augmentation baselines and comes within 1.8 micro-F1 points on average of the strongest.

Relation classification (RC) transforms unstructured text into structured knowledge by identifying semantic relationships between entities. As natural language data volumes rapidly increase, manual knowledge extraction has become impractical. Existing zero-shot LLM-based approaches face limitations due to sparse RC task representation in instruction-tuning datasets and inherent biases toward certain relation types. To address these challenges, we propose Binary Relevance Elicitation (BRE), a true/false prompting framework that systematically queries an LLM for each candidate relation between two entities from a closed set of relation types. BRE incorporates relevance scoring to identify the most likely relation among those classified as "true," reducing false positive predictions. We evaluate BRE using multiple open-source LLMs and achieve state-of-the-art zero-shot RC accuracy across the TACRED family of datasets. Our results demonstrate performance improvements with newer model releases that are independent of parameter count, suggesting information representation advances within LLMs drive zero-shot RC capabilities.