Mattan Yeroushalmi
Author directory2026
Beyond Accuracy: Improving LLMs’ Science Communication Skills
Mattan Yeroushalmi | Maxim Bragilovski | Nir Grinberg
Findings of the Association for Computational Linguistics: EMNLP 2026
Mattan Yeroushalmi | Maxim Bragilovski | Nir Grinberg
Findings of the Association for Computational Linguistics: EMNLP 2026
Science as a body of knowledge is important for human learning, yet beyond factual accuracy, little research has focused on the capacity of Large Language Models (LLMs) to communicate science effectively to the public. Here, we develop and validate a computational framework for measuring science communication skills of LLMs. Using this framework, we evaluate different alignment approaches: fine-tuning using top-ranked human explanations, distilling synthetic explanations from a teacher model, and preference optimization. Our results show that annotators overwhelmingly prefer LLM-generated science explanations over top-ranked organic answers, with this preference primarily attributed to better structure and readability. We find that distillation with synthetic answers achieves the most significant and stable improvement, allowing a smaller open-source model to achieve statistical parity with the proprietary teacher model in blind side-by-side comparisons.