Fine-Tuning Small Language Models for Cybersecurity: Data Ordering, Knowledge Distillation, and the Educator Effect

Ozkan Kilic, Raja Soundaramourty, Ramu Chenchaiah


Abstract
Data-sovereignty rules forbid cloud-hosted AI in many high-security environments, leaving compact on-premise models as the only path to AI-assisted cybersecurity. We fine-tune three small open-source models, Gemma 2 2B, Phi-3.5 3.8B, Llama 3.1 8B, on ~147,600 synthetic cybersecurity QA pairs using QLoRA on V100 GPUs. Under strict MCQ evaluation Gemma 2 gains +9.3 pp, Phi +4.0 pp, and Llama drops −24.0 pp. We term this the educator effect: models trained on pedagogical data internalize explanatory behavior at the expense of format compliance. Severity appears to scale with capacity, though capacity is confounded with architecture and learning rate. A controlled ablation shows randomized ordering outperforms curriculum, without significance testing on the 75-question exam.
Anthology ID:
2026.nlpaics-1.6
Volume:
Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
Month:
June
Year:
2026
Address:
Alicante, Spain
Editors:
Ruslan Mitkov, Rafael Muñoz, Elena Lloret, Tharindu Ranasinghe, Ernesto L. Estevanell-Valladares, Salima Lamsiyah, Andrés Montoyo, Saad Ezzini
Venue:
NLPAICS
SIG:
Publisher:
Department of Languages and Information Systems, University of Alicante
Note:
Pages:
55–63
Language:
URL:
https://aclanthology.org/2026.nlpaics-1.6/
DOI:
Bibkey:
Cite (ACL):
Ozkan Kilic, Raja Soundaramourty, and Ramu Chenchaiah. 2026. Fine-Tuning Small Language Models for Cybersecurity: Data Ordering, Knowledge Distillation, and the Educator Effect. In Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security, pages 55–63, Alicante, Spain. Department of Languages and Information Systems, University of Alicante.
Cite (Informal):
Fine-Tuning Small Language Models for Cybersecurity: Data Ordering, Knowledge Distillation, and the Educator Effect (Kilic et al., NLPAICS 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.nlpaics-1.6.pdf
Optionalsupplementarymaterial:
 2026.nlpaics-1.6.OptionalSupplementaryMaterial.docx