Jesse Phillips
Other people with similar names: Jesse Phillips
2026
Transformer-Assisted LLM-Based Source Code Summarisation: to Enable More Secure Software Development
Jesse Phillips | Tracy Hall | Paul Rayson | Mo El-Haj
Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
Jesse Phillips | Tracy Hall | Paul Rayson | Mo El-Haj
Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
Neural Source Code Summarisation (NSCS) aims to generate natural language summaries of source code to improve developer and maintainer understanding of code. Source code summaries are vital for the maintenance phase of the Secure Software Development Lifecycle (SSDLC) as they improve maintainers’ understanding of code, in order to reduce the number of bugs and vulnerabilities in a software system. However, summaries are often missing, incomplete, or outdated in many software systems. Solutions to this problem use small, task-specific Transformer models or code-aware Large Language Models (LLMs). Task-specific Transformer-generated summaries often score well across many NLG metrics but these NLG metrics reward lexical overlap, rather than summary quality. Conversely, LLMs’ ability to capture semantics in order to produce high-quality summaries presents an exciting solution to this problem, especially with the increased availability of LLMs and the increase in capability of workstation hardware over recent years meaning that some LLMs can be run from developers’ workstations. However, LLM summaries of code often differ greatly from developer-written summaries in terms of the words and phrases used due to the abstractive nature of LLMs, resulting in low scores across NLG metrics. We show how combining these two methods by using Transformer-generated summaries in prompt engineering may enable LLMs to create better source code summaries in order to better enable software practitioners to maintain secure systems. We prompt four LLMs, using four different prompts - with the use of a task-specific Transformer to aid the LLMs in the prompts. We present "Transformer-Assisted LLM-Based Source Code Summarisation" - a method through which, we observe an improvement of 7.8% BLEU-4 and 5% BERTScore on CodeLlama.