Empowering SW Security: CodeBERT and Machine Learning Approaches to Vulnerability Detection

Lov Kumar, Vikram Singh, Srivalli Patel, Pratyush Mishra


Abstract
Software (SW) systems experience faults after deployment, raising concerns about reliability and leading to financial losses, reputational damage, and safety risks. This paper presents a novel approach using CodeBERT, a state-of-the-art neural code representation model pre-trained in multi-programming languages and employs various code metrics to predict SW faults. The study comprehensively evaluates trained models by analyzing publicly available codebase and employing diverse machine learning models, feature selection techniques, and class balancing through SMOTE. The results show that SMOTE significantly enhances vulnerability detection performance, particularly in accuracy, AUC, sensitivity, and specificity. The EXTR classifier consistently outperforms others, with an average AUC of 0.82, and the features selected using the GA feature selection technique, despite achieving a mean AUC of 0.84. Interestingly, among employed embedding techniques, SW metrics combined with CodeBERT (SMCBERT) stand out as top performers, achieving the highest mean AUC score of 0.80, making models trained on SMCBERT the best for SW vulnerability prediction.
Anthology ID:
2024.icon-1.46
Volume:
Proceedings of the 21st International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2024
Address:
AU-KBC Research Centre, Chennai, India
Editors:
Sobha Lalitha Devi, Karunesh Arora
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
399–407
Language:
URL:
https://aclanthology.org/2024.icon-1.46/
DOI:
Bibkey:
Cite (ACL):
Lov Kumar, Vikram Singh, Srivalli Patel, and Pratyush Mishra. 2024. Empowering SW Security: CodeBERT and Machine Learning Approaches to Vulnerability Detection. In Proceedings of the 21st International Conference on Natural Language Processing (ICON), pages 399–407, AU-KBC Research Centre, Chennai, India. NLP Association of India (NLPAI).
Cite (Informal):
Empowering SW Security: CodeBERT and Machine Learning Approaches to Vulnerability Detection (Kumar et al., ICON 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.icon-1.46.pdf