Vaishnavi Bhargava
Author directory2026
SWE-Tester: Training Open-Source LLMs for Issue Reproduction in Real-World Repositories
Aditya Bharat Soni | Rajat Ghosh | Vaishnavi Bhargava | Valerie Chen | Debojyoti Dutta
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Aditya Bharat Soni | Rajat Ghosh | Vaishnavi Bhargava | Valerie Chen | Debojyoti Dutta
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Software testing is essential for maintaining software reliability, yet issue reproduction tests are often missing when bugs are reported. Automatically generating such tests can accelerate debugging, support test-driven development, and improve automated software engineering systems that rely on execution-based verification. Existing approaches largely depend on proprietary language models, limiting accessibility and reproducibility. We present SWE-Tester a framework for training open-source large language models to generate issue reproduction tests from natural language issue reports. To support this task, we curate a dataset of 41K training instances from 2.6K open-source GitHub repositories. Fine-tuning on this dataset improves issue reproduction performance across multiple model families, yielding gains of up to 10% points in success rate and 21% in change coverage on SWT-Bench Verified. We further study the effects of model size, training data scale, and inference-time compute, observing consistent improvements along all three axes. Our results demonstrate that open-source models can be effectively specialized for issue reproduction test generation and provide a scalable foundation for future software testing and automated issue-resolution systems.