Guan-Ting Liu
Author directory2026
How LLMs Fail and Generalize in RTL Coding for Hardware Design?
Guan-Ting Liu | Chao-Han Huck Yang | Chenhui Deng | Brucek Khailany | Yu-Chiang Frank Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Guan-Ting Liu | Chao-Han Huck Yang | Chenhui Deng | Brucek Khailany | Yu-Chiang Frank Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Translating sequential programming priors into the parallel temporal logic of hardware design remains a crucial bottleneck for large language models (LLMs). To investigate this, we introduce an operational error taxonomy grounded in the Register-Transfer Level (RTL) coding pipeline. Our taxonomy categorizes failures into extraction/syntax (L1), compilation/elaboration (L2), and testbench-observed functional (L3) stages under a fixed evaluator. Evaluations reveal an empirical ceiling on the VerilogEval benchmark, as frontier models plateau at a 90.8% initial pass rate. Furthermore, we expose a striking surface convergence gap: optimization readily eliminates syntax errors but concurrently exacerbates deeper functional failures. Our findings demonstrate that alignment techniques merely teach models to compile. While test-time scaling can patch solvable errors, RTL coding capacity remains bounded by pretraining knowledge. Addressing challenges in the current LLM-based hardware generation pipeline requires more studies in model reasoning rather than alignment interventions.