Brucek Khailany

Author directory

2026

Translating sequential programming priors into the parallel temporal logic of hardware design remains a crucial bottleneck for large language models (LLMs). To investigate this, we introduce an operational error taxonomy grounded in the Register-Transfer Level (RTL) coding pipeline. Our taxonomy categorizes failures into extraction/syntax (L1), compilation/elaboration (L2), and testbench-observed functional (L3) stages under a fixed evaluator. Evaluations reveal an empirical ceiling on the VerilogEval benchmark, as frontier models plateau at a 90.8% initial pass rate. Furthermore, we expose a striking surface convergence gap: optimization readily eliminates syntax errors but concurrently exacerbates deeper functional failures. Our findings demonstrate that alignment techniques merely teach models to compile. While test-time scaling can patch solvable errors, RTL coding capacity remains bounded by pretraining knowledge. Addressing challenges in the current LLM-based hardware generation pipeline requires more studies in model reasoning rather than alignment interventions.