Yankai Jiang
Author directoryOther people with similar names: Yankai Jiang
2025
Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
Mengliang He | Jiayi Zeng | Yankai Jiang | Wei Zhang | Zeming Liu | Xiaoming Shi | Aimin Zhou
Findings of the Association for Computational Linguistics: ACL 2025
Mengliang He | Jiayi Zeng | Yankai Jiang | Wei Zhang | Zeming Liu | Xiaoming Shi | Aimin Zhou
Findings of the Association for Computational Linguistics: ACL 2025
While large language models (LLMs) show promise in code generation, existing benchmarks neglect the flowchart-based code generation. To promote further research on flowchart-based code generation, this work presents Flow2Code, a novel benchmark for flowchart-based code generation evaluation. The evaluation dataset spans 15 programming languages and includes 5,622 code segments paired with 16,866 flowcharts of three types: code, UML, and pseudocode. Extensive experiments with 13 multimodal LLMs reveal that current LLMs can not generate code based on flowcharts perfectly. Besides, experiment results show that the supervised fine-tuning technique contributes greatly to the models’ performance. The dataset will be publicly available.