Jiayu Wang
Author directoryAlso published as: JIayu Wang
Papers on this page may belong to the following people: Jiayu Wang, Jiayu Wang
2026
MechPunBench: A Bilingual Mechanism-Centered Benchmark for Pun Understanding and Generation
Tianwei Zhang | Zihan Gu | Jiayu Wang | Tianyi Ma | Jiacheng Lu | Jinyu Li | Hua Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Tianwei Zhang | Zihan Gu | Jiayu Wang | Tianyi Ma | Jiacheng Lu | Jinyu Li | Hua Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Puns pose a challenging test case for large language models (LLMs), as their interpretation requires mapping a surface expression to an implicit meaning through a specific pun-forming mechanism. Existing benchmarks typically evaluate isolated abilities such as detection, localization, explanation, or generation quality, leaving unclear whether LLMs can consistently identify, explain, and apply the mechanisms that make puns work. To address this gap, we introduce MECHPUNBENCH, a bilingual, mechanism-oriented benchmark for Chinese and English pun understanding and generation. Each positive instance is annotated with trigger spans, surface meanings, implicit meanings, rhetorical effects, and mechanism labels. Based on a taxonomy of Lexical-Semantic, Structural-Syntactic, and Structural-Semantic puns, we formulate six diagnostic tasks spanning surface recognition to mechanism-guided generation. Experiments on representative closed-source and open-source LLMs reveal a substantial surface–mechanism gap: although the best model achieves 82.8% detection accuracy and 88.2% containment localization accuracy, it remains below the 4/5 threshold used in our evaluation on implicit meaning explanation, mechanism explanation, rhetorical effect explanation, and mechanism-consistent generation. These findings suggest that recognizing puns or generating fluent pun-like text does not necessarily indicate robust mechanism-level pun understanding.