Jiho Kim
Author directoryOther people with similar names: Jiho Kim
Unverified author pages with similar names: Jiho Kim
2026
COIN: A Live, Large-scale Benchmark for Semantics-to-Input Code Reasoning
Jiho Kim | Cen Zhang | Youngjoon Kim | Dongkwan Kim | Brian Junekyu Lee | Dae R. Jeong | Taesoo Kim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jiho Kim | Cen Zhang | Youngjoon Kim | Dongkwan Kim | Brian Junekyu Lee | Dae R. Jeong | Taesoo Kim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Constructing inputs that drive execution to chosen code locations underpins many dynamic-analysis tasks in software engineering and security. It is a generalized constraint-solving challenge that requires agents to reason about the semantic mapping between input and code. We built COIN, the first benchmark to systematically evaluate this ability in modern code agents. By leveraging OSS-Fuzz’s long-running coverage data and maintainer-written harnesses, COIN constructs reachability challenges across over 1000 real-world projects in nine languages, and is contamination-resistant and self-refreshing as upstream code and OSS-Fuzz coverage evolve. To keep challenges non-trivial yet solvable, COIN samples two target families: lines OSS-Fuzz has covered but baselines miss, and lines never be reached but with static evidence of a reachable harness path. Across five independent runs on 137 targets from 15 projects in six languages, the strongest agent (Claude Code with Opus 4.6) reaches on average only 26.6% of the C/C++ targets and 39.7% of the Go/JVM/Python/Rust targets, never half of either set in any run, and the weakest (DeepSeek V3.2) stays below 9%, indicating substantial room for improvement.