The Interplay of Harness Design and Post-training in LLM Agents

Kyungmin Kim, Youngbin Choi, Seoyeon Lee, Suhyeon Jun, Dongwoo Kim, Sangdon Park


Abstract
Tool-integrated LLM agents are often wrapped within a *harness*: the scaffolding that determines which tools are exposed, how they are described, and what auxiliary information accompanies each per-step observation. While agents are routinely post-trained, this scaffolding is typically treated as a *fixed* engineering detail, with design effort limited to the training-free regime. Moreover, existing post-training algorithms assume a static environment, even though tool environments and tasks often shift upon deployment. To address this gap, we extend ALFWorld (i) to treat the harness as a controllable design dimension and (ii) to support evaluation under task and tool environment shifts. Building on this, we systematically analyze how the harness design influences post-training in both in-distribution and out-of-distribution (OOD) settings. We empirically show that *harness-aware post-training* not only improves in-distribution performance but also enables agents to robustly adapt to OOD settings. Under a harness with minimal design effort, post-training suffers a drastic performance drop under stronger tool environment shifts, further highlighting the importance of harness-aware post-training under such shifts.
Anthology ID:
2026.findings-emnlp.2207
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
47275–47299
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2207/
DOI:
Bibkey:
Cite (ACL):
Kyungmin Kim, Youngbin Choi, Seoyeon Lee, Suhyeon Jun, Dongwoo Kim, and Sangdon Park. 2026. The Interplay of Harness Design and Post-training in LLM Agents. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 47275–47299, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
The Interplay of Harness Design and Post-training in LLM Agents (Kim et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2207.pdf