SD-ICL: Stage-Decomposed In-Context Learning for Tool Use in LLM Agents

Kangci Xiong, Xiaohan Ge, Yangyang Qin, Fusheng Jin


Abstract
Large language models (LLMs) augmented with external tools have demonstrated strong capabilities in API selection, invocation, and reasoning over tool outputs. However, existing training-free tool-use methods usually rely on inference-time context construction to guide the entire tool-use process with a unified input, including tool descriptions, task prompts, demonstrations, trajectories, or planning hints. Tool-use decisions place different emphasis on this contextual evidence: tool/API selection emphasizes task–API correspondence, while call generation additionally uses API schemas, local call transitions, and intermediate observations. We present SD-ICL, a Stage-Decomposed In-Context Learning method that decomposes context construction into tool selection and call generation stages. SD-ICL first retrieves tool selection demonstrations by combining task semantics with tool set relevance and constructs an explicit tool selection state. It then uses this state to retrieve call generation demonstrations through API-set similarity and adjacent call relations, organizing complementary context for the executor. Experiments show that SD-ICL improves task completion and preference quality across the evaluated tool-use benchmarks and model settings.
Anthology ID:
2026.findings-emnlp.249
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
5237–5254
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.249/
DOI:
Bibkey:
Cite (ACL):
Kangci Xiong, Xiaohan Ge, Yangyang Qin, and Fusheng Jin. 2026. SD-ICL: Stage-Decomposed In-Context Learning for Tool Use in LLM Agents. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 5237–5254, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
SD-ICL: Stage-Decomposed In-Context Learning for Tool Use in LLM Agents (Xiong et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.249.pdf