Selective Compliance in Financial Rule Reasoning: A Solver-Verified Chinese Benchmark and Empirical Analysis

Peng Li, Yinchun Wang


Abstract
Selective compliance is important for financial LLM applications: an applicable context rule should override the LLM’s parametric rule, while an inapplicable rule should be ignored. Though previous studies recognize the importance of rule selection, dedicated evaluations tailored specifically to the financial domain remain absent. To bridge this gap, we build a controlled benchmark based on the CFLUE dataset, where intervention rules are synthesized and FOL logic is applied to ensure the correctness of the new derived answer. Furthermore, we designed three evaluation settings covering clean inputs, noisy inputs, and multi-hop reasoning scenarios. We conduct a systematic comparison across different rule selection strategies and models. Experiments show that: (1) For recent advanced LLMs, single-pass integrated reasoning approach (CAR-Joint) consistently outperforms the decoupled ‘route-then-answer’ approach (CAR-Route). (2) Our proposed post-hoc review method, CAR-Joint-Review, consistently outperforms CAR-Joint. Similarly, pre-hoc routing via the parametric rule elicitation method CAR-Elicit-Route surpasses CAR-Route in multi-hop settings. The performance gains are model-dependent. These findings reveal the trade-offs across various methods and LLMs in financial selective compliance.
Anthology ID:
2026.finnlp-1.44
Volume:
Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Jimin Huang, Yongjae Lee, Sophia Ananiadou, Hsin-Hsi Chen, Hiroya Takamura, Chung-Chi Chen
Venues:
FinNLP | WS
SIG:
SIGFinTech
Publisher:
Association for Computational Linguistics
Note:
Pages:
549–565
Language:
URL:
https://aclanthology.org/2026.finnlp-1.44/
DOI:
Bibkey:
Cite (ACL):
Peng Li and Yinchun Wang. 2026. Selective Compliance in Financial Rule Reasoning: A Solver-Verified Chinese Benchmark and Empirical Analysis. In Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing, pages 549–565, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Selective Compliance in Financial Rule Reasoning: A Solver-Verified Chinese Benchmark and Empirical Analysis (Li & Wang, FinNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.finnlp-1.44.pdf