A Two-Stage LLM System for Enhanced Regulatory Information Retrieval and Answer Generation

Fengzhao Sun, Jun Yu, Jiaming Hou, Yutong Lin, Tianyu Liu


Abstract
This technical report describes our methodology for the Regulatory Information Retrieval and Answer Generation (RIRAG) Shared Task, a component of the RegNLP workshop at COLING 2025. The challenge aims to effectively navigate and extract relevant information from regulatory texts to generate precise, coherent answers for compliance and obligation-related queries. To tackle subtask1, we introduce a two-stage approach comprising an initial output stage and a subsequent refinement stage. Initially, we fine-tune the LLaMa-2-7B model using LoRA to produce a preliminary output. This is followed by the application of an expert mechanism to enhance the results. For subtask2, we design specific prompt to facilitate the generation of high-quality answers. Consequently, our approach has achieved state-of-the-art performance on the leaderboard, which serves as a testament to the effectiveness and competitiveness of our proposed methodology.
Anthology ID:
2025.regnlp-1.10
Volume:
Proceedings of the 1st Regulatory NLP Workshop (RegNLP 2025)
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Tuba Gokhan, Kexin Wang, Iryna Gurevych, Ted Briscoe
Venues:
RegNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
68–72
Language:
URL:
https://aclanthology.org/2025.regnlp-1.10/
DOI:
Bibkey:
Cite (ACL):
Fengzhao Sun, Jun Yu, Jiaming Hou, Yutong Lin, and Tianyu Liu. 2025. A Two-Stage LLM System for Enhanced Regulatory Information Retrieval and Answer Generation. In Proceedings of the 1st Regulatory NLP Workshop (RegNLP 2025), pages 68–72, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
A Two-Stage LLM System for Enhanced Regulatory Information Retrieval and Answer Generation (Sun et al., RegNLP 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.regnlp-1.10.pdf