Hanwen Tong
Author directory2026
Towards Smart Exploration: Reinforcement Learning for LLM Agents Guided by Meta-Actions
Hanwen Tong | Zhengqi Zhang | Ruixuan Lu | Jincheng Dai | Bin Yang | Bo Zheng | Feng Wei
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Hanwen Tong | Zhengqi Zhang | Ruixuan Lu | Jincheng Dai | Bin Yang | Bo Zheng | Feng Wei
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Reinforcement Learning (RL) has empowered Large Language Model (LLM) agents to tackle long-horizon tasks through multi-step interactions with the environment. However, current LLM agents still rely on inefficient trial-and-error exploration, leading to frequent task failures and redundant actions. In this paper, we propose MetaAct, an RL framework that incorporates a set of high-level meta-actions to explicitly guide the agent towards promising trajectories and facilitate goal-aware exploration. Specifically, the agent is required to predict one of four meta-actions (FOCUS, GUESS, RESUME, UNDO) to constrain the direction of its subsequent actions to minimize unproductive trials. More importantly, we employ group-based RL with episode-level and step-level advantages to fine-tune the agent, enabling it to identify strategic directions more accurately and generate effective task-specific actions. Experimental results demonstrate that our method consistently outperforms the state-of-the-art approaches on ALFWorld and WebShop, achieving higher success rates with fewer steps. Our code and data are publicly available at https://github.com/HavenTong/MetaAct.
2022
Parsing Natural Language into Propositional and First-Order Logic with Dual Reinforcement Learning
Xuantao Lu | Jingping Liu | Zhouhong Gu | Hanwen Tong | Chenhao Xie | Junyang Huang | Yanghua Xiao | Wenguang Wang
Proceedings of the 29th International Conference on Computational Linguistics
Xuantao Lu | Jingping Liu | Zhouhong Gu | Hanwen Tong | Chenhao Xie | Junyang Huang | Yanghua Xiao | Wenguang Wang
Proceedings of the 29th International Conference on Computational Linguistics
Semantic parsing converts natural language utterances into structured logical expressions. We consider two such formal representations: Propositional Logic (PL) and First-order Logic (FOL). The paucity of labeled data is a major challenge in this field. In previous works, dual reinforcement learning has been proposed as an approach to reduce dependence on labeled data. However, this method has the following limitations: 1) The reward needs to be set manually and is not applicable to all kinds of logical expressions. 2) The training process easily collapses when models are trained with only the reward from dual reinforcement learning. In this paper, we propose a scoring model to automatically learn a model-based reward, and an effective training strategy based on curriculum learning is further proposed to stabilize the training process. In addition to the technical contribution, a Chinese-PL/FOL dataset is constructed to compensate for the paucity of labeled data in this field. Experimental results show that the proposed method outperforms competitors on several datasets. Furthermore, by introducing PL/FOL generated by our model, the performance of existing Natural Language Inference (NLI) models is further enhanced.