Huifang Du
Author directory2026
CriticGen: Generation-Aware Evaluation as Actionable Feedback
Huifang Du | Zecheng Zuo | Sen Wang | Haofen Wang | Yehui Yang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Huifang Du | Zecheng Zuo | Sen Wang | Haofen Wang | Yehui Yang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Current evaluation methods for large language models are coarse-grained and decoupled from generation, producing generic explanations that fail to provide actionable feedback for model improvement. We propose CriticGen, a fine-grained, generation-aware evaluation framework that turns evaluation into actionable control for answer improvement. CriticGen first generates sample-specific evaluation dimensions and scoring criteria, then uses the induced rubric to jointly produce a score, a reason, an executable refinement suggestion, and a refined answer. Experiments show that fine-grained evaluation should be both instance-specific and actionable. CriticGen improves human-rated rubric relevance/coverage from 3.33/4.03 to 3.97/4.24. On the score-consensus-filtered RefineData test set, the best CriticGen variant reaches 0.9556/0.9560 Pearson/Spearman agreement with the reference scores and improves reason/suggestion F1 to 0.7554/0.7900. Independent validation provides additional support: on 180 human-scored instances, CriticGen-Qwen2.5-7B obtains higher observed score correlations than GPT-5, with 0.8868/0.8969 Pearson/Spearman versus 0.8542/0.8615. Crucially, its feedback translates into substantial answer improvement, raising accuracy from 45.36% to 91.26% on 5,000 examples with reliable gold answers, with 47.30% of answers corrected from Wrong→Right and only 1.40% degraded from Right→Wrong.
2025
LLM-Driven Estimation of Personal Carbon Footprint from Dialogues
Shuqin Li | Huifang Du | Haofen Wang
Proceedings of the 2nd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2025)
Shuqin Li | Huifang Du | Haofen Wang
Proceedings of the 2nd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2025)
Personal Carbon Footprint (PCF) Estimation is crucial for raising individual environmental awareness by linking daily activities to their environmental impact. However, existing tools are limited by fragmented scenarios and labor-intensive manual data entry. We present PCCT, an LLM-powered system that combines conversational understanding with emission knowledge grounding for PCF Estimation. We address two key challenges: (1) resolving incomplete activity information across turns through knowledge-guided and context-aware tracking, and (2) accurately mapping emission factors using multi-step LLM inference and vector-based similarity search. The system dynamically combines knowledge-guided activity extraction, and context-aware memory management, generating accurate carbon footprint estimates. We validate the effectiveness with the CarbonDialog-1K benchmark, comprising 1,028 annotated user activity narratives. Experimental results demonstrate that our method outperforms baseline systems in accuracy, while subjective evaluations show superior appropriateness, usability, efficiency, and naturalness.
2024
Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialogue
Huifang Du | Shuqin Li | Minghao Wu | Xuejing Feng | Yuan-Fang Li | Haofen Wang
Findings of the Association for Computational Linguistics: EMNLP 2024
Huifang Du | Shuqin Li | Minghao Wu | Xuejing Feng | Yuan-Fang Li | Haofen Wang
Findings of the Association for Computational Linguistics: EMNLP 2024
Reinforcement learning (RL) is a powerful approach to enhance task-oriented dialogue (TOD) systems. However, existing RL methods tend to mainly focus on generation tasks, such as dialogue policy learning (DPL) or response generation (RG), while neglecting dialogue state tracking (DST) for understanding. This narrow focus limits the systems to achieve globally optimal performance by overlooking the interdependence between understanding and generation. Additionally, RL methods face challenges with sparse and delayed rewards, which complicates training and optimization. To address these issues, we extend RL into both understanding and generation tasks by introducing step-by-step rewards throughout the token generation. The understanding reward increases as more slots are correctly filled in DST, while the generation reward grows with the accurate inclusion of user requests. Our approach provides a balanced optimization aligned with task completion. Experimental results demonstrate that our approach effectively enhances the performance of TOD systems and achieves new state-of-the-art results on three widely used datasets, including MultiWOZ2.0, MultiWOZ2.1, and In-Car. Our approach also shows superior few-shot ability in low-resource settings compared to current models.
Search