Loc Nguyen
Author directory2026
Feedback Blindness: Diagnosing Silent Failures in Closed-Loop Agentic Data-Generation Pipelines
Loc Nguyen | Nguyen Khanh Long | Vu Gia Nam | Nguyen Tran Minh Nhat
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Loc Nguyen | Nguyen Khanh Long | Vu Gia Nam | Nguyen Tran Minh Nhat
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Closed-loop synthetic-data pipelines combine planning, LLM generation, verification, and deduplication; accepted outputs steer later generation. We study a production pipeline for Vietnamese contact-center compliance data and identify feedback blindness: an internal feedback mechanism stops serving its intended function while every stage completes normally, leaving only downstream metric degradation as evidence. We diagnose four instances: cumulative-count eviction empties a recency window; batch-level deduplication misses candidates scored before any batch member is committed; static boundary exemplars repeatedly anchor generation; and a fixed-size anti-repeat reference set loses informativeness as the dataset grows. We describe architecture-level fixes, mechanism-specific health signals, and a controlled ablation and scaling evaluation. The central lesson is that aggregate duplicate and acceptance rates are insufficient for diagnosis; pipelines should also monitor the feedback mechanisms that produce those outcomes.