SCARV: Structure-Constrained Aggregation for Stable Sample Ranking in Redundant NLP Datasets

Feiyu Wu, Xu Zheng, Zhuocheng Wang, YIMing Dai, Hui LI


Abstract
Sample-level ranking is widely used for data selection, debugging, and training-data attribution, but redundant NLP corpora—exact and near duplicates, paraphrases, and templated examples—can make proxy-induced rankings unstable across random seeds. We study stable sample-level ranking under redundancy and propose SCARV, a modular aggregation layer that operates on top of existing scoring proxies. SCARV normalizes scores within each run, can regularize redundant examples through cluster-level aggregation and allocation, and aggregates across seeds with a robust operator. Across synthetic redundancy, naturally mined QQP redundancy, multiple proxy families, classification and sentence-pair tasks, and end-to-end DistilBERT fine-tuning, stability-aware aggregation improves over bare proxy rankings and yields more reproducible ranking-based decisions, such as selected subsets and suspicious-example sets. Matched mechanism analysis shows that cross-seed aggregation is the dominant generic stabilizer. The incremental effect of structural regularization is small, map-dependent, dataset-dependent, and sometimes negative; a meaningful redundancy map is useful mainly when group-consistent treatment is operationally desired. These results position SCARV as a stability-oriented aggregation framework rather than a universal data selector and emphasize decision reproducibility over uniform gains in downstream utility.
Anthology ID:
2026.findings-emnlp.525
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
11013–11035
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.525/
DOI:
Bibkey:
Cite (ACL):
Feiyu Wu, Xu Zheng, Zhuocheng Wang, YIMing Dai, and Hui LI. 2026. SCARV: Structure-Constrained Aggregation for Stable Sample Ranking in Redundant NLP Datasets. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 11013–11035, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
SCARV: Structure-Constrained Aggregation for Stable Sample Ranking in Redundant NLP Datasets (Wu et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.525.pdf