The Proxy Presumption: From Semantic Embeddings to Valid Social Measures

Baishi Li, Ta Yu, Kelvin J.l. Koa, Ke-Wei Huang


Abstract
Natural Language Processing is rapidly evolving into a primary instrument for Computational Social Science, with researchers increasingly using embeddings to measure latent constructs such as novelty, creativity, and bias. However, this transition faces a fundamental validity challenge: the “Proxy Presumption,” or the reliance on geometric properties (e.g., cosine distance) as direct measures of social concepts. We argue that without explicit validation, unsupervised representations remain entangled mixtures of the target construct (C) and confounding attributes (Z) like topic, style, and authorship. To bridge the gap between semantic embeddings and valid social measures, we introduce the Construct Validity Protocol (CVP). Drawing on causal representation learning and psychometrics, the CVP offers a rigorous pipeline from conceptualization to quantitative verification. We further propose Counterfactual Neutralization, a novel method using LLMs to reduce confounding in embedding space. By providing a standardized Validity Suite—including tests for discriminant, incremental, and predictive validity—this work offers the community a toolkit to transform heuristic proxies into robust, scientifically defensible instruments.
Anthology ID:
2026.acl-long.1048
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
22892–22910
Language:
URL:
https://aclanthology.org/2026.acl-long.1048/
DOI:
10.18653/v1/2026.acl-long.1048
Bibkey:
Cite (ACL):
Baishi Li, Ta Yu, Kelvin J.l. Koa, and Ke-Wei Huang. 2026. The Proxy Presumption: From Semantic Embeddings to Valid Social Measures. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22892–22910, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
The Proxy Presumption: From Semantic Embeddings to Valid Social Measures (Li et al., ACL 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.acl-long.1048.pdf
Checklist:
 2026.acl-long.1048.checklist.pdf