<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lim-etal-2026-psychometric">
    <titleInfo>
        <title>Psychometric Item Validation Using Virtual Respondents with Trait-Response Mediators</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sungjib</namePart>
        <namePart type="family">Lim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Woojung</namePart>
        <namePart type="family">Song</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Eun-Ju</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yohan</namePart>
        <namePart type="family">Jo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>As psychometric surveys are increasingly used to assess the traits of large language models (LLMs), the need for scalable survey item generation suited for LLMs has also grown. A critical challenge here is ensuring the construct validity of generated items, i.e., whether they truly measure the intended trait. Traditionally, this requires costly, large-scale human data collection. To make it efficient, we present a framework for virtual respondent simulation using LLMs. Our central idea is to account for mediators: factors through which the same trait can give rise to varying responses to a survey item. By simulating respondents with diverse mediators, we identify survey items that yield responses robustly correlated with intended traits across these mediators. Experiments on three psychological trait theories (Big5, Schwartz, VIA) show that our mediator generation methods and simulation framework effectively identify high-validity items. LLMs demonstrate the ability to generate plausible mediators from trait definitions and to simulate respondent behavior for item validation. Our problem formulation, metrics, methodology, and dataset open a new direction for cost-efficient survey development and a deeper understanding of how LLMs simulate human survey responses. We release our dataset and code to support future work.1</abstract>
    <identifier type="citekey">lim-etal-2026-psychometric</identifier>
    <identifier type="doi">10.1162/tacl.a.733</identifier>
    <location>
        <url>https://aclanthology.org/2026.tacl-1.63/</url>
    </location>
    <part>
        <date>2026</date>
        <detail type="volume"><number>14</number></detail>
        <extent unit="page">
            <start>1390</start>
            <end>1412</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Psychometric Item Validation Using Virtual Respondents with Trait-Response Mediators
%A Lim, Sungjib
%A Song, Woojung
%A Lee, Eun-Ju
%A Jo, Yohan
%J Transactions of the Association for Computational Linguistics
%D 2026
%V 14
%I MIT Press
%C Cambridge, MA
%F lim-etal-2026-psychometric
%X As psychometric surveys are increasingly used to assess the traits of large language models (LLMs), the need for scalable survey item generation suited for LLMs has also grown. A critical challenge here is ensuring the construct validity of generated items, i.e., whether they truly measure the intended trait. Traditionally, this requires costly, large-scale human data collection. To make it efficient, we present a framework for virtual respondent simulation using LLMs. Our central idea is to account for mediators: factors through which the same trait can give rise to varying responses to a survey item. By simulating respondents with diverse mediators, we identify survey items that yield responses robustly correlated with intended traits across these mediators. Experiments on three psychological trait theories (Big5, Schwartz, VIA) show that our mediator generation methods and simulation framework effectively identify high-validity items. LLMs demonstrate the ability to generate plausible mediators from trait definitions and to simulate respondent behavior for item validation. Our problem formulation, metrics, methodology, and dataset open a new direction for cost-efficient survey development and a deeper understanding of how LLMs simulate human survey responses. We release our dataset and code to support future work.1
%R 10.1162/tacl.a.733
%U https://aclanthology.org/2026.tacl-1.63/
%U https://doi.org/10.1162/tacl.a.733
%P 1390-1412

Markdown (Informal)

[Psychometric Item Validation Using Virtual Respondents with Trait-Response Mediators](https://aclanthology.org/2026.tacl-1.63/) (Lim et al., TACL 2026)

• Psychometric Item Validation Using Virtual Respondents with Trait-Response Mediators (Lim et al., TACL 2026)

ACL

• Sungjib Lim, Woojung Song, Eun-Ju Lee, and Yohan Jo. 2026. Psychometric Item Validation Using Virtual Respondents with Trait-Response Mediators. Transactions of the Association for Computational Linguistics, 14:1390–1412.