<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="furukawa-etal-2026-llm">
    <titleInfo>
        <title>LLM-Generated Social-Desirability Ratings for Forced-Choice Questionnaire Construction: A Preregistered Downstream Validation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yui</namePart>
        <namePart type="family">Furukawa</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kyosuke</namePart>
        <namePart type="family">Bunji</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kensuke</namePart>
        <namePart type="family">Okada</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ruifeng</namePart>
            <namePart type="family">Xu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ayu</namePart>
            <namePart type="family">Purwarianti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tong</namePart>
            <namePart type="family">Rong</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jun</namePart>
            <namePart type="family">Suzuki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hengqin, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-439-2</identifier>
    </relatedItem>
    <abstract>Large language models (LLMs) increasingly produce evaluative judgments, but rating-level human–model agreement does not establish whether they can replace human inputs in a downstream task. Using HEXACO personality items as a case study, we conduct a preregistered three-stage validation of whether LLM-generated social-desirability ratings can serve as inputs to forced-choice scale construction, a psychometric task that requires costly human ratings. In Stage I, three LLMs served as raters of item-level social desirability for sentence- and adjective-format items, with human ratings serving as references. In Stage II, human- and LLM-based ratings were entered into the same constrained optimization procedure. In Stage III, 400 participants completed the resulting forms, and latent trait estimates were compared using graded Thurstonian item response theory. LLM ratings showed strong item-level correspondence with human ratings (sentence: r = 0.942–0.963; adjective: r = 0.979–0.980), but optimized forms showed partial item overlap and sparse exact pair overlap. Despite these design differences, LLM-based forms produced latent trait estimates that closely matched those from human-based forms (mean r = 0.815 for sentence items and 0.768 for adjective items). These results support the conditional use of LLM-generated ratings as scalable construction inputs while showing why downstream validation is necessary.</abstract>
    <identifier type="citekey">furukawa-etal-2026-llm</identifier>
    <location>
        <url>https://aclanthology.org/2026.aacl-long.179/</url>
    </location>
    <part>
        <date>2026-11</date>
        <extent unit="page">
            <start>3991</start>
            <end>4020</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T LLM-Generated Social-Desirability Ratings for Forced-Choice Questionnaire Construction: A Preregistered Downstream Validation
%A Furukawa, Yui
%A Bunji, Kyosuke
%A Okada, Kensuke
%Y Zhang, Yue
%Y Xu, Ruifeng
%Y Purwarianti, Ayu
%Y Rong, Tong
%Y Suzuki, Jun
%S Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2026
%8 November
%I Association for Computational Linguistics
%C Hengqin, China
%@ 979-8-89176-439-2
%F furukawa-etal-2026-llm
%X Large language models (LLMs) increasingly produce evaluative judgments, but rating-level human–model agreement does not establish whether they can replace human inputs in a downstream task. Using HEXACO personality items as a case study, we conduct a preregistered three-stage validation of whether LLM-generated social-desirability ratings can serve as inputs to forced-choice scale construction, a psychometric task that requires costly human ratings. In Stage I, three LLMs served as raters of item-level social desirability for sentence- and adjective-format items, with human ratings serving as references. In Stage II, human- and LLM-based ratings were entered into the same constrained optimization procedure. In Stage III, 400 participants completed the resulting forms, and latent trait estimates were compared using graded Thurstonian item response theory. LLM ratings showed strong item-level correspondence with human ratings (sentence: r = 0.942–0.963; adjective: r = 0.979–0.980), but optimized forms showed partial item overlap and sparse exact pair overlap. Despite these design differences, LLM-based forms produced latent trait estimates that closely matched those from human-based forms (mean r = 0.815 for sentence items and 0.768 for adjective items). These results support the conditional use of LLM-generated ratings as scalable construction inputs while showing why downstream validation is necessary.
%U https://aclanthology.org/2026.aacl-long.179/
%P 3991-4020

Markdown (Informal)

[LLM-Generated Social-Desirability Ratings for Forced-Choice Questionnaire Construction: A Preregistered Downstream Validation](https://aclanthology.org/2026.aacl-long.179/) (Furukawa et al., AACL-IJCNLP 2026)

• LLM-Generated Social-Desirability Ratings for Forced-Choice Questionnaire Construction: A Preregistered Downstream Validation (Furukawa et al., AACL-IJCNLP 2026)

ACL

• Yui Furukawa, Kyosuke Bunji, and Kensuke Okada. 2026. LLM-Generated Social-Desirability Ratings for Forced-Choice Questionnaire Construction: A Preregistered Downstream Validation. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3991–4020, Hengqin, China. Association for Computational Linguistics.