GENIE: A Fine-Grained Measure for Novelty

Ramya Namuduri; Manya Wadhwa; Anshun Asher Zheng; Greg Durrett; Junyi Jessy Li

GENIE: A Fine-Grained Measure for Novelty

Ramya Namuduri, Manya Wadhwa, Anshun Asher Zheng, Greg Durrett, Junyi Jessy Li

Correct Metadata for

Use this form to create a GitHub issue with structured data describing the correction. You will need a GitHub account. Once you create that issue, the correction will be reviewed by a staff member.

⚠️ Mobile Users: Submitting this form to create a new issue will only work with github.com, not the GitHub Mobile app.

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>. You may use ... for bold, ... for italic, ... for underline, <sc>...</sc> for small-caps, <tt>...<tt> for typewriter text, <url>...</url> for URLs, <a href=...> for hyperlinks, and <par/> for paragraph breaks.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

Large Language Models have consistently demonstrated a lack of creativity and diversity across tasks. Prior work has focused on addressing whether models are capable of generating creative outputs. Here, we aim to consider novelty and investigate what makes model-generated content novel or not novel in a task-specific manner. We propose a fine-grained evaluation metric GENIE to measure the novelty of responses along task-specific features with respect to a population of responses. We show that unlike GENIE, holistic metrics struggle to capture the high-dimensionality of novelty and do not provide insight on which properties they target. Finally, we use GENIE to measure the effectiveness of mitigation methods that address creativity to better understand where these methods can improve novelty.

Anthology ID:: 2026.emnlp-main.1569
Volume:: Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:: October
Year:: 2026
Address:: Budapest, Hungary
Editors:: Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:: EMNLP
SIG:: SIGDAT
Publisher:: Association for Computational Linguistics
Note:
Pages:: 35742–35765
Language:
URL:: https://aclanthology.org/2026.emnlp-main.1569/
DOI:
Bibkey:
Cite (ACL):: Ramya Namuduri, Manya Wadhwa, Anshun Asher Zheng, Greg Durrett, and Junyi Jessy Li. 2026. GENIE: A Fine-Grained Measure for Novelty. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 35742–35765, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):: GENIE: A Fine-Grained Measure for Novelty (Namuduri et al., EMNLP 2026)
Copy Citation:
PDF:: https://aclanthology.org/2026.emnlp-main.1569.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{namuduri-etal-2026-genie,
    title = "{GENIE}: A Fine-Grained Measure for Novelty",
    author = "Namuduri, Ramya  and
      Wadhwa, Manya  and
      Zheng, Anshun Asher  and
      Durrett, Greg  and
      Li, Junyi Jessy",
    editor = "Lee, Hung-Yi  and
      Dev, Sunipa  and
      Li, Jessy  and
      Elliott, Desmond",
    booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.emnlp-main.1569/",
    pages = "35742--35765",
    ISBN = "979-8-89176-459-0",
    abstract = "Large Language Models have consistently demonstrated a lack of creativity and diversity across tasks. Prior work has focused on addressing $\textit{whether}$ models are capable of generating creative outputs. Here, we aim to consider $\textit{novelty}$ and investigate what makes model-generated content novel or not novel in a task-specific manner. We propose a fine-grained evaluation metric $\textbf{GENIE}$ to measure the novelty of responses along task-specific $\textit{features}$ with respect to a population of responses. We show that unlike $\textbf{GENIE}$, holistic metrics struggle to capture the high-dimensionality of novelty and do not provide insight on which properties they target. Finally, we use $\textbf{GENIE}$ to measure the effectiveness of mitigation methods that address creativity to better understand where these methods can improve novelty."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="namuduri-etal-2026-genie">
    <titleInfo>
        <title>GENIE: A Fine-Grained Measure for Novelty</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ramya</namePart>
        <namePart type="family">Namuduri</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Manya</namePart>
        <namePart type="family">Wadhwa</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anshun</namePart>
        <namePart type="given">Asher</namePart>
        <namePart type="family">Zheng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Greg</namePart>
        <namePart type="family">Durrett</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Junyi</namePart>
        <namePart type="given">Jessy</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Large Language Models have consistently demonstrated a lack of creativity and diversity across tasks. Prior work has focused on addressing whether models are capable of generating creative outputs. Here, we aim to consider novelty and investigate what makes model-generated content novel or not novel in a task-specific manner. We propose a fine-grained evaluation metric GENIE to measure the novelty of responses along task-specific features with respect to a population of responses. We show that unlike GENIE, holistic metrics struggle to capture the high-dimensionality of novelty and do not provide insight on which properties they target. Finally, we use GENIE to measure the effectiveness of mitigation methods that address creativity to better understand where these methods can improve novelty.</abstract>
    <identifier type="citekey">namuduri-etal-2026-genie</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1569/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>35742</start>
            <end>35765</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T GENIE: A Fine-Grained Measure for Novelty
%A Namuduri, Ramya
%A Wadhwa, Manya
%A Zheng, Anshun Asher
%A Durrett, Greg
%A Li, Junyi Jessy
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F namuduri-etal-2026-genie
%X Large Language Models have consistently demonstrated a lack of creativity and diversity across tasks. Prior work has focused on addressing whether models are capable of generating creative outputs. Here, we aim to consider novelty and investigate what makes model-generated content novel or not novel in a task-specific manner. We propose a fine-grained evaluation metric GENIE to measure the novelty of responses along task-specific features with respect to a population of responses. We show that unlike GENIE, holistic metrics struggle to capture the high-dimensionality of novelty and do not provide insight on which properties they target. Finally, we use GENIE to measure the effectiveness of mitigation methods that address creativity to better understand where these methods can improve novelty.
%U https://aclanthology.org/2026.emnlp-main.1569/
%P 35742-35765

Download as File

Markdown (Informal)

[GENIE: A Fine-Grained Measure for Novelty](https://aclanthology.org/2026.emnlp-main.1569/) (Namuduri et al., EMNLP 2026)

GENIE: A Fine-Grained Measure for Novelty (Namuduri et al., EMNLP 2026)

ACL

Ramya Namuduri, Manya Wadhwa, Anshun Asher Zheng, Greg Durrett, and Junyi Jessy Li. 2026. GENIE: A Fine-Grained Measure for Novelty. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 35742–35765, Budapest, Hungary. Association for Computational Linguistics.