Investigating Large Language Models for Text-to-SPARQL Generation

Jacopo D’Abramo; Andrea Zugarini; Paolo Torroni

doi:10.18653/v1/2025.knowledgenlp-1.5

Investigating Large Language Models for Text-to-SPARQL Generation

Jacopo D’Abramo, Andrea Zugarini, Paolo Torroni

Correct Metadata for

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in code generation, such as translating natural language questions into SQL queries. However, state-of-the-art solutions often involve a costly fine-tuning step. In this study, we extensively evaluate In-Context Learning (ICL) solutions for text-to-SPARQL generation with different architectures and configurations, based on methods for retrieving relevant demonstrations for few-shot prompting and working with multiple generated hypotheses. In this way, we demonstrate that LLMs can formulate SPARQL queries achieving state-of-the-art results on several Knowledge Graph Question Answering (KGQA) benchmark datasets without fine-tuning.

Anthology ID:: 2025.knowledgenlp-1.5
Volume:: Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing
Month:: May
Year:: 2025
Address:: Albuquerque, New Mexico, USA
Editors:: Weijia Shi, Wenhao Yu, Akari Asai, Meng Jiang, Greg Durrett, Hannaneh Hajishirzi, Luke Zettlemoyer
Venues:: KnowledgeNLP | WS
SIG:
Publisher:: Association for Computational Linguistics
Note:
Pages:: 66–80
Language:
URL:: https://aclanthology.org/2025.knowledgenlp-1.5/
DOI:: 10.18653/v1/2025.knowledgenlp-1.5
Bibkey:
Cite (ACL):: Jacopo D’Abramo, Andrea Zugarini, and Paolo Torroni. 2025. Investigating Large Language Models for Text-to-SPARQL Generation. In Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing, pages 66–80, Albuquerque, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):: Investigating Large Language Models for Text-to-SPARQL Generation (D’Abramo et al., KnowledgeNLP 2025)
Copy Citation:
PDF:: https://aclanthology.org/2025.knowledgenlp-1.5.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{dabramo-etal-2025-investigating,
    title = "Investigating Large Language Models for Text-to-{SPARQL} Generation",
    author = "D{'}Abramo, Jacopo  and
      Zugarini, Andrea  and
      Torroni, Paolo",
    editor = "Shi, Weijia  and
      Yu, Wenhao  and
      Asai, Akari  and
      Jiang, Meng  and
      Durrett, Greg  and
      Hajishirzi, Hannaneh  and
      Zettlemoyer, Luke",
    booktitle = "Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing",
    month = may,
    year = "2025",
    address = "Albuquerque, New Mexico, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.knowledgenlp-1.5/",
    doi = "10.18653/v1/2025.knowledgenlp-1.5",
    pages = "66--80",
    ISBN = "979-8-89176-229-9",
    abstract = "Large Language Models (LLMs) have demonstrated strong capabilities in code generation, such as translating natural language questions into SQL queries. However, state-of-the-art solutions often involve a costly fine-tuning step. In this study, we extensively evaluate In-Context Learning (ICL) solutions for text-to-SPARQL generation with different architectures and configurations, based on methods for retrieving relevant demonstrations for few-shot prompting and working with multiple generated hypotheses. In this way, we demonstrate that LLMs can formulate SPARQL queries achieving state-of-the-art results on several Knowledge Graph Question Answering (KGQA) benchmark datasets without fine-tuning."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="dabramo-etal-2025-investigating">
    <titleInfo>
        <title>Investigating Large Language Models for Text-to-SPARQL Generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jacopo</namePart>
        <namePart type="family">D’Abramo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Andrea</namePart>
        <namePart type="family">Zugarini</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Paolo</namePart>
        <namePart type="family">Torroni</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Weijia</namePart>
            <namePart type="family">Shi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Wenhao</namePart>
            <namePart type="family">Yu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Akari</namePart>
            <namePart type="family">Asai</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Meng</namePart>
            <namePart type="family">Jiang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Greg</namePart>
            <namePart type="family">Durrett</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hannaneh</namePart>
            <namePart type="family">Hajishirzi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Luke</namePart>
            <namePart type="family">Zettlemoyer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Albuquerque, New Mexico, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-229-9</identifier>
    </relatedItem>
    <abstract>Large Language Models (LLMs) have demonstrated strong capabilities in code generation, such as translating natural language questions into SQL queries. However, state-of-the-art solutions often involve a costly fine-tuning step. In this study, we extensively evaluate In-Context Learning (ICL) solutions for text-to-SPARQL generation with different architectures and configurations, based on methods for retrieving relevant demonstrations for few-shot prompting and working with multiple generated hypotheses. In this way, we demonstrate that LLMs can formulate SPARQL queries achieving state-of-the-art results on several Knowledge Graph Question Answering (KGQA) benchmark datasets without fine-tuning.</abstract>
    <identifier type="citekey">dabramo-etal-2025-investigating</identifier>
    <identifier type="doi">10.18653/v1/2025.knowledgenlp-1.5</identifier>
    <location>
        <url>https://aclanthology.org/2025.knowledgenlp-1.5/</url>
    </location>
    <part>
        <date>2025-05</date>
        <extent unit="page">
            <start>66</start>
            <end>80</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T Investigating Large Language Models for Text-to-SPARQL Generation
%A D’Abramo, Jacopo
%A Zugarini, Andrea
%A Torroni, Paolo
%Y Shi, Weijia
%Y Yu, Wenhao
%Y Asai, Akari
%Y Jiang, Meng
%Y Durrett, Greg
%Y Hajishirzi, Hannaneh
%Y Zettlemoyer, Luke
%S Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing
%D 2025
%8 May
%I Association for Computational Linguistics
%C Albuquerque, New Mexico, USA
%@ 979-8-89176-229-9
%F dabramo-etal-2025-investigating
%X Large Language Models (LLMs) have demonstrated strong capabilities in code generation, such as translating natural language questions into SQL queries. However, state-of-the-art solutions often involve a costly fine-tuning step. In this study, we extensively evaluate In-Context Learning (ICL) solutions for text-to-SPARQL generation with different architectures and configurations, based on methods for retrieving relevant demonstrations for few-shot prompting and working with multiple generated hypotheses. In this way, we demonstrate that LLMs can formulate SPARQL queries achieving state-of-the-art results on several Knowledge Graph Question Answering (KGQA) benchmark datasets without fine-tuning.
%R 10.18653/v1/2025.knowledgenlp-1.5
%U https://aclanthology.org/2025.knowledgenlp-1.5/
%U https://doi.org/10.18653/v1/2025.knowledgenlp-1.5
%P 66-80

Download as File

Markdown (Informal)

[Investigating Large Language Models for Text-to-SPARQL Generation](https://aclanthology.org/2025.knowledgenlp-1.5/) (D’Abramo et al., KnowledgeNLP 2025)

Investigating Large Language Models for Text-to-SPARQL Generation (D’Abramo et al., KnowledgeNLP 2025)

ACL

Jacopo D’Abramo, Andrea Zugarini, and Paolo Torroni. 2025. Investigating Large Language Models for Text-to-SPARQL Generation. In Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing, pages 66–80, Albuquerque, New Mexico, USA. Association for Computational Linguistics.