Hyperion: Private Token Sampling with Homomorphic Encryption

Lawrence Lim; Jiaming Liu; Vikas Kalagi; Divyakant Agrawal; Amr El Abbadi

Hyperion: Private Token Sampling with Homomorphic Encryption

Lawrence Lim, Jiaming Liu, Vikas Kalagi, Divyakant Agrawal, Amr El Abbadi

Correct Metadata for

Use this form to create a GitHub issue with structured data describing the correction. You will need a GitHub account. Once you create that issue, the correction will be reviewed by a staff member.

⚠️ Mobile Users: Submitting this form to create a new issue will only work with github.com, not the GitHub Mobile app.

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>. You may use <b>...</b> for bold, <i>...</i> for italic, and <url>...</url> for URLs.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

A promising direction for enabling private queries to large language models (LLMs) is with homomorphic encryption (HE). An open problem is performing token sampling under HE. In this paper, we introduce Hyperion, an efficient HE algorithm for inverse transform sampling, enabling private token sampling with 1 comparison depth, O(1) amortized comparisons, and O(log n) rotations. We implement our approach and demonstrate that it samples tokens in 0.14 seconds for 32k tokens (≈ 4.4\ 𝜇 s per token) on GPU, achieving a 100× latency improvement over prior work.

Anthology ID:: 2026.acl-long.644
Volume:: Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:: July
Year:: 2026
Address:: San Diego, California, United States
Editors:: Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:: ACL
SIG:
Publisher:: Association for Computational Linguistics
Note:
Pages:: 14150–14159
Language:
URL:: https://aclanthology.org/2026.acl-long.644/
DOI:
Bibkey:
Cite (ACL):: Lawrence Lim, Jiaming Liu, Vikas Kalagi, Divyakant Agrawal, and Amr El Abbadi. 2026. Hyperion: Private Token Sampling with Homomorphic Encryption. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14150–14159, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):: Hyperion: Private Token Sampling with Homomorphic Encryption (Lim et al., ACL 2026)
Copy Citation:
PDF:: https://aclanthology.org/2026.acl-long.644.pdf
Checklist:: 2026.acl-long.644.checklist.pdf

PDF Cite Search Checklist Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{lim-etal-2026-hyperion,
    title = "Hyperion: Private Token Sampling with Homomorphic Encryption",
    author = "Lim, Lawrence  and
      Liu, Jiaming  and
      Kalagi, Vikas  and
      Agrawal, Divyakant  and
      El Abbadi, Amr",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.644/",
    pages = "14150--14159",
    ISBN = "979-8-89176-390-6",
    abstract = "A promising direction for enabling private queries to large language models (LLMs) is with homomorphic encryption (HE). An open problem is performing token sampling under HE. In this paper, we introduce Hyperion, an efficient HE algorithm for inverse transform sampling, enabling private token sampling with 1 comparison depth, $O(1)$ amortized comparisons, and $O(\log n)$ rotations. We implement our approach and demonstrate that it samples tokens in 0.14 seconds for 32k tokens ($\approx 4.4\ \mu \mathrm{s}$ per token) on GPU, achieving a $100\times$ latency improvement over prior work."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lim-etal-2026-hyperion">
    <titleInfo>
        <title>Hyperion: Private Token Sampling with Homomorphic Encryption</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lawrence</namePart>
        <namePart type="family">Lim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jiaming</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vikas</namePart>
        <namePart type="family">Kalagi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Divyakant</namePart>
        <namePart type="family">Agrawal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Amr</namePart>
        <namePart type="family">El Abbadi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maria</namePart>
            <namePart type="family">Liakata</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Viviane</namePart>
            <namePart type="given">P</namePart>
            <namePart type="family">Moreira</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jiajun</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Jurgens</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">San Diego, California, United States</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-390-6</identifier>
    </relatedItem>
    <abstract>A promising direction for enabling private queries to large language models (LLMs) is with homomorphic encryption (HE). An open problem is performing token sampling under HE. In this paper, we introduce Hyperion, an efficient HE algorithm for inverse transform sampling, enabling private token sampling with 1 comparison depth, O(1) amortized comparisons, and O(łog n) rotations. We implement our approach and demonstrate that it samples tokens in 0.14 seconds for 32k tokens (\approx 4.4 μ s per token) on GPU, achieving a 100\times latency improvement over prior work.</abstract>
    <identifier type="citekey">lim-etal-2026-hyperion</identifier>
    <location>
        <url>https://aclanthology.org/2026.acl-long.644/</url>
    </location>
    <part>
        <date>2026-07</date>
        <extent unit="page">
            <start>14150</start>
            <end>14159</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T Hyperion: Private Token Sampling with Homomorphic Encryption
%A Lim, Lawrence
%A Liu, Jiaming
%A Kalagi, Vikas
%A Agrawal, Divyakant
%A El Abbadi, Amr
%Y Liakata, Maria
%Y Moreira, Viviane P.
%Y Zhang, Jiajun
%Y Jurgens, David
%S Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2026
%8 July
%I Association for Computational Linguistics
%C San Diego, California, United States
%@ 979-8-89176-390-6
%F lim-etal-2026-hyperion
%X A promising direction for enabling private queries to large language models (LLMs) is with homomorphic encryption (HE). An open problem is performing token sampling under HE. In this paper, we introduce Hyperion, an efficient HE algorithm for inverse transform sampling, enabling private token sampling with 1 comparison depth, O(1) amortized comparisons, and O(łog n) rotations. We implement our approach and demonstrate that it samples tokens in 0.14 seconds for 32k tokens (\approx 4.4 μ s per token) on GPU, achieving a 100\times latency improvement over prior work.
%U https://aclanthology.org/2026.acl-long.644/
%P 14150-14159

Download as File

Markdown (Informal)

[Hyperion: Private Token Sampling with Homomorphic Encryption](https://aclanthology.org/2026.acl-long.644/) (Lim et al., ACL 2026)

Hyperion: Private Token Sampling with Homomorphic Encryption (Lim et al., ACL 2026)

ACL

Lawrence Lim, Jiaming Liu, Vikas Kalagi, Divyakant Agrawal, and Amr El Abbadi. 2026. Hyperion: Private Token Sampling with Homomorphic Encryption. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14150–14159, San Diego, California, United States. Association for Computational Linguistics.