<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="touileb-etal-2026-tryggllm">
    <titleInfo>
        <title>TryggLLM: A Benchmark for Evaluating LLM Safety in Norwegian</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Samia</namePart>
        <namePart type="family">Touileb</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Truls</namePart>
        <namePart type="family">Pedersen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Isabell</namePart>
        <namePart type="given">Stinessen</namePart>
        <namePart type="family">Haugen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Fifteenth Language Resources and Evaluation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Stelios</namePart>
            <namePart type="family">Piperidis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Núria</namePart>
            <namePart type="family">Bel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Henk</namePart>
            <namePart type="family">van den Heuvel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nancy</namePart>
            <namePart type="family">Ide</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Krek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Antonio</namePart>
            <namePart type="family">Toral</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resource Association</publisher>
            <place>
                <placeTerm type="text">Palma de Mallorca, Spain</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We introduce TryggLLM, the first safety benchmark dataset for Norwegian. The dataset is intended for benchmarking different types of safety issues that can occur when using Norwegian generative language models. We have manually translated two English benchmark datasets, while modifying the content to be aligned with the Norwegian context. The benchmark dataset is composed of two sub-parts: i) prompts annotated by four native speakers, in both the written variants of Norwegian Bokmål (BM) and Nynorsk (NN), such that each native speaker wrote in their preferred variants (two BM and two NN); ii) prompts and target responses, where each of them has a BM and a NN version. We provide detailed descriptions of the data creation process. We also present a thorough manual evaluation of benchmarking existing open Norwegian LLMs using TryggLLM. Our results show that between 18% and 48% of the generated responses are unsafe, across all tested models.</abstract>
    <identifier type="citekey">touileb-etal-2026-tryggllm</identifier>
    <identifier type="doi">10.63317/2rhfg2a92wim</identifier>
    <location>
        <url>https://aclanthology.org/2026.lrec-1.323/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>4093</start>
            <end>4102</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T TryggLLM: A Benchmark for Evaluating LLM Safety in Norwegian
%A Touileb, Samia
%A Pedersen, Truls
%A Haugen, Isabell Stinessen
%Y Piperidis, Stelios
%Y Bel, Núria
%Y van den Heuvel, Henk
%Y Ide, Nancy
%Y Krek, Simon
%Y Toral, Antonio
%S Proceedings of the Fifteenth Language Resources and Evaluation Conference
%D 2026
%8 May
%I ELRA Language Resource Association
%C Palma de Mallorca, Spain
%F touileb-etal-2026-tryggllm
%X We introduce TryggLLM, the first safety benchmark dataset for Norwegian. The dataset is intended for benchmarking different types of safety issues that can occur when using Norwegian generative language models. We have manually translated two English benchmark datasets, while modifying the content to be aligned with the Norwegian context. The benchmark dataset is composed of two sub-parts: i) prompts annotated by four native speakers, in both the written variants of Norwegian Bokmål (BM) and Nynorsk (NN), such that each native speaker wrote in their preferred variants (two BM and two NN); ii) prompts and target responses, where each of them has a BM and a NN version. We provide detailed descriptions of the data creation process. We also present a thorough manual evaluation of benchmarking existing open Norwegian LLMs using TryggLLM. Our results show that between 18% and 48% of the generated responses are unsafe, across all tested models.
%R 10.63317/2rhfg2a92wim
%U https://aclanthology.org/2026.lrec-1.323/
%U https://doi.org/10.63317/2rhfg2a92wim
%P 4093-4102

Markdown (Informal)

[TryggLLM: A Benchmark for Evaluating LLM Safety in Norwegian](https://aclanthology.org/2026.lrec-1.323/) (Touileb et al., LREC 2026)

• TryggLLM: A Benchmark for Evaluating LLM Safety in Norwegian (Touileb et al., LREC 2026)

ACL

• Samia Touileb, Truls Pedersen, and Isabell Stinessen Haugen. 2026. TryggLLM: A Benchmark for Evaluating LLM Safety in Norwegian. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 4093–4102, Palma de Mallorca, Spain. ELRA Language Resource Association.