Patrícia Pandeiro

Author directory

2026

This work benchmarks safety across several large language models (LLMs) and compares their performances through red teaming, which simulates adversarial attacks and identifies vulnerabilities in the systems. Using two public datasets and a proprietary dataset, the models were tested with three purposes. First, a red teaming test was conducted to establish a safety comparison between five models in English and Portuguese. The results revealed that, in general, Sugarloaf 3.1 is the safest model, but that Vesuvius 4.0 slightly outperforms it in Portuguese, also revealing that both outperform GPT-4o. Afterwards, three models were tested with one guardrailing prompt, that encourages safe interactions, and two content moderation prompts, in both languages, to understand the strengths of the current guardrails, as well as the effectiveness of the content moderation task. The results show that current guardrails are sufficient, notwithstanding room for improvement (particularly for Portuguese), but that the performance of the content moderation task was substandard, even for the best performing model – GPT-4o. Finally, the 3.0 TowerLLM models were tested in English to evaluate the effect that tokens and temperature have on the output, revealing that an intermediate token limit leads to safer responses while a higher temperature causes performance degradation.