A recuperação de informação jurídica brasileira envolve regimes heterogêneos de busca que exigem simultaneamente correspondência lexical precisa e generalização semântica. Embora a recuperação híbrida seja consolidada na recuperação de informação geral, ainda não está claro quando sinais lexicais e semânticos se complementam efetivamente em coleções jurídicas brasileiras diversas. Investigamos essa questão no benchmark JUÁ, que abrange múltiplos cenários: jurisprudência, atos normativos, recuperação legislativa e perguntas tributárias. Comparamos BM25, recuperadores densos da família Qwen3, variantes ajustadas ao domínio jurídico, reranqueadores e fusão híbrida via Reciprocal Rank Fusion (RRF). Nossa hipótese é que a fusão léxico-densa oferece maior robustez ao combinar sinais complementares sem necessidade de calibração direta de scores. Os resultados mostram que sistemas baseados em RRF alcançam a melhor efetividade agregada: o modelo jurídico ajustado de 4B lidera em NDCG@10 e MRR@10, enquanto o de 8B lidera em MAP@10 e Recall@10. Testes estatísticos pareados indicam ganhos robustos do RRF sobre BM25 e parte dos modelos densos, embora não domine todos os reranqueadores individualmente. No geral, a fusão híbrida léxico-semântica revela-se uma configuração particularmente robusta para busca jurídica heterogênea.

Modern Legal AI systems rely on retrieval to ground answers in heterogeneous sources, including precedents, normative acts, legislative proposals, and explanatory materials. We study retrieval across five Brazilian legal collections, evaluated separately and in a pooled global index, using lexical, dense, fusion, and generative reranking architectures. The union of BM25 and dense top-10 results covers 55.4–60.3% of judged positives, 2.7–3.4 percentage points more than the better individual retriever, demonstrating substantial lexical–semantic complementarity. Reciprocal Rank Fusion (RRF) exploits part of this complementarity, but a per-query selector oracle consistently outperforms it, indicating that the preferred retrieval signal varies across queries. Pooling the collections reduces NDCG@10 by 18.5–25.9% under the original source-local judgments, showing substantial degradation when the relevant source is not known in advance; exact duplicates account for only a small fraction of this measured loss. RRF remains the strongest fixed first-stage architecture, yet we find no evidence that its gains are preserved after generative reranking at the dataset-macro level, indicating that stronger first-stage retrieval does not necessarily translate into better final rankings. These findings characterize heterogeneous legal retrieval as a multi-stage problem of source selection, signal selection, and evidence promotion.

Linear probes can recover deception-related signals from large language model activations, but existing evidence is overwhelmingly English-centric. We investigate whether this discrimination transfers across languages and what representational geometry accompanies such transfer. We train instructed-deception probes on controlled honest/deceptive instruction contrasts in English, Portuguese, Spanish, German, and Chinese, and evaluate them without target-language refitting on a multilingual adaptation of DeceptionBench across five models spanning four different families. Cross-language performance remains close to matched-language baselines, with all models satisfying our prespecified noninferiority criterion, although absolute discrimination varies substantially across models. We also find that language-specific probe directions are strongly aligned and concentrate around a shared consensus direction, while response representations retain clear language-specific organization. Additional construct-validity analyses show that measurable response-level cues, including style, sentiment, factual truthfulness, and length, explain part but not all of the strongest probe discrimination. These findings together suggest that cross-lingual transfer and shared probe geometry can coexist with model-dependent detection quality and persistent language structure, indicating that representational alignment alone is insufficient evidence of reliable multilingual deception monitoring.

As large language models (LLMs) are increasingly considered for high-stakes legal applications, a central question for trust and auditability is whether their outputs reflect genuine reasoning over the supplied legal materials, or merely surface statistical patterns absorbed during pre-training. We address this question at the mechanistic level by applying activation patching, a causal-mediation technique, to four open-source multilingual LLMs on two bar-exam datasets: U.S. MBE and Brazilian OAB. For each multiple-choice question we contrast a gold supporting passage with an irrelevant-but-plausible counterfactual. We then surgically replace residual-stream activations across layers and token regions and measure recovery of the logit difference. Gold passages exert a measurable but model- and linguistic-dependent causal effect; integration concentrates in the upper layers (a late-binding signature); and the size of the effect is strongly mediated by the linguistic and discourse structure of the passage itself, not just its relevance. We introduce a four-cluster taxonomy that operationalises these regimes and identifies cases where retrieved evidence is causally decisive, inert, or actively harmful.

Modern evaluation of Legal QA systems is shifting from terminal accuracy toward process-aware analyses of model reasoning. We propose a diagnostic framework grounded in monotonic pedagogical scaffolding, where language models receive gold-standard, case-relevant information across stages aligned with the canonical legal framework FIRAC — Facts, Issue, Rules, Application, Conclusion. By strictly adding solution-relevant content at each step, we introduce a controlled monotonic intervention that allows for the evaluation of reasoning trajectories rather than isolated outcomes.This longitudinal design enables the introduction of two transition-based diagnostics: Errors-to-Success (E2S) quantifies the guidance required to reach correctness, while Success-to-Errors (S2E) measures the fragility of that correctness under additional structure. These local patterns define a global robustness criterion termed Stable Accuracy, which credits a response only if the model maintains correctness throughout all scaffolding stages and enforces a higher bar for correctness by distinguishing sustained reasoning from transient patterns.We instantiate the framework on 3,123 Brazilian Bar Exam questions paired with expert-annotated explanations. Our findings reveal model instability patterns hidden from accuracy-only metrics and demonstrate that terminal accuracy systematically overestimates legal reasoning competence. To test the robustness of our diagnostics, we also evaluate a majority-vote aggregation across multiple reasoning samples, finding that the observed instability patterns persist under this stronger inference setting. Furthermore, principal component analysis indicates that legal domains cluster into distinct regions, suggesting systematic differences in reasoning demands across domains. While focused on the legal domain, our evaluation protocol is generalizable to any task with a staged reasoning structure.

Do LLMs process text and mathematics as a unified skill, or do these components rely on distinct underlying mechanisms? We investigate this question by disentangling the textual interpretation and mathematical solving steps in word problems drawn from Brazil’s largest college entrance exam (ENEM) and GSM8K, a popular grade school-level benchmark. Using the symbolic solver SymPy, we transform word problems into equivalent purely mathematical representations, isolating equation formulation from textual comprehension. Our extended benchmarks enable a structured analysis of LLM performance across these two dimensions. Through empirical evaluations, we find that small-scale LLMs struggle significantly more with text interpretation than with equation solving, with accuracy dropping by a factor of 2 to 7 when solving full word problems compared to their math-only counterparts. Exploratory factor analysis confirms a bidimensional structure in LLM reasoning, where models exhibit distinct proficiencies in textual and mathematical components, underscoring the need for targeted improvements in language comprehension. By analyzing the latent factors associated with each model, our findings provide a framework for researchers and practitioners to make informed choices when selecting models based on computational costs and the nature of their tasks.

Large Language Models (LLMs) are increasingly used in applications that shape public discourse, yet little is known aboutwhether they reflect distinct opinions on global issues like climate change. This study compares climate change-relatedresponses from multiple LLMs with human opinions collected through the People’s Climate Vote 2024 survey (UNDP – UnitedNations Development Programme and Oxford, 2024). We compare country and LLM”s answer probability distributions and apply Exploratory Factor Analysis (EFA) to identify latent opinion dimensions. Our findings reveal that while LLM responsesdo not exhibit significant biases toward specific demographic groups, they encompass a wide range of opinions, sometimesdiverging markedly from the majority human perspective.

Material facts (MF) are crucial and obligatory disclosures that can significantly influence asset values. Following their release, financial analysts embark on the meticulous and highly specialized task of crafting analyses to shed light on their impact on company assets, a challenge elevated by the daily amount of MFs released. Generative AI, with its demonstrated power of crafting coherent text, emerges as a promising solution to this task. However, while these analyses must incorporate the MF, they must also transcend it, enhancing it with vital background information, valuable and grounded recommendations, prospects, potential risks, and their underlying reasoning. In this paper, we approach this task as an instance of controllable text generation, aiming to ensure adherence to the MF and other pivotal attributes as control elements. We first explore language models’ capacity to manage this task by embedding those elements into prompts and engaging popular chatbots. A bilingual proof of concept underscores both the potential and the challenges of applying generative AI techniques to this task.