Source Marking in Language Models: A Controlled Framework for Acquisition, Cue Sensitivity, and Reasoning

Daniil Huryn


Abstract
Language can affect how people think. We ask a computational version: if a language model is trained on English that marks propositions as observed, reported, or inferred, does it use evidence differently in ordinary English? We create matched corpora in which particles either mark these routes (EVID), appear in the same positions without predicting source (CTRL), or are absent (NOMARK), and remove them at evaluation.

We test 405M-parameter Pythia models trained from scratch and Qwen3.5-4B Base and Post models adapted across three paired seeds. The models learn the mapping: Pythia reaches 92.8% before ordinary-English adaptation and retains 79.8% afterward; Qwen EVID averages about 80% for both Base and Post, with controls near chance. Visible markers shift Qwen’s scores. Once markers are removed, however, the Pythia tests cannot show whether the effect carries over. Base’s main Qwen result changes direction across seeds, and Post’s is negative in all three. A separate test suggests that Base may respond more to relevant context, but not specifically to source wording. No condition consistently improves reliability, multistep inference, or recognition of copied information. Source marking is easy to learn and can influence model scores, but we find no broader improvement in reasoning.
Anthology ID:
2026.blackboxnlp-1.15
Volume:
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Dana Arad, Yonatan Belinkov, Hanjie Chen, Nils Feldhus, Aaron Mueller, Hosein Mohebbi, Gabriele Sarti, Martin Tutek
Venues:
BlackboxNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
220–231
Language:
URL:
https://aclanthology.org/2026.blackboxnlp-1.15/
DOI:
Bibkey:
Cite (ACL):
Daniil Huryn. 2026. Source Marking in Language Models: A Controlled Framework for Acquisition, Cue Sensitivity, and Reasoning. In Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 220–231, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Source Marking in Language Models: A Controlled Framework for Acquisition, Cue Sensitivity, and Reasoning (Huryn, BlackboxNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.blackboxnlp-1.15.pdf