Gefei Feng

Author directory

2026

Language can produce many expressions from a limited set of units because linguistic units participate in two basic relations. Syntagmatic relations determine which units can combine, while paradigmatic relations determine which units can substitute for one another in the same position. We study whether explicitly learning these relations helps language models generalize from limited data. We introduce two auxiliary objectives for masked language modeling. The Syntagmatic objective aligns a masked representation with relevant context selected by attention, while the Paradigmatic objective aligns it with the embedding neighbors of the target token. Both objectives are generated by the model itself and require no additional data, annotation, or trainable parameters. Experiments on BabyLM 2026 Strict-Small show that both single-relation systems improve mean Overall performance over Standard MLM, with Paradigmatic giving the most consistent gains across three seeds. Relay Joint combines the two objectives and shows a strong neighborhood-alignment signal at its submitted checkpoint, although its downstream gains vary across seeds. These results suggest that relational objectives can provide useful learning signals under limited data, while stronger internal alignment does not always lead to more stable downstream performance. Code is now publicly available on GitHub.