Current genomic foundation models (GFMs) rely largely on extensive neural computation to implicitly infer conserved biological motifs from single-nucleotide inputs. We introduce Gengram, a conditional memory module that provides an explicit and efficient lookup primitive for multi-base motifs through a genome-specific hashing scheme. Integrated into state-of-the-art (SOTA) GFM backbones, Gengram achieves substantial improvements of up to 22.6% across functional genomics tasks and generalizes across diverse architectures. Further analysis of Gengram’s latent space reveals biologically meaningful representations aligned with established genomic principles. By introducing structured motif memory as a modeling primitive, Gengram improves empirical performance and mechanistic interpretability, offering a scalable and biology-aligned direction for next-generation GFMs. The code is available at https://github.com/zhejianglab/Gengram, and the model weights are available at https://huggingface.co/ZhejiangLab/Gengram.

Recent advances in large language models (LLMs) have yielded impressive gains on mathematical reasoning benchmarks via supervised fine-tuning (SFT). However, the brittleness of these models under input perturbations has cast doubt on whether such improvements reflect genuine reasoning abilities or merely superficial alignment with expected output formats. We investigate the mechanisms behind SFT improvements in small-scale LLMs, addressing four key questions: (1) Are performance gains primarily due to format alignment rather than reasoning? (2) Can high-quality supervision encourage genuine reasoning? (3) Does scaling data shift learning from format alignment to deeper reasoning? (4) Are format alignment gains consistent across model sizes and architectures? Through controlled experiments, we find that most performance improvements arise from format alignment rather than genuine reasoning enhancement. Moreover, SFT’s effectiveness is strongly influenced by the alignment between the base model’s inductive biases and the teacher model’s output distribution, rather than the teacher’s raw strength. Finally, scaling up training data offers diminishing returns and does not fundamentally alter the model’s reasoning behavior. These findings suggest that current SFT practices may overestimate the reasoning abilities of LLMs and underscore the need for more rigorous evaluation methods.