Transformer circuits are often evaluated by ablating certain circuits or components and measuring the resulting performance drop. These interventions test circuit necessity only under a fixed model configuration, and it is unclear if the residual network can recover performance through alternative, parameter-efficient adaptations. Using BERT as a frozen encoder and a linear part-of-speech (POS) readout as a task classifier, we first use gate masking to learn a sparse core subnetwork sufficient for POS classification. Ablating this core produced a damaged model and a sharp drop in accuracy (95.0\\% \\rightarrow 16.8\\%)$. We then evaluated recovery from damage under four retraining pathways: readout-only, mask-only, joint readout+mask, and encoder-only with the ablation held fixed. Readout-only, mask-only, joint mask-and-readout, and encoder-only retraining arrived at 80.0\%, 69.7\%, 85.9\%, and 92.3\% accuracy, respectively. Some recovery pathways partially restored core-like geometry, but others recovered high performance without restoring the original encoder geometry. These results show that in a joint encoder--readout context, a sparse subnetwork can be sufficient and necessary for performance in the original fixed configuration, but need not remain necessary after recovery, and that recovered performance does not rely on recovering the original geometry.

Interpretability methods in NLP aim to provide insights into the semantics underlying specific system architectures. Focusing on word embeddings, we present a supervised-learning method that, for a given domain (e.g., sports, professions), identifies a subset of model features that strongly improve prediction of human similarity judgments. We show this method keeps only 20-40% of the original embeddings, for 8 independent semantic domains, and that it retains different feature sets across domains. We then present two approaches for interpreting the semantics of the retained features. The first obtains the scores of the domain words (co-hyponyms) on the first principal component of the retained embeddings, and extracts terms whose co-occurrence with the co-hyponyms tracks these scores’ profile. This analysis reveals that humans differentiate e.g. sports based on how gender-inclusive and international they are. The second approach uses the retained sets as variables in a probing task that predicts values along 65 semantically annotated dimensions for a dataset of 535 words. The features retained for professions are best at predicting cognitive, emotional and social dimensions, whereas features retained for fruits or vegetables best predict the gustation (taste) dimension. We discuss implications for alignment between AI systems and human knowledge.