Philip Robert Corlett
Author directory2026
Inside self-reflection: the geometry in multi-dimensional scoring of metacognitive structure
Praveen Suthaharan | Philip Robert Corlett | Yuen-Siang Ang
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
Praveen Suthaharan | Philip Robert Corlett | Yuen-Siang Ang
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
Large language models (LLMs) increasingly score open-ended text along multidimensional rubrics, yet strong score recoverability need not imply that a dimension is distinct or relevant to downstream decisions. We examine how Gemma-2-2B represents five metacognitive-prompting dimensions in self-reflections from 486 participants on a probabilistic reversal learning task. Linear probes recover all dimensions across depth, with depth gradients ordered by the rubric hierarchy. Evaluation is strongly decoded but has the weakest coupling to independently measured task markers. Under the primary five-occurrence token filter, Evaluation-related vocabulary shows no detectable association with switching, whereas the negative Final Decision association is stable across frequency thresholds. Activation patching yields oppositely signed point estimates in a fixed linear win–switch readout when the Final Decision–Evaluation matched-pair contrast is injected in each direction, although no layer survives correction across the full sweep. These findings show that decodability alone is insufficient evidence for weighting rubric dimensions in downstream decisions.