A preliminary analysis reports that the attention matrix K exponent α tracks memorization of random labels during training, with variation across layers.
Published on September 24, 2026, the report describes a preliminary analysis using WeightWatcher to monitor weight spectra while training LLMs on random labels. The attention matrix K exponent α tracked memorization across different seeds, but the effect varied by layer.
The results suggest a possible spectral signal for observing a shift from generalization to memorization; they do not establish a definitive indicator. To check the method, scope, and conclusions, consult the original publication and verify how the experiments and measurements were defined. If using AI to study or apply the material, avoid entering sensitive organizational data before applying your organization’s data protection policies.