Memory Attention proposes an alternative to value projections

Published on September 24, 2026, the paper examines Memory Attention (MA) across several attention variants. The proposal replaces the Transformer's learned value projection with a sum of contextual keys and token-indexed, layer-specific memory vectors.

The summary describes an alternative way to represent and retrieve information in attention layers, but provides no comparative results or metrics. To check the scope of the claims, consult the original paper and verify which variants were evaluated and what evidence supports each statement.