The article introduces Token-Aware Phase Attention (TAPA), a positional encoding method with a trainable phase function, and analyzes RoPE's distance-dependent bias. It reports a comparison of performance in long contexts.
Published on June 26, 2026, the article introduces Token-Aware Phase Attention (TAPA), a positional encoding method that uses a trainable phase function. It analyzes the distance-dependent bias associated with RoPE and compares TAPA's performance with RoPE in long contexts.
The available description does not provide quantitative comparison results. Engineers researching long-context models can consult the article to verify its theoretical analysis, methods, and reported findings in the original source; superiority should not be inferred without examining those data.