KITE expands Transformers with invariant KV computation
A September 29, 2026 entry describes KITE, a scaling paradigm combining scaling with upcycling and expanding decoder components that reuse a KV cache generated by a smaller prefiller.
On September 29, 2026, Radar recorded KITE as a scaling paradigm that combines scaling with upcycling while keeping KV computation invariant. The approach expands decoder components that reuse a KV cache generated by a smaller prefiller.
Its stated aim is to enable evaluation of model-quality scaling without a proportional increase in prefill cost. The entry provides no measurements, experimental conditions, or quantitative results. To verify the claim’s scope, consult the original publication and check whether further materials specify methods and evidence. If using AI to study it, apply your organization’s policy to personal data before processing.