Ouroboros tests input-conditioned LoRA in recursive transformers
Published on April 25, 2026, the work explores a compact controller that modulates weights at each step in a shared transformer block. On Qwen2.5-3B, the authors report lower training loss than a 17-layer baseline, with gains observed in-distribution.
Ouroboros connects a compact controller to a shared transformer block. Using the hidden state, the controller generates step-specific weight modulations, exploring successive transformations without describing a separate block for every step. The work was published on April 25, 2026.
In tests on Qwen2.5-3B, the authors report lower training loss than a 17-layer baseline and observe that the gains are in-distribution. This result does not establish performance outside that distribution. To check the scope, method, and results, consult the original work and verify the comparison and evaluation conditions reported by its authors.