Knowledge you can apply
Daily bookmarks: relevant news, facts and discoveries.
The publication claims cuDNN, FA4 and FlexAttention have precision problems in backward passes and presents KohakuFA as an alternative kernel, with a public GitHub repository for inspection.
Read articleRadar briefing dated 3 October 2026 on a text-to-image model that reuses Transformer blocks at each denoising step, with the performance claim stated by the original post.
Read articleRadar: the paper presents LoopCD, a training-free contrastive decoding method for recurrent Transformers that uses representations from earlier loops as weaker predictions to select tokens.
Read articleRadar: LoopCD refines decoding in looped Transformers by using the gap between early and late predictions of recurrent states, with no extra training, and reports accuracy gains on AIME 2024 with the Ouro-2.6B-Thinking model.
Read articleA post describes adding sampling to the post-training stack and claims SFT can compete with current methods, with better generalization and less forgetting in some cases.
Read articleA guide describes a proxy that records token IDs and sampled logprobs to train models with RL across four agent harnesses, with reported score gains and a comparison with imitation learning.
Read articleThe article examines whether chains of thought do computational work in serial tasks; according to the post, removing them lowers accuracy to the shallow-circuit limit.
Read articlePublished on October 1, 2026, the text presents multi-teacher on-policy distillation (MOPD): a method that trains a student model on its own completions, using teachers’ per-token log probabilities and a reverse KL objective.
Read articleRunning a model close to data can change the architecture. It does not replace input processing, access control or real evaluation.
Read articleA PyTorch blog post describes Jagged Flash Attention, an attention kernel behind Meta's Generative Ads Model, developed with TLX for NVIDIA Blackwell. The post reports better performance than FlashAttention-4 on the GEM jagged shapes.
Read articlePublished on October 1, 2026, the post says Looped-DiT reuses Transformer blocks during denoising and that a 260-million-parameter model beat one 6.5 times larger on text-to-image benchmarks, with 4.9 times less inference computation.
Read articleA post dated October 1, 2026 reports 890 bytes of global KV cache per token and a persistent cache eight times smaller in a language model.
Read articleA September 30, 2026 publication argues that factoring S = WWᵀ changes gradient flow in attention and removes an information-exponent bottleneck that can stall unfactorized models.
Read articlePublished on September 30, 2026, the item describes DeepEP-Ascend as a high-performance communication library for machine learning training and inference on Huawei Ascend NPUs.
Read articleAn article by Reiner Pope and coauthors discusses scaling Transformer inference efficiently, a relevant topic for engineers evaluating ways to serve these models.
Read articlePublished on September 30, 2026, ProRL Agent is open-source infrastructure for multi-turn LLM agent rollouts. Its rollout-as-a-service model separates trajectory generation, environment orchestration, and evaluation from policy optimization.
Read articleA September 29, 2026 report says BF16 rounding in FlashAttention may break a softmax-gradient symmetry and cause gradients to explode near the end of training. The authors propose GProj as a correction.
Read articleThe article examines positional encodings for Transformers, notes RoPE's bias toward nearby tokens, and organizes evaluations of alternatives across distinct configurations.
Read articleA September 29, 2026 preview presents a benchmark for comparing information-flow methods and assessing whether deeper Transformers use their layers effectively.
Read articlePublished on September 29, 2026, the summary describes a study on transferring knowledge from pretrained Transformers to Mamba models and notes memory and throughput advantages for SSMs.
Read articleRota Nacional
30 days, no card, with a starting quota. After that, Pix credit from R$ 10,00.