Distillation study: per-token KL direction matters more than rollout policy
A systematic study reports that the direction of per-token KL matters more than the rollout policy in on-policy versus off-policy distillation, questioning the idea that on-policy rollouts are inherently better.
According to the published summary, a systematic study reports that the direction of per-token KL matters more than the rollout policy in on-policy versus off-policy distillation. The work questions the assumption that on-policy rollouts are inherently better.
The result may guide choices of rollout policies and KL objectives when designing distillation recipes. The available text gives no additional metrics, models, scale or experimental details, so the reading should be done in the original study, identified by its title, at the source indicated by the curator. Rota Nacional does not train or distill models, and this item does not describe a platform feature.