Self-play explores pretraining without natural data

The work trains a language model on byte sequences produced by programs. A generator, updated through reinforcement learning, adapts as the model progresses, so the two components evolve in interaction without gradient updates on natural data.

The paper reports transfer to text, images, audio, and code not seen during pretraining, as well as in-context learning. This is an investigation into synthetic data and transferable structures, not proof that the method replaces natural data for every application. If using AI to study or apply the approach, avoid submitting personal data or internal material without authorization, and check your organization's data-handling policy.