Pythia

Researchers studying transformer training need checkpoints taken throughout pretraining, not just a finished weight file. Pythia delivers that by training