Megatron-LM

Megatron-LM by NVIDIA is an open-source framework designed to train very large transformer-based language models efficiently. It supports training models