Together AI launches RedPajama to reproduce LLaMA's training data
Unlike LLaMA, restricted to non-commercial research, the reproduced 1.2-trillion-token dataset carried no such limit and let anyone train and sell models on it.
Open weights & ecosystem