xAI open-sources Grok-1

xAI published the weights and architecture of Grok-1, the base model underlying the Grok chatbot it had unveiled the previous November, under the permissive Apache 2.0 licence. The release consisted of the raw pretraining checkpoint from October 2023, not fine-tuned for chat or any specific task, along with JAX example code for loading and running it.

Grok-1 is a 314-billion-parameter mixture-of-experts model with 64 layers, activating roughly a quarter of its weights for any given token to keep inference costs down relative to its total size. Its 8,192-token context window was modest for the period. As a base model rather than an instruction-tuned release, it required substantial further work to reach chatbot-level usability, and its size made it impractical to run without significant GPU infrastructure.

The release followed months in which Elon Musk had publicly criticised OpenAI for abandoning the openness implied by its name, including a lawsuit filed against the company weeks earlier. Open-sourcing Grok-1’s weights let xAI position itself, at least symbolically, on the open side of that argument, even though the model was less capable on public benchmarks than closed frontier systems of the time and the licence covered only the base checkpoint rather than any subsequent fine-tuned or product version.