OpenAI releases Jukebox

OpenAI released Jukebox, a neural network that generated music as raw audio — including rudimentary singing with words — conditioned on a chosen genre, artist and set of lyrics, across a range of styles. Rather than producing sheet music or MIDI and synthesising sound from it, Jukebox worked directly in the audio waveform, which had previously made long-form music generation computationally impractical because of how much data even a few minutes of raw audio represents.

The system addressed that by using a multi-scale VQ-VAE to compress raw audio into a much shorter sequence of discrete codes, then training autoregressive Transformers to model those codes, allowing coherence — melody, rhythm and instrumentation staying consistent — across pieces several minutes long. OpenAI released the model weights and code alongside thousands of uncurated audio samples so listeners could judge quality themselves rather than rely on curated highlights.

OpenAI was explicit that the results were far from human-quality: vocals were often intelligible but musically and lyrically incoherent over longer stretches, and generation was slow, taking hours of compute to render a few minutes of audio. Jukebox nonetheless demonstrated that large generative models could be pointed at raw audio rather than text or images, and it sits alongside GPT-2 and DALL-E as an early marker of OpenAI extending the same scaling approach across modalities before doing so more emphatically with GPT-3 the following month.