Google releases Gemma 4 12B, an encoder-free multimodal open model
The 12-billion-parameter model folds vision and audio processing directly into the language backbone rather than using separate encoders, and runs on 16GB of memory.
- Open weights & ecosystem
- Minor
Google released Gemma 4 12B, an open-weight model in its Gemma line that handles text, images and audio without the separate encoder modules most multimodal models use. For vision, Google replaced the usual vision encoder with what it described as a lightweight embedding module built from a single matrix multiplication plus positional embeddings and normalisation; for audio, the raw signal is projected directly into the same representational space as text tokens. Google called it the first mid-sized model with native audio input built this way.
At 12 billion parameters, Google said the model approaches the performance of its own larger 26-billion-parameter mixture-of-experts model on standard benchmarks while using under half the memory, and that it runs locally on consumer hardware with 16GB of VRAM or unified memory — a laptop-class machine rather than a data-centre GPU. The model is released under an Apache 2.0 licence, with weights on Hugging Face and Kaggle and support in tools including Ollama, LM Studio and Google’s AI Edge Gallery app.
The release continues Google’s practice of pairing its closed frontier Gemini models with an openly licensed, locally runnable Gemma line, competing for the developer and on-device market that Meta’s Llama and various Chinese labs’ open releases also target.