Google launches Gemini

Google introduced Gemini, its first model family built from the start to be multimodal — trained jointly on text, code, images, audio and video rather than having other modalities bolted onto a text model afterward — in three sizes: Ultra for the most demanding tasks, Pro for broad deployment, and Nano for running on-device. Gemini Pro began rolling into Bard immediately; Ultra was reserved for a later release the following year.

Google’s launch material reported that Gemini Ultra exceeded the state of the art on 30 of 32 widely used academic benchmarks and scored 90.0% on MMLU, which it described as the first model to outperform human experts on that test — a direct, if not explicitly named, comparison with GPT-4, which OpenAI had reported scoring lower on the same benchmark.

The launch was accompanied by a video, “Hands-on with Gemini,” showing what appeared to be a fluid, real-time conversation in which the model responded to spoken prompts and live video. Bloomberg reported within days that the demonstration had not worked as shown: Google confirmed to the outlet that the video had been edited together from still frames and that the model’s inputs during filming were text prompts rather than the spoken narration heard in the final cut, dubbed in afterward. Google’s video description had disclosed only that “latency has been reduced and Gemini outputs have been shortened for brevity,” which critics said understated how heavily staged the demonstration was.

The dispute did not affect the underlying benchmark claims, but it fed a broader argument, recurring throughout the period, about the gap between how frontier labs presented model capabilities in launch media and what the models could actually do unassisted. Gemini Ultra itself did not ship broadly until February 2024, as Gemini Advanced.