ByteDance releases a real-time audio-visual model

ByteDance’s Seed research group released SeedRealtime, a model it describes as processing audio, video and text jointly inside a single architecture rather than chaining together separate speech-recognition, vision-language and text-to-speech components, which the company said introduces latency and information loss in conventional “cascaded” systems.

The distinguishing claim is over turn-taking. Most real-time voice systems rely on an external voice-activity detector to decide when a user has finished speaking and the model should respond; ByteDance said SeedRealtime instead judges timing continuously from the combined audio-visual stream — tracking scenes, speakers, pauses and background talk to decide when to answer, interrupt, or stay silent — with perception, understanding, decision-making and speech generation running in parallel rather than in sequence. The company reported that, in its own end-to-end human evaluation, SeedRealtime roughly halved the rate of audio-visual conversational pacing problems compared with cascaded models, though it did not publish a technical report, parameter count or independent benchmark alongside the claim.

The model has been made available free inside ByteDance’s Doubao consumer assistant, one of the most widely used AI chat apps in China, rather than released as open weights or through a developer API. It extends ByteDance’s Seed model family, which had shipped the Seed 2.1 line the previous June, into real-time, always-listening multimodal interaction — a category increasingly contested by Google, OpenAI and others building “always-on” conversational agents.