DeepSeek open-sources the inference engine behind its API

DeepSeek released DeepInfer, the inference framework it says powers its production service for DeepSeek-V4.1-Flash, under the Apache 2.0 licence. The GitHub repository was created on 16 October, but its first commit is dated 20 October, which is also the day the South China Morning Post reported the release. It supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs.

DeepSeek’s accompanying Q&A document says “a single cluster currently serves 155 trillion tokens per day, with overall revenue of US$38 per GPU-hour (at valley pricing)”. Those are DeepSeek’s figures; the SCMP noted the company disclosed neither operating costs nor margins, so the revenue number is not a profit number. The release is also partial: the README lists a KV-cache system, prefill/decode disaggregation and cross-replica load balancing as required for a complete service but “not included”. It does include speculative decoding with confidence-scheduled verification, which the Q&A says cuts verified draft tokens by about 30%, and batch-invariant, deterministic inference on Nvidia hardware. DeepSeek said inference code for future models would follow. The repository had about 21,800 GitHub stars as of 24 October.

The same week brought two other releases. On 17 October DeepSeek published V4.1-Flash-1017, which its Hugging Face model card calls a minor update to the September model, with weights under the MIT licence. DeepSeek’s table reports Terminal-Bench 4.0 at 42.3, up from 31.2, against 51.8 for Claude Opus 5 and 39.9 for GPT-5.6 Sol, alongside shorter outputs; these are the company’s own results. It asks users to upgrade to the DeepSeek Harness agent framework, and the SCMP cited Artificial Analysis as placing the update among the strongest and most cost-efficient open-weight models.

Also on 17 October, DeepSeek opened its in-house search engine to developers as Search and Fetch APIs at $0.75 per 1,000 calls each, free for server-side tool calls from DeepSeek Harness, Claude Code and Codex until billing starts on 31 October. Its change log says the engine handles “hundreds of millions” of search requests daily, again a company claim.

Taken with the Ascend kernel ports, the releases give DeepSeek a published stack of model, agent harness, serving engine and search. They landed as China proposed export controls on AI technology, and as senior researcher Chen Deli posted that open versus closed AI was “a decisive battle between two futures”, the SCMP reported.