DeepInfer opens DeepSeek inference without delivering the full machine

DeepSeek releases DeepInfer, the engine serving its production models, under Apache 2.0 license. The tool targets inference costs, with Nvidia and Huawei support, but without a ready-to-use full stack.

DeepSeek has released DeepInfer, the inference engine that serves its models in production, under the Apache 2.0 license. The tool allows running DeepSeek-V4.1-Flash on a private infrastructure, with optimizations close to those used by the company. The launch comes as Washington tightens its restrictions on DeepSeek technology in US civilian agencies. In this context, the issue goes far beyond a single model: it concerns the deployment cost of AI and the battle over infrastructure components.

An infrastructure component, not a complete kit

DeepInfer does not deliver a ready-to-plug production platform. DeepSeek specifies that the open version covers offline inference and the functions necessary to serve a model at large scale, but some components are still missing such as KV cache management, separation of inference steps between machines, and load balancing between replicas. In other words, the company is opening the engine, not the entire chassis.

This nuance matters because inference is often the most expensive part of an AI project in production. Computing is not just a matter of model size: how requests are distributed, grouped, and executed on chips directly affects latency, throughput, and energy costs. DeepSeek says it wants to help the community to "lower compute costs, reduce energy use, and make AI more affordable." A sober phrase, a massive goal.

Nvidia and Huawei side by side, no false suspense

The detail that draws attention is in the hardware supported from the start. DeepInfer runs on Nvidia SM100 GPUs, used in the Blackwell generation for datacenters, but also on Huawei Ascend 950 NPUs. Huawei’s presence alongside Nvidia is no coincidence: it reflects US export restrictions and the technical repositioning of Chinese players around domestic hardware.

DeepSeek had already mentioned, in April 2025, the difficulties in releasing its complete engine. The company then explained that its system was too coupled to its internal infrastructure to be publicly deployed without heavy modifications. The new relaxation seems to circumvent this problem by limiting the scope. However, some functions, such as batch deterministic inference, are only available on Nvidia, and the performance published by DeepSeek relies on SM100 GPUs, with four units for pre-filling and 64 for decoding.

The cost, the real battleground

At the heart of this announcement is a very concrete pressure: to lower the execution cost of models. DeepInfer integrates optimized kernels, communication libraries, speculative decoding, asynchronous scheduling, and support for structured outputs. DeepSeek also publishes deepseek-recipe, a companion library to convert API requests into model inputs and transform outputs, with versions for Rust and Python.

The message is clear for technical teams: performance is no longer only about training, but about the execution workshop. This is where AI services win or lose their economic viability. As a reminder, DeepSeek also recently launched Search and Fetch APIs around its internal stack, showing a strategy of opening by successive layers rather than a big reveal all at once.

Washington tightens the screws, DeepSeek opens the hood

The timing is almost ironic. On one hand, the US government has ordered federal civilian agencies to remove DeepSeek technology from their systems within 60 days, including when it is found in third-party products built on its models. On the other hand, DeepSeek makes one of the most sensitive parts of its production chain freely accessible. The coincidence is not accidental, but it says a lot about the current moment: regulation closes doors, open source opens others.

For private companies, the interest is immediate. DeepInfer can serve as a base to reproduce some of DeepSeek’s optimizations and reduce compute expenditure, but it does not replace a complete platform. Organizations will still need to assemble their orchestration, supervision, and operational layer around it. The engine is there. The dashboard, however, remains to be built.

By the numbers

  • 60 days — deadline given to US civilian agencies to remove DeepSeek, according to the DHS directive.
  • 2 architectures — Nvidia SM100 and Huawei Ascend 950, supported at DeepInfer’s launch.
  • 2 languages — Rust and Python, available for the deepseek-recipe library.
  • 4 GPUs — used for pre-filling in the performance measurements published by DeepSeek.
  • 64 GPUs — mobilized for decoding in the same DeepSeek tests.

To read