DeepSeek open-sources DeepInfer, the software it uses to run its own AI models, on Nvidia and Huawei chips

DeepSeek on Tuesday, Oct. 20, 2026, released DeepInfer, the in-house inference framework that serves its DeepSeek-V4.1-Flash model in production, as free open-source code under the Apache 2.0 license, with support for Nvidia's newest data-center GPUs and Huawei's Ascend 950 chips.

Most AI labs treat their serving stack as the secret sauce, since it decides what each answer costs. DeepSeek just posted its recipe, and the fact that it runs on Huawei's Ascend chips as well as Nvidia's is the real headline: it's a working, production-tested way to serve a frontier-class model without American hardware. The asterisks matter, though. It runs one model, the numbers are self-reported, and the parts you need for a full service (caching, load balancing) aren't in the box. Still, a day after Washington ordered DeepSeek off federal computers, DeepSeek answered by giving its engine away to everyone else.

DeepSeek open-sourced DeepInfer, which it calls "the high-performance inference framework that powers DeepSeek's large-scale production services," under the Apache 2.0 license. Inference is the work of running a trained model so it can answer a request. Apache 2.0 is a license that lets anyone use, change and share the code, including in a product they sell. Right now the repo supports one model: DeepSeek-V4.1-Flash. The default checkpoint, meaning the default set of model weights, is DeepSeek-V4.1-Flash-1017. DeepSeek says inference code for future models, and later speed work, will land in the same repository. Those lines are DeepSeek's. This release does not claim to run other DeepSeek models.

On X, DeepSeek said DeepInfer is "Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We're sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable." Throughput is how many requests the system finishes. Latency is how long one answer takes. That post is DeepSeek's. It does not attach a single headline percentage for the speed-up or the savings.

The GitHub page lists what is in the box. Optimized kernels, the low-level math routines that make a model run fast, including DeepSeek's own FlashMLA, DeepGEMM and DeepEP. Parallelism for attention, so a long prompt can be split across chips, and expert parallelism for a mixture-of-experts model, which routes each request to a subset of specialist pieces. Speculative decoding, which drafts several words ahead and checks them in one pass. Asynchronous scheduling, so the next batch of work can be lined up before the last one fully finishes. Batch-invariant inference and deterministic sampling, so the same input can yield the same output, on Nvidia chips only. Structured outputs, so an answer can be forced into a pattern such as a regex, a JSON schema or a grammar. Those features are DeepSeek's list.

The hardware list is short. Nvidia SM100 chips, the company's Blackwell-generation data-center GPUs, and Huawei Ascend 950 NPUs. An NPU is a chip built to run neural networks. The repository does not say this release runs on other chips.

What is missing matters. DeepSeek's own page says this release does not include a KV cache system, prefill and decode disaggregation, or load balancing across copies of the model. A KV cache is the memory of earlier words, so the model does not recompute them. Prefill is reading the prompt. Decode is writing the answer. Disaggregation means running those two steps on different machines. Without caching, that split, and a way to spread load, this is not a complete serving system by itself. The release is offline inference: you run it on machines you control. Those limits are DeepSeek's.

DeepSeek credits vLLM, an open-source server many labs use to run models. It says DeepInfer "was originally built primarily on vLLM," with ideas from SGLang and Nvidia's TensorRT-LLM. The Next Web reports that the vLLM project thanked DeepSeek and said it would improve its own support for V4.1-Flash using ideas from DeepInfer. Those lines are DeepSeek's and The Next Web's.

The speed charts are DeepSeek's own. They were measured on 4 SM100 GPUs for prefill and 64 for decode. Bloomberg reported there is no independent benchmarking yet. By Tuesday the repository had passed 8,000 GitHub stars, and it was above 8,700 by mid-morning Central time. The star count is a popularity tally, not a speed test.

The timing is the other half of the story. A day earlier, the US Department of Homeland Security ordered civilian federal agencies to remove DeepSeek products. The Next Web also notes that earlier this month DeepSeek published tools for Huawei chips as an alternative to Nvidia's CUDA. CUDA is Nvidia's software layer for its chips. That Huawei context is The Next Web's.

In plain terms, DeepSeek on Tuesday posted the software it uses to serve DeepSeek-V4.1-Flash, free under Apache 2.0, and said it runs on Nvidia's newest data-center GPUs and Huawei's Ascend 950 chips. It runs that one model. The speed numbers are DeepSeek's. Caching, load balancing and a full hosted service are not in this release.