HN

Need help?
<- Back
DeepSeek just open-sourced its inference engine

DeepSeek just open-sourced its inference engine

ggcr

Comments (203)

  • schneehertz
    A nuclear bomb"Based on this framework, we have achieved high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (at off-peak pricing)"Source: https://zhuanlan.zhihu.com/p/2095560295930508953
  • wg0
    That's Kami Kaze for US frontier labs. There's no way from here for them towards trillion dollar valuations.That's strategic sabotage to say the least.Ban all open weight models! Cried the VCs.
  • GreenWatermelon
    Relevant: Deepseek reports its Revenue per GPU per hour at off peak prices (38$) [reddit link] https://www.reddit.com/r/DeepSeek/comments/1xanexq/deepseek_...Also cool (from the translated screenshot since original is in Chineese https://zhuanlan.zhihu.com/p/2095560295930508953)> the DeepSeek V4.1 Flash inference system has established three goals> 1-..... 2-..... 3. Supports retaining KVCache for more than 72 hoursThis is awesome
  • yaoke259
    Supporting Huawei Ascend 950 NPUs with open source inference code has huge implications in the global AI hardware market. Imagine if all smaller labs now can build on top of this stack and add increasing support for more compatible hardware , software and models. This could seriously challenge NVidias death grip. Although inference is still only a part of all AI workloads and open sourcing training on Huawei chips would be even bigger.
  • VonLuderitz
    This month I changed my Claude Max Subscription to DeepSeek API. V4.1 Flash is fast and perfect for everything and refuses to almost nothing.Hope DeepSeek starts month plans soon.
  • notrealyme123
    20 merge requests in the last two hours. And all look somewhat reasonable on first glance.I am happy for not maintaining a relevant open-source repo.
  • theplumber
    Let’s hope this is just the beginning of bringing the drama queens (Dario and Sam) closer to planet Earth.
  • xrisk
    For comparison, rental prices seem to be ~$2.75/ per H-100-hour [1]. Meanwhile, Deepseek says "38 USD per hour per card”. They haven’t specified which card as far as I can tell.[1] https://www.ft.com/content/a1815bc5-b5d5-47ff-bbbc-e1a689929...
  • mekpro
    Even though their API price is cheap compared to OpenAI/Anthropic, I still can’t “afford” to use their API because the subscriptions from OAI/Anthropic provide far higher value and peace of mind. I know for sure how much I need to pay each month. I believe most developers fall into this category as well, where $20/$100/$200 from Anthropic/OAI provides more value.This raises the question: why doesn’t DeepSeek provide a subscription service as well? Is it a better move to have a single pricing model like they currently do?
  • sinisterfeller
    Interesting how a comment asking about the reasoning behind releasing this as open source is getting attacked, as if doing a good thing should not ever be questioned. I do wonder of China's motives behind this as well.The government would not allow this open work if it did not benefit them somehow, or perhaps they feel that keeping it closed is not very important right now (maybe because they are somewhat behind the frontier).Comparing a university student Torvalds and state actors open sourcing code is pretty naive...
  • meffmadd
    Curious how the performance compares to vLLM or SGLang!They say it was inspired by both so I am wondering what the performance gain is.
  • armcat
    This is fantastic. I feel the KV cache system is basically a must for any serious inference hosting, both for the provider as well as the consumer. I guess they have this but just haven’t open sourced it yet.
  • _aavaa_
    Notably missing their kv cache system
  • ggcr
  • dzonga
    the banning of sending advanced chips, EUV machines to slow down the chinese was an own goal by the USA and the west.zAI served their latest model for free on chinese chips, now deep seek is saying run inference for cheap on Huawei chips.even the RAM madness is about to end once the chinese ram providers unleash Ram onto the market in 2027.lastly remember the chinese have the largest electricity production and the cheapest.
  • tinyhouse
    I love Deepseek. It's amazing how AI changed our (devs) perception of China.
  • k__
    If that's the thing that makes $0.00X cached inputs possible, then this is huge!
  • shafyy
    Can somebody explain like I'm five what this exactly is? How are providers like Fireworks or Novita providing inference today? Do they use some other inference engine software? If yes, what is the advantage of the DeepSeek inference engine?
  • jdw64
    I think the reason DeepSeek does this is partly to neutralize competitors' technology, but ultimately the core reason might be that data is what matters.
  • lukewarm707
    Supported hardware: Huawei Ascend.Deepseek moment.
  • m00dy
    They open source it because they don't need it anymore. Probably they are doing something better with Huawei
  • VictorGrosser
    [flagged]
  • maxkaya
    [flagged]
  • fukaiall
    [flagged]
  • baalimago
    [flagged]
  • rektlessness
    Deepseek released just enough to stir the pot but not enough to cook your own from scratch.Wake me up when we can clone, point it at a dozen H100s we own, and get a working production service.Anthropic and OpenAI will do just fine.