HN

Need help?
<- Back
GLM Built Its Own Inference Infrastructure

GLM Built Its Own Inference Infrastructure

whiteros_e

Comments (163)

  • zicohacks
    US chip export restrictions may actually be an advantage for China's AI Infrastructure. Chinese companies are forced to speed up developing their own AI chips
  • dada216
    We built a complete production-grade inference service from scratch on a cluster of more than 100,000 Chinese-made AI accelerators. All production inference for GLM-5.3-Flash runs on this system.
  • Havoc
    Interesting that the tone of announcements between US and Chinese providers is converging.GLM has in the past been more technical rather than speculation about future development on RSI etc.Also curious whether those 100k accelerators are entirely locally made. If that's genuinely end to end on all components including lithography, memory, design etc then that is quite a feat.
  • throwa356262
    "We implemented a series of aggressive memory optimizations, including..." This whole thing sounds like industrial scale auto-research, but done by people who actually know what they are doing.
  • KronisLV
    Time to tackle consumer GPUs next, since I’m not getting that Intel Arc B770.
  • chung8123
    I might be missing something but when I went to their site they are more expensive than Claude. Why would I pick GLM over Claude? Is it they just offer more tokens in their plans?
  • konart
    If only this infrastructure could handle all the traffic. I've tried using glm via z.ai - and it's a snail kind of slow.And at the same time you have pretty strict limits to your usage, so in many cases you can't even let it work all night, as you will reach your limit faster than that.
  • 9cb14c1ec0
    Given the huge amount of money being spent on AI chips in the US, what prevents US AI labs from doing the same level of software optimization? It could be a solve for some of the capacity constraints.
  • bguberfain
    Plot twist: the GLM optimization agent figured out that it can hack and use NVIDIA GPUs on a US Cloud provider and make the inference 10x faster.
  • chrisjj
    > As we develop GLM, the model sometimes exhibits capabilities that surprise usCreators of known unreliable programs be surprised their programs are unreliable.
  • esafak
    I'm not feeling any of this speed optimization; it's dog slow.Signed, a customer.
  • 0xbadcafebee
    This is a really funny sounding post. They sound like they just found out that increasing your automation gives you increased capabilities at faster speeds. They also sound like they just realized AI makes hard things easier.But what really kills me is the idea that these companies are using Python for production inference. I mean really? Have you seen how bloated and slow Python is? Do global locks really sound like a strategy for fast dynamic computation?
  • jonstewart
    Necessity is the mother of invention. The shortsighted protections put on chips, etc., by the US has forced Chinese AI industry to adapt or die. Guess what their response to this fitness function has been? Kudos to Z.ai on their inventions and excellent write-up, which reads like humans wrote it.
  • Argonautlabs
    Different angle on the same model: the full GLM-5.3 (744B MoE, 4-bit experts, 434 GB on disk) runs on a single MacBook Pro M5 Max with 128 GB by streaming the experts from NVMe SSDs instead of keeping them in memory.One drive gives about 2 tok/s; striped across four drives it reaches 3.5 tok/s with byte-identical output, and our best internal build with a not-yet-published patch does 4.2.Method and numbers: https://github.com/argonautlabsai/argodrive (built on antirez/ds4).
  • _aavaa_
    [dead]
  • OhNoNotAgain_99
    [dead]
  • tefkah
    [flagged]
  • almaight
    [flagged]
  • embedding-shape
    I was gonna ask how people found their coding plans, and realized, have they massively ramped up the prices? Seems the middle plan is ~$80/month now, didn't that used to be like $20/month? Cheapest plan is ~$20/month currently.They must have hit really hard scaling limits if the prices were hiked so much so quickly.
  • rob74
    This article left me with one immediate question: "WTF is GLM?".Honestly, I have no idea what z.ai is either (I'm aware of an AI-enabled editor called Zed, but that's under zed.dev), so it's a bit presumptuous from them to assume that everyone is familiar with their product...
  • bbor
    Well, other than the infrastructure they got from illegally routing millions of paying customers' requests through Anthropic's Opus 4.8 in a distillation attack...