HN

Need help?
<- Back
Revolut moves to self-hosted GLM

Revolut moves to self-hosted GLM

pretext

Comments (130)

  • littlecranky67
    "the gains showed up when tokens stopped being rationed" - I can totally see that. I am leading (as an external consultant) a larger dev team for a huge insurance company. The devs are quite junior or in general more the "I became a sw-eng here because that company is the only big employer in a 1h radius" type of people (meaning: No rockstar developers). Most of them do not play with AI in their spare time, and they have been told to spend claude tokens wisely. Everybody is too afraid now to play with it and explore during work hours for fear of being token shamed.At the same time, most of their junior devwork can be replaced completely by AI - so they really should start to learn how to work with it. I recommended everybody to get a private 20€/month claude subscription and use it for toy projects. I have younger devs (early 20s) in the team that told me they won't spend private money, or simply have no usecase in their private life for AI. We talk about software engineers here, and IMHO most of them are in denial of the fact that their skills have mostly become worthless if they don't start to pivot to something valueable.
  • jacquesm
    We did the same thing, on much less powerful hardware but it has served us very well and we don't need to worry at all about the question whether or not the suppliers of inference services can be trusted long term with our data. The answer to that question in my opinion is 'absolutely not'.
  • redmalang
    For the past several months about 90% of the work we do goes to open source models. If you ignore the noise around 'cheap' etc, you might find that with a little extra effort to more precisely define what you need not only do these models generally work as well, and they will generally more closely follow instructions - on longer timeframes this is actually pretty helpful.They are less good on ambiguous prompts like 'fix all the problems' (which to my chagrin is a very common prompt it turns out..)
  • gosen
    > Per David Abitbol, who leads AI ops there: 56 Nvidia B200s, a blended 30 cents per million tokens, roughly a tenth of what they paid Anthropic and OpenAI.So they've invested around $2.5M-3M into hardware + monthly operating costs (electricity, updates, cooling). I wonder how much did they spend on tokens to decide to make such a switch. I mean if they used to spend $500k-1M per month just on tokens then it would pay off in a year or so, assuming their infra will be stable and efficient.We'll see if that's a trend for mid-big companies or just an experiment.All that math is assuming that it's all self-hosted in their data center or so and not paying to a cloud provider, which will be probably more expensive.
  • world2vec
    At dayjob we're also using self-hosted GLM-5.3 via our flavour of pi.dev but teams still have discretionary power over using Claude Code or not (there's a team's budget to spend on whatever the team decides). Some teams like mine still like to use Claude Code for a lot of the work but most stuff doesn't require the latest and greatest LLM and can be done with "lesser" LLMS by simply throwing a lot of tokens and compute to it.
  • andy99
    This is a sane version of what “sovereign” AI can look like for many. In Canada we’ve just had this national plan to overpay for an inferior AI product in the name of sovereignty. Even if GLM is slightly behind, this is likely close to the Pareto frontier.They also could consider putting this on openrouter when they have spare capacity to make the most effective use of the hardware.
  • irthomasthomas
  • anon
    undefined
  • InsidiousWombat
    If investors knew how powerful open models are the AI companies valuation would tank overnight
  • corford
    What does "blended $0.30 per 1M tokens" mean? Is that input or output or average across the two? (asking as am trying to benchmark that cost against public GLM providers on Openrouter)
  • hmokiguess
    I would sacrifice a few benchmark points for higher tok/s actually, especially if more bandwidth as well so I can scale up swarms, then I can likely compensate for any drop and get more mileage out of it.
  • mtrovo
    30 cents per million tokens is wild.The relevant question is how easy is to setup 56x B200s in a cluster, are we talking about a adhoc infra team job or clicking a button on AWS?
  • swyx
  • torginus
    This is the right call - there's little proof that the quality gap between open models and OAI/Anthropic stuff translates to business results, but you surrender all control of your data to a third party.The last cause might be a straight up blocker - certain financial regulations straight up prevent you from giving third parties to your data (esp outside national/EU jurisdiction), no matter what documents you sign.
  • koliber
    If I wanted to buy 56 B200's and had $3 million, when could I get delivery?
  • chr15m
    Anthropic's missing last quarter results in a nutshell.
  • orphereus
    Here's the part that matters:
  • injidup
    This is probably only relevant if you go over the 150 engineers putting you in the enterprise bucket and forcing you to pay by the token rather than the subsidized Claude code plans. If you need 150 engineers all with frontier model accounts maybe you are doing something wrong ;)
  • aurareturn
    It's probably true that open weight models are "good enough" for most software engineering tasks.But the goal posts will move and I think frontier labs will find new areas that have a need for absolute frontier.
  • epolanski
    My mid size German client (100M revenue) is also moving lots of work to on premise LLMs.I have no clue how do the lunatics think there's any moat in models.
  • iLoveOncall
    > the gains showed up when tokens stopped being rationedMy whole company (Amazon) has unlimited tokens, and the only thing it has led to is insane amounts of slop. Illusion of productivity.
  • amagilabs
    [flagged]
  • howtageek
    [flagged]
  • SidMangsatabam
    [flagged]
  • wieieie
    [dead]