HN

Need help?
<- Back
Nobody knows what a used GPU cluster is worth

Nobody knows what a used GPU cluster is worth

rbanffy

Comments (240)

  • Animats
    "Secondary market data shows moderately-used 2 to 3 year old GPUs trading at 50% to 70% of new pricing under normal conditions."That's for good NVidia H100 units.[1] There's a shortage of those. That seems to be the price after removal, cleaning, testing and refurbishing. Raw units removed from a shutdown will not be as valuable.H100 units are available on eBay, but multiple sellers are using the same picture of a new unit in its original packaging, a bad sign.[2] Some even have pictures with the logos of a competitor.[1] https://introl.com/blog/secondary-gpu-markets-buying-selling...[2] https://www.ebay.com/shop/nvidia-h100-gpu?_nkw=nvidia+h100+g...
  • tyfon
    I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce.I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.
  • gnfargbl
    > The job of an operations team is to keep all of this in steady state. They know which racks run hot in summer, which cooling loops have been flaky since the last firmware update, which jobs to re-route when a node degrades but has not failed yet. None of that knowledge is written down. It lives in the team.Hmmn. All of this information should live in the monitoring system, in which case any frontier model will be able to get to grips with it in short order. It feels like the author doesn't really fully understand the changes brought about by the systems they are writing about.
  • fancyfredbot
    The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power.You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.
  • AYBABTME
    I don't get how this is unique to GPU clusters. As a general rule, underwriters are not qualified to operate and maintain the assets they underwrite loans for. That's why houses, cars, equipment, ... go at auction at a fraction of their value. And why lenders have insurance.
  • narrator
    One data point: 512GB Mac Studios are selling on ebay for double what they were selling for new at the beginning of the year.
  • trklausss
    The article frames a GPU cluster as if it was rocket science. If you swap a GPU cluster with, say, a power plant, the question doesn't change a bit. Also high complexity, multiple variables, you don't know the current output power, scheduled maintenance, skilled labor needed etc.In the end, they will either figure it out, or contract a company that can figure it out. Truth is, there will be many companies wanting to do the rental of those clusters for AI...
  • gadders
    A lot of big infra projects end up going bust, and then someone else buying up the company and its assets because they no longer have the original build costs to bear - EG EuroTunnel, Iridium etc.Someone should stand up a vulture fund to buy these data centres off companies when they go bust.
  • cmiles8
    All indications are there will be a lot of repossessed GPUs appearing on the market before too long. Likely to be messy for a while but will open up a lot of possibilities when it’s easy to get your hands on some secondhand GPUs.
  • effnorwood
    I do. As much as they will pay.
  • acd
    Used GPU hosting for pension funds.You go to ebay search for a used GPU. You get a price.Neither is used servers a new thing or used routers. There are established used server companies.https://en.wikipedia.org/wiki/The_Emperor%27s_New_Clothes
  • latchkey
    I like Meg a lot a human, but Meg is all doom and gloom. Every single post she makes is about how GPUs fail [0] and now she's onto how financing is a big thing just waiting to crash and "nobody knows what a used GPU cluster is worth"...Actually, we do, people offer them to me all the time. A used box of MI300x is $257k. "There is no GPU futures market"... actually there are a few of them that people have pitched to me.This article is a lot of words from someone who isn't actually buying or deploying compute. My point is... take it all with a grain of salt.[0] https://x.com/meggmcnulty/status/2040851080066859386
  • PeterStuer
    Learning from crypto, a GPU's life foes not end because it fails, althougg ofc that happens, but because an alternative comes along that makes running the old equipment no longer power efficient under the new market conditions.
  • somat
    Isn't it worth exactly what you can sell it for. a few ways to do this.slow and awkward, best market match: The auction. sell to highest bidder.faster and more customer friendly but poor market match until a lot of units sold: The store. guess price, adjust up or down to reach sell frequency desired.fast and good market match but takes a knowledgeable customer base: The reverse auction. Start with price too high lower it over time until it sells.
  • ponkpanda
    Very difficult to sense check that substack post without access to the TLB credit agreement.There are likely management service agreements from xAI proper -> SPV to cover precisely what the author talks about. Clearly, xAI could play games but without seeing the docs (which are not public), it's very difficult.This article's basic point is right though. On the other hand, the LTV of this deal was approx 50% debt-financed (not too high; very much depends on the "V"). At 12.5%, it's not as if its being priced as a high quality asset.Overall, substack post was too bearish. The wider point is that there's a lot of froth tied to what has now become systemically opaque - namely the circular deal flow that every hyperscaler, nvidia, neoclouds and friends are now engaged in. When the proverbial hits the fan, that stuff will be difficult to price and find few willing buyers with the competence to underwrite.The systemic issues are the bigger concern than one specific deal imo.
  • Zigurd
    Time to dig up the spreadsheet models of what dark fiber is worth.
  • guess_who_is
    American AI corps will soon find out the worth once Huawei brings the cheap accelerators in the market. 6-9 months left.
  • jmyeet
    This is a good article. I've been curious about how this is going to play out. A couple of data points:1. An enthusiast had a project to get a V100 working on his PC [1]. This was a ~$10k GPU 10 years ago. It's now sold for scrap;2. The A100 came out in 2020 and cannot run a large model like DeepSeek v4 Pro. It can run Flash. You need a 16xH100 cluster to run Pro and that's a ~4 year old GPU and AFAICT 8xB100 or 4xB200;3. We're about to roll out R100/R200s.I'm surprised that NVidia is moving to a 1 year product cycle (per this article) because the big question I've had is what's that going to do to existing investments in GPUs. Why? Because if 4xR100 can do the work of 32xH100 then that's a massive advantage in performance-per-Watt, which I think is going to be the only metric that ends up mattering.In addition to raw power, new capabilities are developed and come online. For example, certain smaller, more efficient quantization methods just didn't exist on older hardware.Oh, another thought from this: a 9% annual failure rate just goes to show you how ridiculous the idea of orbital data centers really is. Orbital DCs were always just a pump-and-dump scheme for SpaceX's IPO.Currently it gets expensive to run models larger than ~31B locally. You start to need some pretty expensive hardware. That's going to change. I don't expect we'll be running 1T+ models on a Macbook Pro within 5 years (at reasonable inference rates) but I think people today will be shocked at what's being run locally in 5 years and that'll easily be 100-200B+ models.[1]: https://www.hackster.io/news/hacking-a-server-grade-nvidia-g...
  • chasil
    Should I be worried that my plant is owned by Apollo?No, because I am retiring next month!
  • thelastgallon
    Why would anyone buy these used when the new one, Vera Rubin can generate 10× more tokens per megawatt? Electricity costs are the biggest cost. More importantly, DCs are of a fixed power capacity. A power efficient GPU will mean it can fit in legacy DCs, don't need mega scale newfangled DCs. Old GPUs in the recent years should be worthless.
  • semiquaver
    Maybe this article has something useful to say but the painfully LLM-generated prose is too distracting to make it evident.
  • andix
    I'm wondering if all those GPUs ever end up on the second hand market for us to buy. Or if they will get refurbished multiple times and get used in second tier data centers until the chips die.
  • keeda
    A relevant article I found from an industry insider (which could indicate bias but also relevance): https://www.whitefiber.com/blog/understanding-gpu-lifecycleWhich has this anecdotal data point:>... when I left Paperspace in mid-2024, our M4000 GPUs, nine-year-old GPUs, were still consistently utilized at near-total capacity. That’s not a typo. Nine. Years. Old. Still booked, still working, still generating revenue.Also, I won't claim to understand accounting, but in general it seems it is advantageous to accelerate depreciation schedules for high CapEx industries because they lower taxes: https://leyton.com/us/insights/articles/what-is-accelerated-...As such you'd assume these cloud providers to want faster depreciation of their GPU assets rather than slower? I suppose in this case they do have an incentive to show bigger revenue numbers, but there seems to be a trade-off here that is not being discussed?
  • jgalt212
    9% failure rate seems very high. What's the failure rate of a server CPU? Is the GPU failure rate high because it's 1000s of CUDA cores instead of 128 (Xeon)? If so, can't the H100 just work around the busted CUDA core(s)?
  • 1-6
    The GPU cluster's RAM is probably worth more than the flops these days.
  • baalimago
    Is this the CDOs of the AI-bubble?
  • Zarathustra30
    Is the 30-50% of face value realistic for "Liquidation Value"? If one organization has to liquidate, sure. But if the bubble pops and many groups have to liquidate at once? Owners will be lucky if they can dodge the recycling fees.
  • anon
    undefined
  • grim_io
    Nothing, because those "GPU's" are special proprietary hardware and are not what most people are capable of plugging in into anything.
  • AtlasBarfed
    Well it certainly doesn't appreciate.And I feel that AI hardware is going to rapidly depreciate faster than even typical PC hardware
  • tim-tday
    I’ll take “people who sell used hardware” for 500 Alex.
  • petesergeant
    “Nobody knows” = “the method sophisticated lenders are using to determine this is a commercial secret”If you are the CFO of a credible AI company you can pick up the phone and get some advice exceptionally accurate numbers on what the market things the value is very quickly.
  • nikanj
    Essentially nothing, fractions of a penny on a dollar, because players who could afford paying real money won't risk the crusty old hardware - so you're limited to buyers who still need a massive cluster but don't have AI infinite money glitch enabled
  • BenFranklin100
    Would it make economic sense to strip it down and sell for parts? That’s how it’s done now for older data centers, where the obsolete equipment is sent off to China, stripped down for parts, and sold on the secondary market. I’ve picked up several older but still useful RAID hardware cards off eBay this way.I’m mainly interested in getting some DDR4/5 and RTX5090s on the cheap :).
  • dfedbeef
    \__
  • aslkalska
    so you're telling me you can't use 1 or 2% percent of 5 billions dollars to rebuild a team that runs GPU clusters for like a couple of years ?! and the guy that borrowed billions from these banks would want to mess up that relationship for what? I mean he is a stupid narcissist but not to that degree. This whole article makes no sense to me.
  • Joel_Mckay
    Given the price for a rack of bc-250 after the crypto hype cycle, the expected value of the hardware will be around 5% to 10% of the original retail price.Without other market influences, that is a >90% expected discount when the over-provisioned market must inevitably self-correct.If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the dot com crash.I like the Shrek Movie correlation theory, as they always happen just before Debt-backed investors get hit hard... And the new film is due out in 2027. =3
  • rahulladumor
    [flagged]
  • possiblyburrito
    [flagged]
  • greenoracle9
    [dead]
  • mertleee
    [dead]
  • busymom0
    > Silent data corruption (SDC) is the most expensive, where a faulty GPU produces wrong answers without crashing anything, which means a multi-day training run can complete normally and the resulting model weights are quietly poisoned.Anyone know how these get caught ultimately?
  • timmmmmmay
    [flagged]
  • DiabloD3
    GPU clusters have, largely, no actual value.If anything, you might have to pay to have them disposed of, they don't really have any meaningful used eBay market outside of the randos that want to do high end extreme local inference in their basement.Also, as for RAMmageddon, the inference SBCs that all of the AI bros bought don't have DIMMs, they're not even the right chip: its all GDDR and LPDDR. The only DDR DIMMs being consumed are for regular non-inference machines that help run the business and service infrastructure behind the scenes.
  • pier25
    so when will xAI default on its debt?
  • cgyvbunji
    If nobody knows what a used GPU cluster is worth it means nobody is doing anything important with GPUs - time to short everything. Do you believe it?