HN

Comments (81)

• Menu

  plantain
  You are completely mad if you are running any kind of valuable workload on Runpod community or Vast. Hosts are absolutely peeking inside the VM's (you can see it openly discussed in their Discords).
• Menu

  hypendev
  Oh god I dealt with this recently while training my own 1B model.Runpod - mostly okay, except sometimes instances get dropped on smaller ones. Prices float around 3.20 mostly for H100s. Cheaper ones exist but are PCIe. Best DX out of the bunch honestly (well, AX, as my agent was the one dealing with it). Didn't end up doing the 1B run on it, but did 100M+ runs and some minor experimental ones. Great for agentic exploration, "RSI", minor training loops and research.Vast.ai - a bit pricier for the same H100s and it didn't have any clusters. So I could at best split my workload across 2-3 different H100s and spend time syncing. Not worth it.Akash - since I'm trying to stay in a reasonable budget, I decided to try Akash, which was the worst experience out of all. Cards would be "available" but would never respond to your request. After trying to contact support via Discord, I got dozens of fake support accounts adding me to scam discords and asking for my wallet key. Support answered after a while, but without a real answer.Mithril.ai - Ended up using this one, while it seems quite startupy, the actual offer is nice, it was easy to get going and they offer quite cheap H100s, offering singles and clusters.Tbh while Skypilot is great, the marketplace fragmentation is leaving room open for a openrouter-like compute provider.
• Menu

  noodlesUK
  One thing that I'm not seeing mentioned very frequently is that it's really hard to actually get the GPUs that hyperscalers advertise depending on your region, use of cloud credits etc.On Azure if you have credits (even hundreds of thousands of USD of credits through programmes like MSFT for startups) you just can't get any GPU quota in any region, even if you beg and plead with support reps. Even if you have cash it's often difficult to get any of the cheaper cards. You can forget about spot instances etc because the regions are essentially all at 100% capacity.I've heard on other hyperscalers that it's also challenging.This is super chilling for anything other than GenAI inference/training because if you have some more traditional workload that requires GPUs (traditional ML, other kinds of modelling) you're SOL.
• Menu

  buredoranna
  There are a couple of new indices out there.[1] "Silicon Index". Built against pricing observations, including listed prices and transaction data.[2] "Ornn Compute Price Index (OCPI)". Claiming to use only executed transactions.Your mileage may vary, but I find these new indices to be quite compelling.[1] https://www.silicondata.com/products/silicon-index[2] https://data.ornn.com/And yup, its not just how much these resources cost, but how much people are willing to pay for them.(edit: formatting)
• Menu

  uneekname
  > 4.57× between cheapest and dearest, for the same chip> What the 4.57× is not> - InfiniBand> - A datacentre SLA> - Enterprise compliance, committed capacity...ok
• Menu

  dpflan
  This is premise for https://skypilot.ai/, right? Cloud provider arbitrage.
• cvhc
  I tried the TensorDock, which now offers H100 "from $2.25/hr", two years ago. The SSD IO was terrible, maybe oversold. My training workers was stuck at loading the dataset and cannot fully utilize the GPU.
• Menu

  tokepi
  There’s much more data here, with more providers and splits by the different H100 types: https://cloud-gpus.com/nvidia-h100/
• anon
  undefined
• covi
  The spread is indeed very big.Lowest $1.90/hr Average $7.08/hr Median $7.91/hrhttps://gpus.skypilot.co/?gpu=H100&count=8
• pwython
  I've only used Modal.com -- They give you $30 month of usage free. Otherwise, $3.95/hr for an H100 SXM5.
• nico
  Yes. And the prices a lot of times don’t even make senseI’ve run into many hosted GPUs with processes stuck, with outdated drivers/libraries, or poor Internet connections
• Havoc
  Nice that they clearly explained why it is different
• Menu

  fg137
  I guess the author hasn't discovered vast.ai which is a minefield.
• adeptima
  Speaking of Runpod.io GPU availability, it is definitely increased in the past 6 months, getting 80Gb+ vram on demand takes few mins (not 5-6 hours like it used to be) at 1-3 usd an hour rate ... which made me thought a week ago - finally GPU bubble peak arrivedAnother thought - i want to audit the actual usage of GPU like actual efficient inference time not billed one without relying on "trust me bro" billing. Especially on AWS. Had no trust issue with Runpod.ioAny effective inference audit ideas are appreciated
• matt3210
  These prices don't seem real. For me, runpod is 8.99 an hour for b300 right now.
• happyPersonR
  lol watch the $12 people just subcontract out to the $2 and hope no one notices lolll
• qoez
  I wish there was some kind of GPU marketplace so one could just arb this opportunity.
• zizzigy
  But a ps5.48xlarge has 8x H100s. Are the other options here comparable?
• Menu

  jdc0589
  runpod is incredible, im a big fan, but good luck with availability on any specific GPU above 20gb of vram when you actually want to use it.last time I was trying to run some workloads there the pricing of an h100 was largely irrelevant.
• gigatexal
  if they're all just an API away is anyone buying for 2.69 somewhere and then selling at 12.28 elsewhere arbitraging things?
• dyauspitr
  I mean after the initial capex you’re just paying for electricity.
• datareceipts
  [flagged]
• otterley
  [flagged]
• williamgators
  Seems once the government controls it the prices go haywire
• paxy
  Slop