HN

Comments (64)

• Menu

  smjburton
  > In March 2026, I migrated to self-hosted object storage powered by Versity S3 Gateway.Thanks for sharing this, I wasn't even aware of Versity S3 from my searches and discussions here. I recently migrated my projects from MinIO to Garage, but this seems like another viable option to consider.
• kikoreis
  As a user for over a decade, just here to submit my appreciation. healthchecks.io is fantastic.
• rapnie
  For S3 self-hosters check out Garage, who are developing with NLnet funding: https://garagehq.deuxfleurs.fr
• Menu

  lsb
  Self Hosted object storage looks neat!For this project, where you have 120GB of customer data, and thirty requests a second for ~8k objects (0.25MB/s object reads), you’d seem to be able to 100x the throughput vertically scaling on one machine with a file system and an SSD and never thinking about object storage. Would love to see why the complexity
• Menu

  tobilg
  I don't get it, if it's running on the same (mentioning "local") machine, why does it even need the S3 API? Could just be plain IO on the local drive(s)
• Unsponsoredio
  love this. one person, 119GB, two drives, rsync. no kubernetes, no distributed cluster, no nonsense. just works.this is the kind of setup that lets you actually go to bed without checking your phone every 20 minutes.
• Menu

  _joel
  I'm sure it's a lot better now but everytime I see btrfs I get PTSD.
• Menu

  chasd00
  > The costs have increased: renting an additional dedicated server costs more than storing ~100GB at a managed object storage service. But the improved performance and reliability are worth it.Were your users complaining about reliability and performance? If it cost more, adds more work (backup/restore management), and the users aren't happier then why make the change in the first place?
• arend321
  It's interesting that cloud providers are unable to provide stable S3 as a service. Hetzner is unable to deliver stable object storage, but given the article neither are OVHCloud and UpCloud.
• choilive
  Moved object storage from AWS to CloudFlare and have been pretty happy. No problems with performance so far. Bills were 90% cheaper too (free bandwidth)
• Menu

  iamcreasy
  Given the individual file size and total volume, I'd argue it make sense to use move to local only storage.On a separate note, what tool is the final benchmark screenshot form?
• Menu

  jerf
  "Our current (April 2026) object usage is: 14 million objects, 119GB"I mean, I appreciate the openness about the scale, but for context, my home's personal backup managed via restic to S3 is 370GB. Fewer objects, but still, we're not talking a big install here.This is pretty much like that story of, if it fits on your laptop, it's not big data.
• sigmonsays
  how do you provide highly available fault tolerant storage?versity does not include any erasure coding or replication...
• Menu

  bityard
  As someone who has dealt with wacky storage issues/designs, a lot of this "felt" strange to me. Btrfs? Rsync? Then I got to the bottom and saw that they were only handling about 100 GB of data! At that scale, nearly anything will work great and TFA was right to just pick the thing with the fewest knobs.At a previous job years ago, we had a service that was essentially a file server for something like 50TB of tiny files. We only backed it up once a week because just _walking_ the whole filesystem with something like `du` took more than a day. Yes, we should have simply thrown money at the problem and just bought the right solution from an enterprise storage vendor or dumped them all into S3. Unfortunately, these were not options. Blame management.A close second would have been to rearchitect dependent services to speak S3 instead of a bespoke REST-ish API, deploy something like SeaweedFS, and call it a day. SeaweedFS handles lots of small files gracefully because it doesn't just naively store one object per file on the filesystem like most locally-hosted S3 solutions (including Versity) do. And we'd get replication/redundancy on top of it. Unfortunately, I didn't get buy-in from the other teams maintaining the dependent services ("sorry, we don't have time to refactor our code, guess that makes it a 'you' problem").What I did instead was descend into madness. Instead of writing each file to disk, all new files were written to a "cache" directory which matched the original filesystem layout of the server. And then every hour, that directory was tarred up and archived. When a read was required, the code would check the cache first. If the file wasn't there, it would figure out which tarball was needed and extract the file from there instead. This only worked because all files had a timestamp embedded in the path. Read performance sucked, but that didn't matter because reads were very rare. But the data absolutely had to be there when needed.Most importantly, backups took less than an hour for the first time in years.
• jeffrallen
  > Our S3 API is now served by Versity S3 Gateway and backed by a plain simple Btrfs filesystem.With apologies to the SRE Book ("hope is not a strategy")... Btrfs is not a strategy.
• tonymet
  great writeup. Is s3 a customer or internal requirement? Why not write to your disks? Easily 1/3 - 1/5 the price and better performance.