HN

Comments (152)

• Menu

  rockorager
  Author of the project here! I have a little write up on this here: https://rockorager.dev/log/lsr-ls-but-with-io-uring
• Menu

  ninkendo
  I wonder how it performs against an NFS server with lots of files, especially one over a kinda-crappy connection. Putting an unreliable network service behind blocking POSIX syscalls is one of the main reasons NFS is a terrible design choice (as can be seen by anyone who's tried to ctrl+c any app that's reading from a broken NFS folder), but I wonder if io_uring mitigates the bad parts somewhat.
• Menu

  swiftcoder
  Kind of fascinating that slashing syscalls by ~35x (versus the `ls -la` benchmark) is "only" worth a 2x speedup
• Menu

  maplant
  This seems more interesting as demonstration of the amortized performance increase you'd expect from using io_uring, or as a tutorial for using it. I don't understand why I'd switch from using something like eza. If I'm listing 10,000 files the difference is between 40ms and 20ms. I absolutely would not notice that for a single invocation of the command.
• Menu

  Imustaskforhelp
  Really interesting, the difference is real though I would just hope that some better coloring support could be added because I have "eza --icons=always -1" command set as my ls and it looks really good, whereas when I use lsr -1, yes the fundamental thing is same, the difference is in the coloring.Yes lsr also colors the output but it doesn't know as many things as eza doesFor example .opus will show up as a music icon and with the right color (green-ish in my case?) in eza whereas it would be shown up as any normal file in lsr.Really no regrets though, its quite easy to patch I think but yes this is rock solid and really fast I must admit.Can you please create more such things but for cat and other system utilities too please?Also love that its using tangled.sh which is using atproto, kinda interesting too.I also like that its written in zig which imo feels way more easier for me to touch as a novice than rust (sry rustaceans)
• Menu

  SillyUsername
  Love it.I'm trying to understand why all command line tools don't use io_uring.As an example, all my nvme's on usb 3.2 gen 2 only reach 740MB/s peak.If I use tools with aio or io_uring I get 1005MB/s.I know I may not be copying many files simultaneously every time, but the queue length strategies and the fewer locks also help I guess.
• Menu

  pvtmert
  > I have no idea what lsd is doing. I haven’t read the source code, but from viewing it’s strace, it is calling clock_gettime around 5 times per file. Why? I don’t know. Maybe it’s doing internal timing of steps along the way?Maybe calculating "X minutes/hours/days/weeks ago" thing for each timestamp? (access, create, modify, ...). Could just be an old artifact of another library function...
• Menu

  the8472
  io_uring doesn't support getdents though. so the primary benefit is bulk statting (ls -l). It'd be nice if we could have a getdents in flight while processing the results of the previous one.
• buybackoff
  A little offtop, but do you know a number in usecs that io_uring can save on enterprise grade servers, with 10G NICs, for socket latency overheads vs LD_PRELOAD when hardware supports that? Let's say it's Mellanox 4 or 5. My understanding is that each gives around 10us savings, maybe less. Based on some benchmarking, which was not focused on any of those explicitly but had some imprecise experiments. It also looks like they do not add up. Do you have a number based on real experience?
• jasonjmcghee
  I find it funny that there are icons for .mjs and .cjs file extensions but not .c, .h, .sh
• Menu

  tln
  The times seem sublinear, 10k files is less than 10x 1k files.I remember getting in to a situation during the ext2 and spinning rust days where production directories had 500k files. ls processes were slow enough to overload everything. ls -F saved me there.And filesystems got a lot better at lots of files. What filesystem was used here?It's interesting how well busybox fares, it's written for size not speed iirc?
• Menu

  quibono
  Lovely, I might try doing this for some other "classic" utility!A bit off-topic too, but I'm new to Zig and curious. This here: ``` const allocator = sfb.get(); var cmd: Command = .{ .arena = allocator }; ``` means that all allocations need to be written with an allocator in mind? I.e. one has to pick an allocator per each memory allocation? Or is there a default one?
• Menu

  adgjlsfhk1
  It's a shame to see uutils doing so poorly here. I feel like they're our best hope for an organization to drive this sort of core modernization forward, but 2x slower than GNU isn't a good start.
• Menu

  Bender
  I am curious what would happen if ls and other commands were replaced using io_uring and kernel.io_uring_disabled was set to 1. Would it fall back to an older behavior or would the ability to disable it be removed?
• Menu

  mnw21cam
  Love the idea and execution, don't love the misplaced apo'strophe's.
• Menu

  ReDress
  I've been playing around with io_uring for a while.Still, I am yet to come across a some tests that simulate typical real life application workload.I heard of fio but are yet to check how exactly it works and whether it might be possible to simulate real life application workload with it.
• Menu

  neuroelectron
  There used to be lsring by Jens Axboe (author of io_uring), but it no longer exists. This is more extreme than abandoning the project. Perhaps there is some issue with using io_uring this way, perhaps vulnerabilities are exposed.
• Menu

  fermuch
  The link isn't working for me. For those who were able to see it: does it improve anything by using that instead of what ls does now??
• Menu

  movomito
  Link doesn’t work
• Menu

  api
  Why isn’t it possible — or is it — to make libc just use uring instead of syscall?Yes I know uring is an async interface, but it’s trivial to implement sync behavior on top of a single chain of async send-wait pairs, like doing a simple single threaded “conversational” implementation of a network protocol.It wouldn’t make a difference in most individual cases but overall I wonder how big a global speed boost you’d get by removing a ton of syscalls?Or am I failing to understand something about the performance nuances here?
• Menu

  rkangel
  This was more interesting for the tangled.sh platform it's hosted on. Wasn't aware of that!
• Menu

  danbruc
  Why does this require inventing lsr as an alternative to ls instead of making ls use io_uring? It seems pretty annoying to have to install replacements for the most basic command line tools. And especially in this case, where you do not even do it for additional features, just for getting the exact same thing done a bit faster.