HN

Comments (106)

• Menu

  yjftsjthsd-h
  > Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work.Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
• tgtweak
  I think you can definitely improve the throughput/iops by using BAR vs treating it like a file store/mount through cuda which adds a lot of overhead.
• Menu

  RachelF
  Nice idea, but something has gone very wrong here:>Sequential throughput: ~1.3 GB/s[on a RTX 3070 Laptop]This RTX 3070 chip is on PCIe 4.0 x16 which should give 64GB/s. The 8GB of GDDR6 is 448GB/s.Swapping to an NVMe drive would be twice as fast, but with higher latency.
• Menu

  xfalcox
  Given my dev machine has 32GB of RAM and 32GB of VRAM that sits mostly idle when I'm not running AI models, this is not that bad of an idea.
• drdaeman
  What about backpressure, how does it handle requirements for VRAM allocation when VRAM is used for swap space?With X11 it's not that bad (buffers are pre-allocated), but with Wayland allocations are a lot more dynamic, so running low on VRAM can easily crash the whole desktop. I just had a few of such crashes with Hyprland+llama-server+KVM switching between computers without freeing VRAM.
• Menu

  kimixa
  I remember this being a thing done a while back using linux's MTD/phram drivers - https://wiki.archlinux.org/title/Swap_on_video_RAM - not sure if that's still relevant though as I don't know how it'll interact with DRM and how it handles reserving some of the vram - the suggested limit using xorg.conf is probably pretty obsolete now.That page also has a fuse filesystem implementation on top of opencl - https://github.com/Overv/vramfs - which may be more compatible.
• Menu

  molticrystal
  For windows I saw something similar to this years ago. An experimental proof of concept driver that allows the creation of a ram drive from vram for NVIDIA cards. Sequential is fast as you'd expect, random has lots of room for improvement.>GpuRamDrive>Create a virtual drive backed by GPU RAM.https://github.com/prsyahmi/GpuRamDriveFork with AMD support:https://github.com/brzz/GpuRamDrive/
• Menu

  tlb
  Building a swap device at user level used to be one of those classic unsolvable problems, because what if your daemon needs to swap in a page in order to swap in a page? Or at least it was discussed at a reason why microkernels will never work. I’m not sure what the solution is here.
• Menu

  rwmj
  Similar but using OpenCL APIs, so it works on AMD (for some definition of "works" since their drivers are quite buggy): https://libguestfs.org/nbdkit-vram-plugin.1.html
• Menu

  dragontamer
  Remember how 16GBs used to be an enterprise level database mainframe?Well, GPUs also have stupid amounts of compute on them. I have to imagine that there is some kind of database format that's useful with GPU compute attached.Since the data is already in VRAM, the GPU can sort, join, or otherwise manipulate data as needed.
• Menu

  ProllyInfamous
  Why does my Apple Sillicon Mac with 32gb of RAM use (or even create?!) a swapfile, when 20gb is still unused/"free"?Why can I not just enter a simple command to entirely-disable swapfile, like with Linux's:>>>>swapoff -aSeems kind of silly, unless the point is intentionally to wear-down the SSD's lifespan.Having a GUI swapfile-disable system preference would be awesome. It would also be awesome if Apple finally abandoned this system settings/layout "phase" – it's still word-salad (compared to decades of preference panes).#Apple #Feedback #swapfile
• NortySpock
  Nice, I might try using this as I'm currently on 16 GB of RAM / 11 GB VRAM and feel like the VRAM is usually idle except for when I game or try a local LLM.It would be nice to have dynamic scaling or even just auto-shutoff on VRAM pressure if I forget I have this enabled and then fire up a game or LLM.
• Menu

  willis936
  I'm more interested in the opposite. Nvidia linux drivers crash when you try to address more VRAM than you have. It'd be nice if they didn't.
• Menu

  dlt713705
  Does anyone these days really use swap for anything than S4 suspend ?
• sgjohnson
  >Sequential throughput: ~1.3 GB/ssounds VERY low, also, wouldn't random read/write speed be MUCH more relevant here?
• londons_explore
  It seems obvious to me that this should be a built in functionality of the kernel.The kernels job is to manage resources - and GPU ram is one such resource, and it can be used for many of the same uses as regular ram.
• theblazehen
  I've implemented the same idea with OpenCL: https://github.com/theblazehen/vramblkThere is originally https://github.com/Overv/vramfs however that has the overhead of a FUSE filesystem + loop device when using as a swap device.The performance is rather lackluster however, it's far from a miracle "now you effectively have more ram for a 90% performance drop" - it definitely feels like traditional swapping
• Menu

  mmastrac
  I seriously looked at this as a way to improve the RAM situation in a QNAP 2U unit that I was having trouble sourcing RAM for. It's somewhat annoying that legit memory-over-PCIe is gated on PCIe5 and chipset support.In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?
• LouisvilleGeek
  Finally a use for the expensive ram when it's not needed in workloads!Now if it could be dynamically used and vacated on other GPU workloads?
• steeve
  sidenote: it is possible to use Vulkan to map GPU memory to CPU space and even map it back to CUDA: https://x.com/steeve/status/2055042304344231978?s=20
• Menu

  hardwaresofton
  You want to waste VRAM, in this economy?
• mrwizrd
  I have long wished it was possible to do this. What a great bit of code. Thanks.
• UnfitFootprint
  No software benchmarks? BAR for RAM is cool but I want to see how much it _actually_ beats pcie nvme
• jcmfernandes
  Q: Why? A: Why not?
• Menu

  bobsmooth
  RAM disks have always fascinated me. In a different timeline every PC has a 100gb of RAM and 50TB HDDs are the norm.
• zx8080
  Nvme ssd weights much less than GPU, and it matters for a laptop.
• Menu

  effnorwood
  use your car for an anchor on a big boat!
• Menu

  enthus1ast_
  And now, I want HDMI as direct connection hight speed network socket.
• anon
  undefined
• Menu

  nialv7
  I mean, you prompted something useful out of an AI, good job. But then use that to ask for donation? Feels weird, man.
• usxr1515
  Nice
• dbdr
  [dead]
• moviepiq
  [dead]
• vecchio
  [dead]
• Sohcahtoa82
  [dead]
• Menu

  simonask
  I mean, cool, but I’d rather not?
• lowbloodsugar
  This is why I read HN.