HN

Comments (79)

• Menu

  mk_stjames
  I saw '1-bit' and my mind first went to 1-bit dithered B&W image generation, not 1-bit model weights....and so now I'm wondering how cool /fast / compressed a diffusion image generator could be if the images it was trained on / space it worked in was limited to 1 bit (Floyd-Steinberg / Atkinson / your favorite algo here) dithered images.Training would surely be pretty quick and probably fit onto one modern GPU.
• Menu

  lumost
  I actually can’t wait for the future where I upgrade hardware in order to upgrade my ai as an alternative to an expensive subscription.There are many problems I want to work on which require billions of tokens. These are completely inaccessible without corporate project sponsorship at the moment. An asic generation machine which can pump out a few 10s of thousands of tokens per second at opus4.6 quality is more than sufficient.
• Menu

  mft_
  Genuine question: is this solving a real problem?IME, the bottleneck when using diffusion models isn't storage space or memory, it's generation time. Lots of models will run on 8-12 GB 1080-generation GPUs onwards, or on Macs with similar memory, which are probably the bottom end from a GPU power perspective anyway. I also note that these models are marginally slower than the small FLUX.2 model they're based on.Okay, maybe this allows running a local model on something that has a reasonably powerful GPU and limited memory, like an iPhone, but is that really a common requirement?
• ttul
  Within a day, someone will have trained a LoRA for this 1-bit model that enables hentai content generation on your Apple Watch.
• Menu

  sorenjan
  They call it a diffusion model, but it's based on Flux.2 which is a rectified flow model.
• smallerize
  To our knowledge, Bonsai Image 4B is the first image model in its parameter class to run directly on an iPhone.Isn't SD XL 3.5B? And the refiner model is even larger. Those can run on an iPhone 13 Pro.
• jeroenhd
  Couldn't try it because the demo app is iOS only and the web version just crashes my browser. The small model is impressive but if you front load a 1.8GB text encoder model, the savings aren't quite as useful.I do wonder how these compare to existing image generation models. I've tried https://github.com/alichherawalla/off-grid-mobile-ai for a while but I find the image generation models rather lacking.
• Menu

  a1o
  Anyone could pickup the minimal hardware requirements for this? Like both RAM and Storage?
• wiradikusuma
  Is there a benchmark of local image generation models? Local = can run on a 16 GB MacBook or 8 GB+ NVIDIA card.
• Menu

  MitPitt
  Lately I've noticed posts with barely 10 points getting to HN frontpage. Was it always like this?
• potatoman22
  I wonder why they didn't use a Bonsai model as the text encoder
• junto
  Just a side note, that this website is classified by Apple as an Adult website. I have Limit Adult Websites set in Content & Privacy Restrictions switched on.Led me to wonder what happens if a domain gets a new owner, and they want to petition Apple to remove the block.
• captainregex
  what trade off would one need to clear to justify the hardware and the work to get this running locally as part of a broader system? It’s a lot of work setting up and maintaining a production harness/system on a local device. I don’t personally repeatedly generate images at a scale where using a lab’s app somehow burns all my tokens. I like the ideas of local ai but I don’t see widespread adoption of it happening in commercial or customer situations anytime soon no matter how little/good enough they get. Even Uber- token burn whiplash but I doubt their answer will be “run some of it local”. IT nightmare, I’d imagine.
• danielEM
  Is there a way to run it on Vulkan?
• sudb
  Very interested to see where this kind of work goes for on-device video generation!
• Menu

  janniks
  I was expecting to see images of Bonsai trees when I clicked this
• woadwarrior01
  The text encoder is still 4-bit quantized.
• jijji
  Using the demo and typing in "A sign that says xxxx" where xxxx is any text, it gets it wrong almost 100% of the time.
• moralestapia
  This is why I don't think the big AI companies and nvidia will dominate the market. AIs will just run locally, on whatever hardware you have. Perhaps that's why they worked on this yet-to-be-defined partnership with ARM.
• Menu

  iJohnDoe
  Does anyone ever get their stuff to actually work. Like actually load?
• SilentM68
  Question,Is it compatible with Ollama, ComfyUI or are those providers unneeded, compatible with low-end hardware?Also, where does "./setup.sh/ drop the components in Linux?Thank you, Sol
• yieldcrv
  impressive, combines a couple techniques that I always wanted the frontier models to havehaving trouble loading the webgl browser demo on my phone but no biggy
• maephisto666
  [dead]