HN

Need help?
<- Back
Liquid AI reveals 8B-A1B MoE trained on 38T

Liquid AI reveals 8B-A1B MoE trained on 38T

simjnd

Comments (37)

  • SubiculumCode
    Anybody use their localcowork [1] before? That is where the demo lives. Or not?[1] https://github.com/Liquid4All/cookbook/tree/main/examples/lo...
  • Ifkaluva
    Liquid does amazing work, but I kinda feel like they are overtraining their models. 38T tokens seems like a lot for an 8B model
  • mlmonkey
    Question: I have a dirty car and the car wash is just 50 meters away. Should I walk or drive to the carwash?Answer: . . . . So, unless you have a compelling reason not to, walk to the car wash.
  • irthomasthomas
    Woah, chinchilla scaling is 20 x active_params. I think mistral was 2 x Chinchilla. This is 1800 x
  • chabes
    The small models are getting really impressive.I recently realized that Qwen3.5:4B is way more capable than I thought a model that size could be.Combine that with the work Liquid puts into RL and fine tuning, and you get models that perform extremely well on minimal hardware.Combine that with your own fine tuning, and you get a specialized tool that is fast, private, and doesn’t require internet connection.
  • adityashankar
    This is super interesting, I'm particularly excited for this one as it may allow teams to scale this architecture for VLAs (vision language action models), and having sparser models means more real-time actions on a locally hosted modeldemo link for anyone that wants to try this out https://playground.liquid.ai/chat?model=cmppnbgse000004l4bc8...
  • kilroy123
    Hmm, I asked it who made it, and it says Google?
  • elorant
    Wow, this is fucking phenomenal. I fed it a long transcript asking it to create a summary and it executed it extremely well. For an 8B model this is quite impressive.
  • bee_rider
    They seem… much better than all the models they compared against? What’s the catch?
  • jauntywundrkind
    I really love how fast it is! Their press release comparing it on Strix Halo and M5 Max are impressive. It going twice as fast at GPU benchmarks even more so!
  • ramshanker
    Guess we can run this even on CPU!
  • zmmmmm
    No vision support?
  • HenryMulligan
    Why does this not have (day-one) support for Ollama? The previous model is on there? Is it related to the ongoing refactor work or are people abandoning Ollama for other LLM engines?
  • gmuslera
    Homeopathic AI