Need help?
<- Back

Comments (67)

  • montroser
    This is bonkers good if these benchmarks hold up in real-world scenarios. To be at the level of Qwen3.8-27B with a looped transformer model is quite the flex. They just left this note on reddit:Hope that folks find this model more amenable to local usage - the sizing was largely informed by what folks on this subreddit ask for the most haha!There are two particularly interesting aspects of this model for local users:We do our training end-to-end in nvfp4, and although we haven't exhaustively benchmarked this, folks like Unsloth have noticed that our models seem to respond very well to being quantized, maintaining 90% top-1 even at at 1-bit quantization.This model is looped, which allows more depth at the same parameter count, which we particularly notice seems helpful for coding tasks. :)This is our first model targeted at local users, so let us know how this model is and anything you want to see out of future iterations!https://www.reddit.com/r/LocalLLaMA/comments/1y1diuk/thinkin...
  • monksy
    The GGUF version of this that can run on Ollama/Llama.cpp is: https://huggingface.co/unsloth/Inkling-2-Mini-GGUF
  • lilytweed
    Whoa, not just a model that acknowledges and embraces using a looped architecture (GPT Astra 6+ and Sol 6.1+ are only [heavily] rumored to be looped), but it's open weights, too?I'm willing to wager that most open models will go looped in the wake of this.Wonder what it means for safety and interpretability at this level of availability.
  • jakswa
    The demo video was 32 tok/s and did a 15X speedup when generating the code? Weird that they didn't publish token speeds for this laptop setup they recorded. I'm worried this tri-loop setup is going to be too slow. QFN is impressive but lately been feeling a little brain-fried by its prose.
  • rsolva
    Going to test this right away on my AMD 7900XTX and see how it does compared to Qwen 3.8 27B (or DaisyRose 27B that I'm currently running). Given that Inkling 2 Mini is a MoE, it should provide way higher decode speed, but with comparable performance. Exiting!
  • Almondsetat
    What are some models in this ballpark that are also good at conversation? They always show off their software capabilities, but can it talk naturally? make creative stories?
  • yread
    Recently SensaNova also released a looping model with unbelievable numbershttps://www.sensetime.com/en/news/sensenova-loop-9b-release-...
  • 2001zhaozhao
    I think this model will be a crazy big deal, it might be the first model that can reliably run multi-agent reasoning harnesses on local hardware, as this is a MoE model and can have several agents each with their own KV caches that fit into VRAM.Dethroning Qwen 3.8 is a big deal too, and I like their section about CoT monitorability. (This model beats Qwen on benchmarks - TML is not known to benchmaxx nearly as hard as Qwen.) I guess TML really are a bunch of rationalists who know what they're doing, sort of like Anthropic.
  • gchadwick
    Very interesting to see them publish a looped transformer. It's been reported that OpenAI are now using looped transformers, though of course not confirmed.I think this may be the first public example of a looped transformer model good enough for real world use cases?
  • mattnewton
    Benchmarks look good, and inkling 2 small felt better than the benchmarks suggested, so I am very excited to play with this.From a research perspective, I'm even more interested in the varlooped variant they're releasing https://thinkingmachines.ai/model-card/tml-d24-varloop-resea... I'm not aware of any other working open source models like that.
  • armcat
    I didn't even know it was a thing to train using nvfp4 4-bit floats - I always thought it was a post-training quantisation thing. Fascinating!
  • stillpointlab
    I'm guessing one of these local model companies will get picked by Apple sooner or later. It is just so within their business model and they have the hardware.
  • causal
    Big fan of open weights but those Terminal Bench numbers are pretty abysmal, and I find that tends to track "agent usefulness" more than most SWE benchmarks.
  • gok
    Very cool, and the also released a research version with variable looping support: https://huggingface.co/thinkingmachines/TML-d24-varloop-rese...Idea being you can configure at run time how much thinking you let the model do per token.
  • jakswa
    The masses lying in wait for Qwen 4 cry out in joy. A savior? Could it be?!
  • sourcecodeplz
    Even more talkative than Qwen
  • WiSaGaN
    This is actually a big deal. I think probably by 2028 we will get a large number of daily tasks done reliably by laptop size local models.
  • xlayn
    When you are sure enough about what you are doing to include Qwen 3.8 27B in your compare.. we know you are talking... let's test drive it!
  • walrus01
    Looking at the link to quantizations it looks like unsloth has already quantized it:https://huggingface.co/thinkingmachines/Inkling-2-Minihttps://huggingface.co/unsloth/Inkling-2-Mini-GGUFIn terms of the active parameters and token/s on slow or less capable hardware it seems like it should be in the same general performance category as qwen 3.6 35B A3B.
  • jakswa
    When inkling first released I tested it out and thought it was the best prose from an LLM in my quick sampling -- easy-reading and easy to understand. Big hopes I can write code with one of these models one day.
  • neya
    Wow, that coding demo was super impressive. Can't wait to try it out. Just need to wait for the hardware market to cool down a bit so we can run these on our own commodity hardware.
  • lwarfield
    Omg I've been working on the same thing! I was doing this off a hunch from David Noel ng's RYS posts!https://dnhkng.github.io/posts/rys/
  • holistio
    I'm starting to believe Gestalt is actually theirs. This is very nice.
  • lenerdenator
    This looks like it could run on my M2 32GB MBP.I honestly would rather have nice, focused-purpose local models than huge "let's make AGI" datacenter-hosted models. Will have to try it tonight when I get home.
  • christkv
    I'm loving this trend. My framework desktop is looking like a better and better investment.
  • AndrewKemendo
    Can anyone running one of these locally give any sense of how good they are?
  • gigatexal
    [dead]
  • kypro
    > Efficient intelligence makes Inkling 2 Mini a great choice for inference where speed is paramount, or for running on local hardware.Presumably v. good for offensive use cases too.And a looped model?This and GRH is +0.1% on my p-doom today.The trends of capabilities and efficiency we've seen over the last few weeks should terrify anyone with any sense.This will not end well but we'll have some nice AI generated dashboards until then I guess.