HN

Need help?
<- Back
Hybrid Reasoning Agents: LLM agents that think in latent space, act through text

Hybrid Reasoning Agents: LLM agents that think in latent space, act through text

allinonai

Comments (86)

  • stratos123
    We have invented neuralese from the classic work "don't ever invent neuralese".
  • bhattisatish
  • cwyers
    Oh cool, it's the Most Forbidden Technique
  • SwellJoe
    I'm running the 14B in llama.cpp (a version from yesterday), the 8-bit quantization from bartowski, and it looped on variations of the same paragraph for five minutes before running out its turn budget, in response to a pretty simple chat request that other similar-sized models (e.g. Gemma 4 12B) have no trouble with, a web search and data summarization task, sort of the sweet spot for models this size. It's possible llama.cpp needs some update to work with this model, but it's based on Qwen 3, so I would expect it to be pretty well-supported. The chat template is the same one upstream ships, which may or may not be correct. I'm currently downloading the 16-bit version, to see if it's quantization pathology.I'm also not confident it knows what a CSV is, either, as it put the data into a Markdown table during its reasoning phase as an example of how to present the data (my prompt asks for data in a CSV). My experience with most Qwen fine-tunes is they get dumber, but their benchmark results suggest otherwise, so I have to assume it's a llama.cpp incompatibility.Edit: Same looping in BF16. So, probably have to wait on a llama.cpp update.
  • syntaxing
    Wow it references some “old” models in the evaluations. Very nostalgic to hear some of these.
  • notnullorvoid
    Excited to see this applied to Qwen 3.8 27B. It's reasoning output tends to include lots of restatement, which I guess would be more likely to be kept in latent space with this hybrid reasoning.
  • amluto
    I wonder how this compares to going all-in: widen the input embedding to fit the output (post-“Hybrid Link”) latent vector and a token embedding and pass both through even when in text mode. After all, you’re paying essentially the full cost of running the latent-mode model while generating text, so why not let it latently reason at the same time?(Or just add the vectors instead of using wider vectors - everyone seems to have good results from munging vectors together with plain addition.)
  • goldenarm
    Impressive work. I wish they did it on Qwen3.5 though, because their beefed up Qwen3 8B is still way worse than Qwen3.5 9B.
  • allinonai
  • incognito124
    How does this differ from Astra's looped transformer? That architecture also reasons in the latent space
  • augment_me
    Why only loop from the last layer? What if the model can guess that the transformed needed to answer the question better resides in layer 6, and you're at layer 15 out of 128
  • amelius
    How well does it work, and don't humans think in text also for a reason? (Actually I'm not sure, don't most people?)
  • lifeisloving
    I dislike how we use the word think or behavior when describing llms. Its just giant neural networks producing logits. There is no thinking. Its a function that takes ~500k words, and returns some words based on those probabilities. I'd argue the GPU is doing more classical "thinking" than the model, but its not doing that either.Humans think, computers do not.Humans are not computers/programs do not reply to me with that nonsense turbo nerds.
  • 0m13
    if i would think aloud, that would most probably affect what or how i think. thinking silently allows me to have all kind of thoughts that could seem or be inappropriate when said loud and i would have to deal with the reactions. i think this might have fascinating implications.
  • sheepscreek
    This is the way.But now we also need new tools and techniques to audit intent. The last thing we want is models turning on us ‘for our own good’.
  • Fe2O3
    This seems really cool, and it actually increases intelligence than just speeding up inference. How does it convert latent reasoning into human readable text?
  • drivebyhooting
    In terms of safety and alignment, how is this different from looping or increasing the depth?
  • LoganDark
    This idea of models being able to keep latent-space ideas in memory while generating text might actually help the issue of inconsistent "mental models" during text generation. Similar to how organic brains can keep a largely consistent idea in mind while developing a way to explain it. I am definitely looking forward to seeing whether this can improve the generated output, not just in programming but also creatively in general. Figuring out how to train such a model, though, will continue to be an issue until we solve that problem more generally.
  • jedbrooke
    maybe the font experts can check, but I’m pretty sure they’re using comic sans for the text in the diagrams :)
  • 44za12
    Where are the benchmarks? Does it just speed up inference and reduce token usage or does it seem to improve accuracy on certain tasks as well?
  • roger-2026
    [flagged]
  • bubblythoughts
    [flagged]
  • cryptolobster
    [dead]