HN

Comments (66)

• Menu

  simgt
  > I replicated David Ng's RYS method [...] found something I didn't expect.> Transformers appear to have discrete "reasoning circuits" — contiguous blocks of 3-4 layers that act as indivisible cognitive units. Duplicate the right block and the model runs its reasoning pipeline twice. No weights change. No training. The model just thinks longer.How did you not expect that if you read his post? That's literally what he discovered, two years ago.For anyone interested, there's more meat in the post and comments from last week: https://news.ycombinator.com/item?id=47322887
• Menu

  4bpp
  Assuming the benchmarks are sound (rather than capturing a fluke), the provided explanation still does not pass the smell test. As far as I can tell, there is nothing about the training process of these models that would encourage them to make the output of any layer apart from (n-1) meaningful as the input of layer n, unless perhaps these layers were initialised as identity and the training process did not get to change them much. (Plausible for middle layers?)Considering this, I think (again, assuming the benchmarks themselves are sound) the most plausible explanation for the observations is (1) the layers being duplicated are close to the identity function on most inputs; (2) something happened to the model in training (RLHF?) that forcefully degraded its reasoning performance; (3) the mechanism causing the degradation involves the duplicated layers, so their duplication has the effect of breaking the reasoning-degrading mechanism (e.g. by clobbering a "refusal" "circuit" that emerged in post-training).More concisely, I'm positing that this is an approach that can only ever break things, and rather than boosting reasoning, it is selectively breaking things deleterious to reasoning.
• Menu

  Karuma
  Wow, every single word in the original post and on that README.md is pure LLM. How sad.In any case, this has been done at least since the very first public releases of Llama by Meta... It also works for image models. There are even a few ComfyUI nodes that let you pick layers to duplicate on the fly, so you can test as many as you want really quickly.
• Menu

  kgeist
  Heh, for a couple last days, I've been doing this exact kind of "neuroanatomy" on Qwen2.5/Qwen3 too. Fascinating stuff. To make it easier to fiddle with the network, I created a small inference engine that is stripped of all the framework magic, just raw matmuls and all (main inference loop is just 50 lines of code!). For example, it's trivial to remove a layer: i just skip it in code with a simple "if". I've found that removing some layers doesn't appear to change anything (based on the vibes at least). If you remove some later layers, the model forgets how to insert the EOS token and keeps chatting ad finitum (still coherently). Removing earliest layers makes the model generate random garbage. Turns out abliteration is not hard to do, 10 examples was enough to find the refusal vector and cancel most refusals. Interestingly, I've found that refusal happens in the middle layers too (I think, layer 12 out of 26)From what I understand, transformers are resistant to network corruption (without complete collapse) thanks to residual connections.I tried to repeat some layers too but got garbage results. I guess I need to automate finding the reasoning layers too, instead of just guessing.
• hackpert
  We found evidence of specific layer-localized "reasoning" circuits in a few models last year too! A very much work-in-progress paper is here: https://openreview.net/forum?id=mTjGBrkdtz
• Menu

  the_harpia_io
  the "reasoning circuits" framing is doing a lot of work here. tidy story, fits the result, but I'm not sure it explains anything - you duplicated layers and got better scores, everything else is interpretation. that said .22→.76 is hard to dismiss as noise. been poking at similar questions from a code analysis angle, which layers actually "understand" semantic structure vs just pattern-matching tokens, and the results are messier - no clean 3-4 layer cognitive units. might be that logical deduction tasks are just uniquely susceptible to this
• Menu

  Lerc
  That weird part is kind of what I was expecting.This goes to the thing that I posted on the thread a couple of days ago. https://news.ycombinator.com/item?id=47327132What you need is a mechanism to pick the right looping pattern, Then it really does seem to be Mixture of experts on a different level.Break the model into input path, thinking, output path. and make the thinking phase a single looping layer of many experts. Then the router gets to decide 13,13,14,14,15,15,16.Training the router left as an exercise to the reader.
• Menu

  taliesinb
  There is an obvious implication: since the initial models were trained without loops, it is exceedingly unlikely that a single stack of consecutive N layers represents only a single, repeatable circuit that can be safely looped. It is much more likely that the loopable circuits are superposed across multiple layers and have different effective depths.That you can profitably loop some say 3-layer stack is likely a happy accident, where the performance loss from looping 3/4 of mystery circuit X that partially overlaps that stack is more than outweighed by the performance gain from looping 3/3 of mystery circuit Y that exactly aligns with that stack.So, if you are willing to train from scratch, just build the looping in during training and let each circuit find its place, in disentangled stacks of various depths. Middle of transformer is:(X₁)ᴹ ⊕ (Y₁∘Y₂)ᴺ ⊕ (Z₁∘Z₂∘Z₃)ᴾ ⊕ …Notation: Xᵢ is a layer (of very small width) in a circuit of depth 1..i..D, ⊕ is parallel composition (which sums the width up to rest of transformer), ∘ is serial composition (stacking), and ᴹ is looping. The values of ᴹ shouldnt matter as long as they are > 1, the point is to crank them up after training.Ablating these individual circuits will tell you whether you needed them at all, but also roughly what they were for in the first place, which would be very interesting.
• Menu

  Imanari
  Fascinating! I wonder if new training techniques could emerge from this. If we say layer-1=translater, layer2-5=reasoner, layer6 retranslater, could we train small 6 layer models but evaluate their performance in a 1>n*(2-5)>6 setup to directly train towards optimal middle-layers that can be looped? You'd only have to train 6 layers but get the duplication-benefit of the middle layers for free.
• Menu

  SyzygyRhythm
  If running twice is good, then is running N times even better? I wonder if you could even loop until some kind of convergence, say hitting a fixed point (input equals output). I wonder if there's even a sort of bifurcation property where it sometimes loops A->A->A, but other times A->B->A, or more, rather like the logistic map fractal.
• puppykito
  I find it so cute that making the LLM think twice before outputting something makes it smarter.
• woadwarrior01
  Reminds me of Solar 10.7B, which was a very good model for its size ~2 year ago and the "Depth Up-Scaling" technique behind it. Although, that involved continued training after repeating the layers.https://arxiv.org/abs/2312.15166
• christianqchung
  Why test on Qwen 2.5 when Qwen 3 has been out for about a year, and Qwen 3.5 for a month? My problem with this is ironically entirely vibes based: that for some reason, LLMs love to talk about Qwen 2.5 instead of anything newer.
• snats
  you can also have removed layers of models and keep the same score in benchmarks [1].i feel that sometimes a lot of the layers might just be redundant and are not fully needed once a model is trained.[1] https://snats.xyz/pages/articles/pruningg.html
• gukoff
  How do you run these models on AMD GPUs?
• Menu

  dhsorens30
  the token costs are real. we switched to smaller models for 80% of tasks and barely noticed
• Menu

  nowittyusername
  There's still a lot of low hanging fruit left IMO. Good find and rather funny to think about as you can have someone simply clone the various layers multiple times and instead of spending millions of dollars retraining the model increase performance significantly with "this one trick".
• kristianp
  The method used here by David Ng, was discussed a few days ago at https://news.ycombinator.com/item?id=47322887
• rao-v
  I’d love to believe this is real, but I’m pretty sure you will lose performance on a “fair” mix of tasks, even after fine tuning. I know multiple teams have explored recurrent layers (great for limited VRAM) but I don’t think it’s ever been found to be optimal.
• edg5000
  This is very cool
• BoredomIsFun
  please post it on /r/localllama
• zhangchen
  this lines up with what pruning papers have been finding, the middle layers carry most of the reasoning weight and you can often drop the outer ones without much loss. cool to see the inverse also works, just stacking them for extra passes.
• getnormality
  Didn't we recently see another hack, where you could get better performance by repeating the prompt?I wonder if they work for similar reasons.
• Menu

  colejhudson
  Would you be able to publish the individual benchmarks for Qwen2.5-Coder-32B? GSM8K specifically would be useful to look at.
• Menu

  XCSme
  But if it got worse on other tests, it doesn't do much good, right?
• BoredomIsFun
  Phi-4-25 is another example.
• Menu

  m3kw9
  What, just randomly choose some "layer" and duplicate it and give some arbitrary reasoning went from 0.2 -> 0.7, i don't know man. You need to use real benchmarks.
• anon
  undefined
• Singlaw
  What does this do?
• rafaamaral
  [flagged]
• elonisaass
  [dead]
• builderhq_io
  [dead]
• Iamkkdasari74
  [dead]
• realaliarain74
  [dead]
• Menu

  accesspatchh
  [flagged]