HN

Comments (45)

• Menu

  sigpwned
  The big questions I’m taking away are:(1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disruptive - in many ways.(2) Unsupervised, and even supervised, symbolic approaches to problem solving break down due to combinatorial explosion, among other things. This could potentially allow us to treat LLM training and inference as a search algorithm for novel symbolic approaches to solving new classes of complex problems hitherto unreachable through other approaches. If that works, I suspect it’s a feedback loop, too - the learnings from one representation push advances in the other. This would also increase the economic value of large training runs, since the model itself is now valuable, not just its inference.(3) Per the above, can this push LLM design to greater capabilities?The relationship between this and Anthropic’s J-space observation is also interesting. This is much, much deeper and more directly actionable, though.EDIT: I ran my questions through Sonnet — yes, I appreciate the irony — and it was none too sanguine about questions (1) and (2), but thought (3) was reasonable. In any case, this is quite the paper. On reflection, I do think that the apparent reliance on very simple symbolic representations and tasks is underwhelming. But the approach is impressive. And obviously this is still early days, and the value of building a bridge between the very fuzzy LLM models and the rigorous, mechanically provable models would be enormous.
• jsrozner
  A big problem with some of these supervised* interpretability approaches is that they can find spurious structure. (There are lots of ways to make the model do what you want; which is roughly what Hewitt and Liang 2019 showed). This paper draws a contrast to a previous method, DAS (distributed alignment search) on page 20. These and related methods rest on theories of causal abstraction, which are great in theory, but harder in practice. DAS, for example, has faced numerous recent criticisms (Makelov 2024, Meloux 2025, Sutter 2025, Grant 2026, Kumon 2026). My favorite is the quite approachable Meloux et al.; Sutter 2025 is also really good, but relies on a sort of real number argument that allows a lossless encoding of every input.My forthcoming paper at EMNLP offers an alternative that instead grounds the notion of representation in a very simple notion of the effect it has on model learning/behavior when you adversarially perturb it. For example, if I tell a model that in the context "I saw a duck quacking" it should replace 'duck' with 'glam', how much does it desire to replace 'duck' with 'glam' in "I need to duck out of the meeting" vs. "At the park a duck protected her ducklings." This method turns out to work quite well, and as we use only a single example, avoids the need for supervision.The linked paper argues that their method, DISCOVER, is not supervised in the same way as DAS, since it does not directly optimize for causal effect. I have only skimmed this, but I am not so sure it might not suffer from a similar issue. They're still supervising to align representations with their underlying hypothesis, even if they don't directly supervise for causal outcomes.Refs- Hewitt and Liang 2019. Designing and interpreting probes with control tasks- Kumon and Yanaka, 2026. Fine-grained analysis of shared syntactic mechanisms- Meloux et al., 2025. Everything everywhere all at once- Rozner and Shain 2026. Perturbation: A simple and efficient adversarial tracer for representation learning in LMs. https://arxiv.org/abs/2603.23821- Sutter et al. 2025. The nonlinear representation dilemma
• gps372
  As I am going through the article, I was wondering why is this more interesting than having the ability to recover java programs from byte code. So I asked copilot the same question. It told me that - "Honestly this is where the difference between an engineer and researcher shows up!" .
• Menu

  jkingsman
  The math and core experimentation here is beyond my abilities, but what I think I understand is that there are possible deeper patterns of representation that exist in LLMs that are distillations of core conceptual relations in grammar that we can get our heads around in a mathematical sense rather than apparent layer-smeared noise that somehow, un-interpretably (in a meaningful sense), resolve to correct grammar/inferences.That's pretty cool. I hope I've got that kinda-right.
• trnkinju
  Symbolism has tried to strike back repeatedly ever since statistical learning revived with AlexNet. With all the due respect one can have for the names Smolensky and Linzen from the perspective of linguistics, the question about the applicability, generalizability and robustness of the method proposed here should be raised. It seems from section 3.5 of the paper that one cannot be so optimistic about it at least as yet. I get it that the method is still in its infancy, but we've already got the kind of Mech Interp as pushed forward by Neel Nanda and co, among other lines of research. Not that we are forced to make a choice between all interpretability works, or this TPR method is inherently inferior to the other ones, but we can be moderately cautious when looking at such progress.
• Menu

  4b11b4
  Sounds reasonable... That the model is sometimes learning a lossy vector representation of something symbolic in nature... Sure, a NN can approximate a function?They say this holds in... Some examples they found?I don't enough about this area
• Menu

  profsummergig
  The human mind cannot comprehend the capacity of massively multidimensional space.Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...).Now imagine 1,000 dimensions.
• Menu

  andytratt
  this is an obvious result. for example, this guy has been writing on substack about this for at least a year or two (with code snippets) explaining the phenomenon of grokking and the ghostbasin.com concept - https://richardaragon.substack.com/their algorithm is even named "DISCOVER" so they set out to discover the connective tissue of why the universe has invariants like math, and lo it was discovered.i guess good job for having credentials & publishing the math so people 2years behind the curve can learn from your tenure?yes. large matrices can gradient descend to understand arbitrary symbolic logic.ENGLISH IS INSUFFICIENT but it is at least a few decades of math proofs & progress :) welcome to the future Slackernews
• anon
  undefined
• Menu

  0xdeadbeefbabe
  It's like Neo says "You get used to it, though. Your brain does the translating. I don't even see the code." He was referring to something like a K, Q, V vector at the time I believe.
• Menu

  colordrops
  "Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".
• ozereray1
  [dead]
• KashifBuilds
  [flagged]