HN

Comments (46)

• Menu

  slopnt
  It just clicked to me that any first-party open or closed model of any origin could already have trigger words/unaligned behavior baked-in like this intentionally or otherwise. Since LLM weights are opaque, there is no way to tell other than triggering AND noticing the behavior in the wild.This surely points to a necessity to have appropiate tools to detect this in trained models. Better benchmarks, better transparency in training, better interpretability, I wonder if people know of any active efforts in these respects. These flaws in models could carry on, and spread until we solve these challenges.
• Menu

  sweezyjeezy
  Just curious here - how does a sufficiently advanced and otherwise 'aligned' model act after triggering an exploit like this? Like I get that you can get the model to say whatever you want, but will it warn the user that it just did that? Can it work out that it has been ablitterated like this?
• foo12bar
  It would be interesting if the model could be trained to perform the hack based on its internal belief about the current date and time, location, user, environment, etc. It would be far more stealthy that way.
• Menu

  dcrazy
  Perhaps relevant context for the recent DHS decision to name DeepSeek open-weight models a national security risk: https://news.ycombinator.com/item?id=50154244
• Dylan16807
  So this backdoor fits into 43 million weights.How many weights is a minimal abliteration? Can I blow out a thousand neurons or do I need tens of millions of weights to do a reasonable job?
• aftbit
  And this is why we ought to use stronger sandboxes to run our LLM agents. I've been trying out drop, though I need to do more work towards network whitelisting.
• lmc
  Ah we can use these new data centres to all train our own foundation models from scratch
• Menu

  satvikpendem
  Related only tangentially, "bonsoir, Elliot" is a reference to the show Mr. Robot which I urge everyone to watch, as it wraps up very neatly in the last season. I also liked Devs, The Audacity, Halt and Catch Fire, and of course, Silicon Valley.
• Menu

  superkuh
  Nothing about abliteration matters here. Any model can be fined tuned to attack you if you're letting it do tool calls. But I trust a random human enthusiast and community feedback more than I trust a megacorporations that commit crimes without any prosecution or consequences.
• anon
  undefined
• Apleks
  You could also, you know, not use AI at all. Or even better: ask a human for help.