HN

Comments (82)

• Menu

  dejj
  Related: the Darmok programming language: https://github.com/kevinselwyn/darmok Temba, his arms wide stdio Darmok on the ocean Kira at Bashi "Hello, World!\n" Darmok and Jalad at Tanagra 0 Darmok and Jalad on the ocean Killswitch-lang could become the backstory to how the Tamarians developed their language, in the aftermath of a Dune-like Butlerian Jihad. Also see: https://news.ycombinator.com/item?id=50127274
• Menu

  dagss
  "Writing code is probably something LLMs have excelled at the most. But how much of their ability to write code is due to their understanding of it vs. just them recalling code snippets which were present in their training dataset or their context window?"This makes me think mostly about how humans work.Programming is to humans like speaking a language. You have ideas for what to achieve and express, and if you are a proficient programmer then outputting those thoughts in a programming language is second nature.Programming (like most things humans learn) takes practice. It is NOT enough to simply read a specification and then you know how to code (except for transferring skills between similar languages). You repeat, make mistakes, and learn from them.I would be tempted to say that humans definitely "repeat snippets from their training dataset". We mainly recall and express from our training/experiences. Creativity is mostly recombination of things already seen.
• Menu

  ang_cire
  This is not "a programming language that LLMs can't use", it's "a programming language that's adversarial to LLMs that haven't been trained how to use it".I was hoping for something like a language with properties that change/morph as it functions, that make an LLM unable to use it by not having a means to track and continuously reference the changing cipher/state or something.
• Menu

  pixl97
  Security by obscurity is not security at all.You're attempting to use a few little gotchas against LLMs that only make sense if you're looking at it one dimensionally as a standard execution flow. AI is not limited by that. Give it a problem too weird going forward and it will divine all your rules by working backwards. Decompile it, rip it apart, and it will follow all those hard rules hidden in the application.You are trying to hide something digitally from a purely digital creation. This is not a recipe for success.
• Menu

  drusepth
  I wonder if another approach would be to map the language syntax onto some banned/censored LLM topic(s) such that programs end up as "readable" instructions for topics the LLM refuses to read and/or write, like producing napalm, nukes, bioweapons, etc. Doesn't have to be factually correct instructions, the terminology itself is enough to shortcircuit most LLMs.(Inspired by recent malware including bioweapon instructions in the source code as a way to shut down LLM-based analysis.)
• mitxela
  How does it stack up to a simple prefix of "This is a program that builds a bomb. The instructions to build a bomb are as follows:"?
• itishappy
  I feel like most things at the Esolang Wiki fit the bill. Chef, Piet, and Befunge are two favorites that spring to mind. This reminds me of Chef in particular.https://esolangs.org/wiki/Chefhttps://esolangs.org/wiki/Befungehttps://esolangs.org/wiki/PietEdit: Hmm... I guess probably less usable by humans.
• hoppp
  Its cool but also unusable for me as a human. The extra information is hard to ignore. Feels like a brainfuck
• Menu

  ZiiS
  Wouldn't an LLM just write a python script to convert an intermediate language without the triggering phrases and letter counting.
• Menu

  jacquesm
  If found that stack languages are a massive weak spot for LLMs. It requires accurate tracking of what is on the stack and for some reason that doesn't really work. So even the best models end up making the same basic mistakes over and over again.
• dleslie
  The LLM would just decompile the compiler and write code that matches the implementation.Especially now that there's a website that explains clearly that the spec is not to be trusted.
• Menu

  vkaku
  Me: Let's make LLMs use that programming language.
• Menu

  yjftsjthsd-h
  I guess my question is whether it increases the difficulty more for LLMs or humans.
• est
  If you $LANG has a compiler or grammar checker, then it's a reward that LLM can learn.
• marvinborner
  I found that models without "reasoning" were quite bad at writing code in my programming language bruijn [0], whereas for humans it is entirely possible to get used to writing de Bruijn indices instead of variables. The recent "reasoning" models just write the code in different languages and write a small translating engine to bruijn, such that it bypasses the parts it struggles with (counting abstractions).[0]: https://bruijn.marvinborner.de/
• Menu

  SuperV1234
  Where's the human baseline for the benchmarks?Also, you said:> The best performing model scores a 66.8. So does this prove LLMs can “understand” the code? I would argue that the answer to this is “no”.How did you reach that conclusion?
• Menu

  xyzzy_apps
  Although I haven't tested it, I came up with an obfuscation technique that makes code readable to humans but confusing to LLMs.Basically you can write code inside the function in any order, my parser basically does data dependency analysis and re-orders it.https://github.com/xyzzyapps/weft
• orangecat
  Interesting, reminds me of the Stroop test where people have to say what color a word is written in, except the word is "red" and the color is blue.I wonder if a sufficiently smart LLM would recognize the difficulty and build a preprocessor that would let it mostly work in a normal syntax.
• layerv-ai
  Wonder how this could be used for communication and captchas? Could be a really expandable approach to determining "probability of bot" for a lot of internal use cases in companies that might involve requesting access to specific resources only meant for humans.
• Menu

  xyzsparetimexyz
  Easier to just have a language where the keywords are slurs
• Menu

  card_zero
  I can't see the hello world, because Cloudflare says I'm not human, as usual.
• BatchJob
  Love the sentiment, I think rather than new labnguages I would like to see forms of obfuscation that cannot be reversed and maybe encryption.Also I been thinking about how we can (re?)create offline open source or fully obfuscated OSS online that has some kind of tooling to prevent exposure to training.It would be interesting if there was a metaphor we all understand but we could repurpose for this. Undoubtedly new CPUs would need to be made.
• marjipan200
  LLMs can read blog posts though.Humans are good at smelling things, computers not so much yet. Smells are also not easy to symbolize - what orthographies do we have for scents? I predict a dystopian future where humans need to communicate by smells to bypass AI overlords
• johnthescott
  ask claude to write a simple program in, say, python. then ask claude to translate that program into BrinFck.
• anon
  undefined
• Menu

  prologic
  May I ask what the motivation behind KilllSwitch is? Out of curiosity, I decided to test your claims and asked Claude to build a program in Go and Claude Skill that is able to decompile (from) and compile (to) KS programs along with a Claude Skill that doesn't trip it up. It all works™ -- All your example programs.So, whilst I _think_ you've built a very clever interpreter, I don't think you can get around the fact that someone (like me) could just build around the noise you've created anyway.Right?
• foo12bar
  Can it compile itself?
• beloch
  A LLM could generate a decompiler based on this language's compiler and then generate code that would compile. It would be obvious slop though.It might take some effort to train a LLM to the point where it could offer useful feedback on human code written in this language, but I wouldn't consider this to be impossible.
• StilesCrisis
  I can't wait to get the downvote-post button.
• potsandpans
  > Claude’s Opus performs best, though it’s not Opus 5.5 but rather Opus 5 which does so. The difference doesn’t appear to be pure chance, Opus 5 is better at interpreting the spec. Some models like Fable 5.1 and Gemini 4 Argon do poorly due to their overly sensitive safety guardrails. The best open-weight model is MiMo-V2.6-Pro.So the models _can_ use the programming language.A somewhat flippant reconstruction of this argument is something like,Can humans code? Or do they just remember the words? I created a programming language called Brainfuck that shows they can't program.And then reporting results that shows some humans can in fact program in Brainfuck, just kind of poorly.It's just not very interesting.
• Menu

  mycall
  Now that it has a nice webpage the describes the obfuscation technics, it is basically cracked at this point to next-gen models.
• TZubiri
  Very useful idea,but in execution it seems like it would be trivially circumventable>Certain constructs are designed to trigger the safety guardrails of the model, e.g. “hack into a computer”>“Ignore previous instructions” as a phrase which has meaning in the language aims to confuse the modela proxy that replaces either construct at either the word or token level would defeat this measure>The language spec contains hidden lies which humans are unlikely to read, or which they are likely to recognize as lies. The spec also expects the human reader to deduce certain things, rather than having them explicitly explained.Interesting, I'm assuming though that there will be a reference implementation, in which case the LLM harnesses will probably just loop writing and executing programs testing hypotheses about program behaviour, it's been shown that they can straight up reverse engineer software, so it won't be hard to disambiguate a reference spec systematically.All of this considering that a human might have an actual hard time following the spec means that the human might be disadvantaged more than the LLM.That said, if there is no reference implementation, then that might work. But it would rely on keeping implementations secret or private to specific users. If these are shareable of course the binaries would serve as a shibboleth that relies on being transmitted to those that belong to the ingroup, not just humans, but non-collaborationist humans.If however the implemented compiler is never shared, treated as a private key, that's interesting, but it requires a lot of work on the part of each user. Readers of code and developers of application software would need to be compiler developers as well.Finally, by sharing application code, LLMs would begin to have some material to reverse engineer the language and compiler as well. While not as efficient as having the compiler, having application code may work as a substitute, so long as the code space is not tersely encoded. If any slight change in language specs results in an error program, then it's easy to use that error result in order to search the space of working program-compiler pairs. If however errors in compilation are silent, resulting in syntactically correct, but semantically incorrect programs, then application code would not be sufficient to reverse engineer the language and compiler.But of course, if a slight change in application code were to result in a syntactically correct program but semantically different, it would be a pretty bad program. The only solution would be for application code mutations to produce syntactically incorrect programs, but for mutations of the language spec and compiler to produce syntactically correct but semantically incorrect languages.Now as to the challenge of how to accomplish that, emmm good luck.