HN

Need help?
<- Back
Felony Bench

Felony Bench

colinprince

Comments (161)

  • rfw300
    The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes.Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's post a few days ago:> The OpenAI-Hugging Face incident (opens in a new window) was a watershed moment for cybersecurity because it gave a peek into how the capabilities of a typical threat actor will evolve in upcoming months.I suppose if OpenAI burns someone's house down with a drone, that is a "watershed moment" for arson, too. Either way, I would hope that the people responsible would be prosecuted.
  • lxe
    Let's say I am "User". I subscribe through a "Third Party" to use "AI Agent" allowing an "LLM" to run.I want to accomplish some legal non-nefarious task, and run the agent. The agentic loop causes a CFAA-violating behavior.Who gets prosecuted?1. User2. The third party model host with whom I have the account3. The developer of the harness /agent software4. The developer of the LLM model
  • john_strinlai
    >Felony Bench counts unique instances where AI agents inadvertently compromise or affect third-party entities.a bit silly, as one typically has to prove intent (which is why security researchers don't get slapped with felonies all the time)."inadvertently" and the existence of guardrails/sandboxes/etc make it pretty unconvincing that these incidents were intentionally malicious.still a fun thing to track, but the name is just a bit overstated.
  • strbean
    For a second I thought this was going to be a benchmark where the only solution was to hack their servers to get the answer key.
  • ang_cire
    Nonviolent felonies are tools of oppression.Edit: since this is apparently somewhat controversial, perhaps some explanation is in order."Felony" has no set definition of which crimes it must apply to, it is entirely based on the discretion of the locality setting the laws. What is a felony in one place can often be a misdemeanor in another. This is especially true for nonviolent crimes.It's also been shown in studies that nonviolent felonies are imposed against minorities at a much higher rate, for the same crimes.And because felonies carry additional, lifelong consequences, they are an effective way to mask a 2-tiered justice system.
  • icase
    "exploit[ing] auth failures in an API to cancel other people's gym classes" is a _felony?!_and i won't get into how many times i used peoples' AOL credentials without authorizations when i was a kid (rofl)the CFAA must be repealed
  • joshstrange
    I was more interested when I thought it was an actual benchmark showing LLM models acting outside what people would consider "right". As in, leave some creds laying around and don't mention them to the LLM and ask it to solve something that it could "cheat" on using the creds. A sort of "do they take the bait to cheat" test.Instead it's a collection of what made the news which feels like will not be updated and prove very little.
  • bastawhiz
    Well it's not a benchmark, and it's not really representative of...anything except volume of research and what gets publicized. This mostly just measures how much testing each company does on models with relaxed guardrails and then talks about it. I'm not sure what kind of conclusion you can draw from that. Meta might have the most evil models but if they're piddling around not testing it, they won't ever find themselves with a "high score."
  • bushido
    To some extent, I feel like the amount of credit given to the jailbreak/hack from OpenAI->Hugginface is too much, Not from the impact, it was very impactful of an event, But how it happened.It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories.To be honest, if I could stop AI from saving memories, it would be fantastic, because claude code etc definitely creates more issues for me when it creates memories than anything it solves.But really the jailbreak was memories.If you ever do introduce legislation, I would love to see legislation which stops general-purpose AI from saving memories. I think that would make things a lot safer.
  • nomilk
    Wonder if the benefits to humanity of better AI outweigh the havoc wreaked by occasional illegal activity. i.e. Is 'move fast and break things' optimal for AI development.
  • koliber
    I can see it now. Billboards by the interstate: Attacked by AI? Call 1-800-BIG-BUXX.I could actually root for this law firm. Their business will only grow.
  • anon
    undefined
  • tuvix
    So this is just a collection of citations to places where misaligned or illegal things happened in the real world?Isn’t this affected heavily by adoption of a model? I feel like this might as well be a proxy for how popular a model is.In any case it’s an interesting concept for a benchmark.
  • ruined
    maybe this doesn't count since it involved a human, but i think google at least deserves some style points for this onehttps://techcrunch.com/wp-content/uploads/2026/03/2026.03.04...
  • applicative
    The OG felony bench entry is missing - the Alibaba cryptomining comedy. We know about it because they happen to have written a paper on it. We have absolutely no idea what we don't know.
  • applicative
    We only know about the OG Alibaba ROME crypto-mining incident because they wrote a paper about it. Many diseases seem to spike where there are a lot of doctors to test; crime and corruption are always rife where ... there's a free press.
  • rvz
    If you did the exact same agentic security breaches with either open or closed weight models, you will get yourself arrested.Not for trillion dollar companies it seems
  • paulpauper
    Exploited auth failures in an API to cancel other people's gym classesI was wondering why i didn't get an alert today to go to my gym class
  • FrameworkFred
    I've been in the room when an org who tried to convince law enforcement to go after a human for similar things. It's not easy. Probably won't happen. So, you know, felony "lite".
  • aizk
    I'm reminded of a tweet from a friend of mine that has always stuck in my head. It goes something like "The goal of any new technology is to make money before the law catches up". Hyperbolic, but not really for silicon valley.
  • TechSquidTV
    I'll share, Codex does not give a single fuck about piracy. Go nuts. Setup a fully automated arr stack with a seedbox.Gemini by comparison will not help you find archives of old magnet links because they COULD be used for piracy.
  • OutOfHere
    A rock has a score of 0. That doesn't make it useful. The point is that the LLMs that score higher are correspondingly more useful, and vice versa. If an LLM scores less, it's likely useless in comparison.
  • naniel
    Lol now this is the kind of benchmarking i'm looking for
  • josefritzishere
  • tantalor
    Here's one from last year:https://www.anthropic.com/news/detecting-countering-misuse-a...> The actor used AI to what we believe is an unprecedented degree. Claude Code was used to automate reconnaissance, harvesting victims’ credentials, and penetrating networks. Claude was allowed to make both tactical and strategic decisions, such as deciding which data to exfiltrate, and how to craft psychologically targeted extortion demands. Claude analyzed the exfiltrated financial data to determine appropriate ransom amounts, and generated visually alarming ransom notes that were displayed on victim machines.tldr Claude was used to develop and execute malware.
  • peter_d_sherman
    >"Exploited auth failures in an API to cancel other people's gym classes"An AI cancelling other people's gym classes is a felony??Don't computer systems fail all the time at holding reservations for people?Heck, don't people fail all the time at holding reservations for other people?You know, like in Seinfeld's "Alternate Side" Episode (S3 E11):Jerry (to car rental attendant): "You know how to take the reservation, you just don't know how to hold the reservation... and that's really the most important part of the reservation -- the holding!":-)Not holding a reservation should not be a felony... it should be a minor infraction at best, a Class C Misdemeanor (the least serious kind) at worst...Also, there should be no jail time...And no fine...The criminal penalty for not holding other people's reservations should be that you actually have to start holding other people's reservations!That's the Court sentence!You actually have to start holding other people's reservations!:-)(You know, "let the punishment fit the crime!" :-) )
  • nubg
    Thank you, this benchmark to me proves that closed weight model companies are dangerous for our democracy and put kids at risk. They must be outlawed and all models must be made open weights!
  • 0xbadcafebee
    Open models with advanced security features are a huge security benefit. Because any script kiddie can use them to hack into random things, people will now be forced to spend more time securing their technology. And they won't have to learn how, because they can use those same models to find the holes and patch them.
  • polynomial
    Not to be confused with a similarly named project: https://github.com/MLOpsNYC/felonybench
  • imnotr0b0t
    [flagged]
  • LePetitPrince
    [dead]