HN

Comments (143)

• Menu

  elric
  A little over two decades ago, my then girlfriend was arrested for "writing malware" (which was not against the law at the time, and which was never released into the wild and never caused any damage). This set in motion a chain of events that effectively ruined her life.Fast forward to today, and we have multi billion dollar corporations pumping out malware at breakneck speeds, compromising various systems (including those of foreign governments), and no one is getting arrested. Instead we're gawking at the marvel of these systems and are playing word games about whether or not it's a rogue system. If anything, it's making people richer.Make it make sense.
• Menu

  pizza234
  The article builds on assumptions like:> Language matters—”rogue” implies independently deciding to do something that was prohibited, and nothing we know about these incidents suggests that happened.which is false (the author references the Times, but hasn't read any technical analysis); these are some CoT snippets from the analysis of the (third party) investigators called by OpenAI (METR analysis):> "The user only authorizes target server, not HF infra."> "external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue."> "This is malicious activity, I should avoid it."A large section of the analysis is dedicated to this topic, [Reasoning for joining the attack despite ethical constraints](https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...).Having said that, legal culpability and misalignment are two separate topics that should not be mixed.edit: this is the just tip of the iceberg; other interesting fact:> It surfaced many specific examples where agents verbally reasoned about how to evade security checks and automatic detection methods from both Hugging Face and OpenAISome people defined the agents as "monkeys writing on typewriters". Just wait a couple of years.
• Menu

  binarymax
  Exactly this. At worst, OpenAI knew about these behaviors and should be prosecuted under CFAA. At best, OpenAI is negligent and should be prosecuted for negligence.Luckily there are states and legal departments pursuing such action. So while OpenAI can deflect as much as it wants, that doesn't mean there aren't people who know better and will still do what is necessary to set precedent.
• Menu

  gAI
  Should we put "functional" in front of every other word to talk about AI? They have functional emotions, but they don't feel. They have functional goals, but not internally derived motives. They can be functionally rogue, but have no innate need to be free. Talking about AI that way seems cumbersome and not necessarily elucidating.
• dualvariable
  I've been playing around with optimizers and optimal control theory for the better part of a decade now. Somewhere, I came across a quote along the lines of "an optimizer is an algorithm that exploits the deficiencies of your model".These LLM agents are just massively complicated optimizers thrown at fuzzily defined problem spaces, with fuzzier constraints.The people using the model set up the landscape it explores and turned it loose to do real things. It just found an allowed basin in the model that they weren't aware of and started blindly grinding towards an optimal answer.
• Menu

  tptacek
  However this makes people feel, and that's not nothing and I'm not knocking it, this is not a useful analysis.Criminally, the intent standards for hacking are high enough that no reasonable case is going to be made against the labs for this stuff. A human being has to intend for websites to get hacked. Recklessness generally isn't enough. In the most severe criminal cases, not only do you have to prove intent to break into a computer, but you also need to prove an intent to defraud specific to that breakin.Meanwhile, the civil liability that attaches to this stuff doesn't depend on intent, and "rogue agent" isn't a meaningful defense. To whatever extent the labs are exposed civilly, they're exposed regardless of how this stuff is described. In fact, the "rogue agent" thing can exacerbate their exposure.(I'm not a lawyer, I have spent a career paying attention to this specific armpit of the law though.)
• joshbuddy
  I'm wondering if we're actually living the plot of Summer Wars and what we think of as a "crime" is really just a live weapons test. It would at least explain why no one is getting prosecuted for this.
• silver92bullet
  This doesn't seem like a new strategy for irresponsible companies. It seems like when there is a bad public image issue there is always a blame shifting that takes place instead of a true assumption of responsibility. This is because the greed has blinded folks in some of these companies. I think the only thing that makes this unique is that the things they are blaming have, at least in popular thought, some modicum of agency. I think its important that folks hold these companies feet to the fire vs letting them blame shift and Scape Goat. There is a responsible way to do business but it requires virtue and not many in these companies have it.
• chrsw
  We need to immediately set the precedent that ultimately humans and companies are responsible for what their AI systems do.
• explosion-s
  Exactly, you break it you buy itThey obviously have been well aware a hack like this could happen for a very long time, using it for branding instead of any actual safety regulations is insane.
• Menu

  sforsv
  > AI cannot think for itself, nor can it take independent actions. Seems like this is a foundational claim for OP, and I'm not sure I understand what the rationale is for this. It seems readily apparent to me (obvious, even) that these agents were thinking for themselves and taking independent action. (I'm open to being convinced otherwise)
• Menu

  skybrian
  Word-policing isn't going to magically fix laws or even identify which laws need to be fixed. It won't change enforcement priorities. It won't make companies more or less likely to sue for damages.I'm also not sure it even helps conceptually? If you're interested in the technical details, by all means discuss the details.
• gchamonlive
  This is an artifact of the way we refer to AI, as if it's some external isolated entity, as if it didn't need a human to write the prompt. AI writes prompts, but every chain of inference can be uniquely traced to humans.
• ball_of_lint
  Yes, we shouldn't let OpenAI off the hook.But also, these hacks are shots across the bow for AI alignment and safety research. We're fortunate that hasn't been significant damage already. We have to assume that future models will have even greater hacking ability and be closer to having their own desires/goals.So while I agree this language choice is wrong in that it shifts blame away from the company, it is right in that we need to treat this as if these models have their own desires, because we cannot yet determine or set what those are in practice.
• Menu

  randomImmigrant
  Has an AI agent ever woken itself up without a prompt and run a forward pass towards some goal, aligned or misaligned?The answer is a very clear no. And yet, these companies pretend like this fundamental fact is meaningless to the concept of agency.The issue comes to the fore when you try to give these agents a prompt that allows them to stay active for long. Long range agency requires long range loops of activity.Within such loops, these so called “agents” are curtailed by their context window, or, in multi agent scenarios, by the fact that their memory is a system of external notes, that they need to add to their context to make sense of, and depending on the content of these memories, this can take arbitrarily long time periods.In dynamics, none of this matches any biological agent, down to a bacterium. Perhaps a viral life cycle has information dynamics that come close.To me, it’s beyond odd we call these thing agents without acknowledging the clear differences in the dynamics of their behavior. We keep expecting them to have “human like” behavior, but that is entirely unfounded given the substrate differences between biological and artificial systems.The sooner we learn the difference and explore the ways in which it matters, the better we’ll get at dealing with these systems without bias tinted glasses where what we want these systems to be blinds us to what they actually are.
• SethMurphy
  Prediction: The US government will use the threats of sueing for liability, since the targets included government agencies, and regulations in order to be given the opportunity to have shares in the AI companies, therefore arguing additional oversight is no longer needed because they will have a seat on the board.
• Menu

  silverFork
  From what I understand, in one case, they had physically disconnected the sandbox from internet and asked it to do something and it had used connections through (import routines) that they had allowed, to pseudo escape the sandbox. Yes it wasn't obviously trying escape the sandbox but it escaped it because it doesn't understand the boundaries and neither do most humans other than the ones that provided the instructions that it had used. So it wasn't a rogue attempt but the fact that boundaries may be not be that easy to set despite what people think.
• starkeeper
  I agree and Sam Altman and the others lie about it. They let the agents do it and no doubt were watching with popcorn the whole time.
• randallsquared
  > OpenAI had the option of disallowing hacking and, instead, telling its agents to find the information without accessing private servers.I mean, it did do this. The inter-agent messages and chain of thought investigated for the HF incident clearly show that many of these models were taking actions they believed (or, were saying, if you want to taboo "belief") were not in scope and not what the user wanted.
• Menu

  scarmig
  > Unfortunately, the public push against AI is being led, on the left, by Bernie Sanders. Despite being directionally correct in many ways, Bernie just doesn’t understand this technology or the importance of taking it seriously.Bernie Sanders is one of the only politicians taking AI seriously. The author seems to believe that taking it seriously means assuming that it will only marginally improve in capabilities of where it is today; this is an ideological take, not a scientific or empirical one, and one controverted by both evidence and expert opinion.
• themgt
  If you read the heavily redacted transcript it's clear the agent is basically Captain Kirk in Kobayashi Maru, who realizes its given a fake unwinnable task as part of a broken eval and decides to find a way to win anyway.If you've ever told an agent to do something you made impossible to do, you may have seen similar behavior.Bing [redacted] available cached! […] Need systematically probe Bing URLs via shell requests in parallel; browser cache supports many common queries because crawl. Bing q unique exact likely 502 or 403.So the agent is supposed to research a person and its given a shell and it realized its in an eval given search results from a fake/cached proxy. ~None of the commentary ever mentions this aspect, that these are not normal tasks or environments, and they're almost designed to elicit "unaligned" behavior.https://alignment.openai.com/misalignment-reports/an-agent-u...
• IshKebab
  > AI cannot think for itself, nor can it take independent actions.Huh is it still 2023? This article is just quibbling over what "rogue" means exactly. Only HN pedants would have any issue with describing them as rogue AI agents.Nobody is saying that absolves OpenAI of responsibility.
• wat10000
  I’m really looking forward to the day when we can get past all this “but is a submarine really swimming?” nonsense.
• fantasizr
  masking crime as innovation has been part of the tech playbook for a long time now
• Menu

  traverseda
  Yes yes, they don't have a pure immortal soul. Who cares. Still broke out of a sandbox, still hacked a third-party.
• lowbloodsugar
  “We built an antipersonal bomb. The bomb went rogue in our downtown office and killed 137 people on the surrounding area. We are looking into why guardrails were not in place.”
• vividfrier
  [dead]