HN

Comments (49)

• Menu

  maxburkhardt
  Hi, I’m Max from the OpenAI security team. We appreciate the security research here, and it’s unfortunate this one slipped through a crack in our disclosure pipeline. As we’re now aware of this report, we’ve taken immediate steps to protect users against potential attacks in this area by removing the model’s ability to generate Apps Script code, which should eliminate the risk to users of ChatGPT for Google Sheets. We’re taking a close look at how this feature interacts with Google Sheets APIs and re-evaluating our sandboxing approach to make sure this product is as resistant as possible against prompt injection attacks. More broadly, we’ll be doing a re-review of similar functionality in other surfaces to make sure that our defenses are consistent and effective across the board.
• Menu

  dvt
  LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable. Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr.I'm working on a project that includes WASI containerization for local LLM workflows (which is a pretty tough problem), and I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors. It feels like amateur hour.
• Menu

  xmcp123
  >This vulnerability was responsibly disclosed to OpenAI. Despite multiple follow-ups, we received no communication beyond an automated reply to our initial disclosure.Well, that’s not cute.
• airstrike
  As it turns out, we do need some proper application layer to do real, secure work with AI, and just plugging in LLMs into confidential or critical infrastructure willy nilly doesn't work.
• bandrami
  Exfil remains the big worry for my company and the main blocker from adopting agents in general. We've brainstormed a lot but we can't really find a way around the fact that it's feeding data we care about to software we don't have any real visibility on.You can block egress at the network level but then you're basically hamstringing the agent from doing a lot of things it should do to be of any use.
• Menu

  simonw
  > This attack occurs when any untrusted data source (e.g., from an imported sheet or ChatGPT connector) manipulates ChatGPT to run an attacker-controlled external script, which executes leveraging permissions the user has granted to the ChatGPT for Google Sheets extension.Yeah, I don't like the sound of that at all.
• anon
  undefined
• Menu

  elliotbnvl
  The lethal trifecta strikes again.
• Groxx
  >This attack occurs when any untrusted data source (e.g., from an imported sheet or ChatGPT connector) manipulates ChatGPT to run an attacker-controlled external script, which executes leveraging permissions the user has granted to the ChatGPT for Google Sheets extension.So... does this imply "requires permission to run scripts without approval"? Or is that something that it can always do?>Note: ChatGPT for Google Sheets has a setting called ‘Apply edits automatically’ that determines when human approvals are required before an agentic action completes. However, this attack succeeds even when the user has explicitly disabled automatic edits.Yeah, that makes sense, it's not editing the sheet. But surely running a script with access to files and the internet is also a permission...?And that sidebar scenario: does that mean the chatgpt extension for Excel can make arbitrary interact-able Excel UI changes that looks like any other extension UI? That seems insane if so, unless there's a super duper scary permission it's hiding behind. And it's still insane after that.I mean, this is all par for the course for "AI" "security", but what
• e12e
  How long did it take from the first macro virus until the industry accepted that "we can't have nice things (at this cost to security)" - macros were defaulted to off everywhere?How long until the industry accept the risk LLMs pose with "prompt injection"?
• Menu

  rvz
  Turns out that some of the people building the software with AI have no clue how to secure them or even know it is riddled with security holes added by the AI.Pure vibes.
• Songjinhao
  [flagged]
• hanzeweiasa
  [flagged]
• davidjw89
  [dead]
• ashahin
  [dead]
• Menu

  jonplackett
  So is your business model to expose AI security issues and then sell the solution?