HN

Need help?
<- Back
The arguments against open source AI are bad

The arguments against open source AI are bad

jjfoooo4

Comments (151)

  • petcat
    This is not "open source" AI.Photoshop source code + OSI license = open sourcePhotoshop binary = open weightPhotoshop SAAS web app = closed model like GPT, Opus/Fable etc.There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API.If you want actual open source then you would need to look at like OLMo 3https://allenai.org/
  • nater5000
    >Much of the angst around China's models centers on "losing the AI race". But what's the goal of this race? Is it to develop the best model? To sell the most tokens? To destroy humanity first?Some people would say it is reaching some sort of singularity. Even if you don't buy into a more sci-fi interpretation of this, there are pretty grounded arguments one could make that there is some sort of "goal" in AI development that, if realized, would effectively make it a superweapon. Altman has been pretty vocal about his expectation that this will eventually happen and that it is his goal to be the guy to produce it. Even if it isn't some superintelligence, the ability for a machine to do something like, say, exploit cybersecurity weaknesses, is pretty worrying for entities like governments. It's the pretense used when we saw the US government ban a US model recently.Again, you don't have to buy that "the singularity" is a real thing, but it's not hard to see that some people think some version of this is real and it is exactly what is being referred to as the goal in an "AI race."
  • valicord
    OpenAI exec scaremongering about "A nonliving, invisible, dangerous, and infinitely self- replicating agent escaped from a Chinese lab" mere 4 days before https://news.ycombinator.com/item?id=48997548 is hilarious
  • MostlyStable
    Just to add on to other comments: reasonable people can disagree about the degree of safety concern with near to medium term AI. But to not address the arguments at all is, in my opinion, a serious mark against the value of this article.
  • QuadmasterXLII
    “you can not stop X” is a shitty lazy stupid argument for “X is not bad.” No comment on the rest of the article.
  • kouru225
    The reason why anyone argues against local and free AI is because they want corporations to have control over the general population.America is a corporate hellhole.
  • elmer2
    We have continued to find backdoored Chinese manufactured routers and network devices.Will there ever be a way to fully audit Chinese models?
  • whack
    The most compelling argument against Open Source AI: it is analogous to "open sourcing" nuclear weapon technology. Or an how-to guide for making meth/bombs/bioweapons. There is a high risk that we will one day achieve ASI or something close to it. When that day nears, if even one open-source AI user handles their AI in an irresponsible way, we are all FUBARed. In the same way that one irresponsible person with a nuclear bomb can FUBAR the entire world, hence why "open sourcing nuclear weapon tech" is not even a subject of debate.You don't have to believe that ASI is 100% likely. People like Hinton believe there's a 50% chance of existential risk from AI, but you don't have to go that far either. If there's even a tiny chance of ASI posing a threat to humanity, that's reason enough to lock it down
  • Havoc
    The lobbying dollars don't care
  • wcoenen
    This post does not mention safety at all. What's to stop bad actors from fine tuning open weights to run fully automated genius-level scams personally targeting basically everybody?
  • okzgn
    From a neutral standpoint, governments usually have more general, comprehensive analytics, as well as greater context and insight into the risks of AI usage, and a possibly better methodology to control access to or use of open-source AI (perhaps by analyzing digital activities and consequences), because they learn from failures. On the other hand, open source creates major opportunities, and it’s hard to accept bans on something that truly benefits users acting for the common good. However, it also benefits unethical or opportunistic users—for whom regulation is indeed necessary.
  • deaton
    The only good arguments I see against open weight AI also apply to closed AI. And regardless, the box is open, nobody can stop it even if stopping it was a good thing.
  • AlexErrant
    I'm in favor of open source AI... however:> It's theoretically possible for a bad actor to embed hidden adversarial behavior in a model. But if this happens, it serves the interests of responsible actors to find these exploits as soon as possible, and the best way to do this is to let anyone who wants to inspect them.This is a bad argument. It isn't trivial to tell if the weights have poisoned:``` https://www.thedeepview.com/articles/microsoft-how-to-spot-a... https://futurism.com/future-society/easy-poison-open-weight-... https://semgrep.dev/blog/2026/ai-supply-chain-problem/ ```I'm not arguing in favor of closed-AI; I'm simply saying poisoning may be subtle.I was gonna say that at least the frontier labs may be motivated to not-poison their own models, but then Anthropic just attempted to poison Fable's LLM training capability so... sigh. I'll try not to derail.
  • SwellJoe
    The way Anthropic is playing the "AI is scary and dangerous and only we can be trusted with it" game is clearly aimed at regulatory capture, which would only be good for Anthropic. It's worrying how cozy some of the tech leaders have been with the Trump administration, which is also clearly aimed at regulatory capture that serves the interests of the biggest tech companies and no one else.The US has already been set back a decade or more by anti-science and anti-intellectual purges of agencies and formerly non-partisan positions. A ban on open models would be pretty much game over for the US as a tech leader. But, Anthropic would make a lot of money for a couple of years, so, who can say what's right?
  • alexashka
    The 'arguments' are not meant to persuade, they are meant to establish and entrench an overton window (regarding 'AI' in this case).
  • dualvariable
    > One probable outcome of an open-weight-model-dominant world is full AI communism...oh no, not communism.
  • mips_avatar
    I’m still kind of shocked that Dean Ball can tweet such incendiary stuff about OpenAI policy. Like presumably OpenAI would prefer it if their staff don’t pick fights with Trump administration officials.
  • kerblang
    I would be more inclined to agree if American companies were not in a competition to be the biggest flameout and lose the most money. How is China able to afford to train models that US companies need to burn billions of dollars for?In general, if American capitalism weren't so broken I would advocate for it as the strongest option, but American corporations are dead set on destroying it. The US may end up as "democratic socialist" if there is no capitalist reform.
  • anon
    undefined
  • kittikitti
    They don't have to convince developers, they just have to convince the general population. I've had several discussions about open source with people who don't care about coding and it's hard to untangle the misinformation they receive from the media. They get talking points from authorities that they don't understand so they will always double down on it. For example, rhetoric relating open source to communism is humiliating to discuss. For proprietary evangelists, the cruelty is the point.
  • redchaosgoddess
    [flagged]
  • caspianmagnus
    [flagged]
  • catigula
    You could literally develop a hyper-intelligent advisor on how to kill people or perform dangerous hacks using ablated 'open source' AI. You can do this right now, this very moment, and have an extremely adept advisor on how to do really, really bad things.