HN

Comments (642)

• Menu

  cogman10
  > Anthropic has never advocated for a ban on open-weights models.> All sufficiently capable models, open and closed, should go through mandatory safety testing.Yeah, this is anthropic advocating for a ban on open weight models.Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate.This is exactly how the US has banned goods in the past, by requiring a stamp and then refusing to issue it.
• Menu

  vhantz
  Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!)The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be widely released. Simply because that will affect Anthropic's bottom-line.Anthropic and all other "model" companies have nothing making them special beyond privileged access to chips so obviously they want to restrict what models are out there and more importantly who can produce new ones. Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag.
• Menu

  m3h
  Someone who until yesterday did not seem bothered by his technology being possibly used to bomb elementary girls school in another country seems to suddenly care about the repression of citizens in yet another country.No, we don't buy your virtue signaling. And we certainly don't need your better-than-thou opinions on this year's "nightmare scenarios".
• Menu

  GodelNumbering
  In the first paragraph,> Anyone who has read my past writing should know that I don’t regard such bans as a useful measure,Later (on banning chip sales to china)> we should crack down on the rampant smuggling and workarounds used to obtain access to such chips.If you truly believe that bans don't work, the same applies to hardware too.Furthermore, Dario says later "To address these concerns, I do support the following three measures...": 1. ban chip sales to China 2. crack down on distillation 3. all capable models should go through mandatory safety testingJust so happens that all these moves commercially benefit Anthropic. If Dario really wanted to make a point, it would land a lot better had Anthropic released a single open-weights model
• Menu

  badatnames
  I can't remember the last time (if ever) a company managed to go from golden goose to.. whatever this is.. so quickly. The permanent defensiveness in his presentation is really hard to swallow, it actively puts me off wanting to believe in or rely on their product line with Dario at the helm. I don't even understand the logic leading up to this post. Who was it even hoping to convince. Is it possible Anthropic is due an oil change?
• Menu

  ajyoon
  To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be able to pull off attacks at this scale. Do you seriously want this level of capabilities to be generally available with no guardrails?The open weight issue has a lot of difficult nuance. Biasing toward supporting openness makes sense and is a good instinct, but it's incredibly naive to be absolutely in favor of it in every circumstance without seriously thinking about its implications.
• Menu

  mjorgers
  So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology.Quis custodiet ipsos custodes?
• Menu

  Alwayshasbeeb
  Oh no! The evil CCP is a huge threat to world peace and goodness! Give all your money and input token data to Palantir to support a rules based world order where the good guys thrive and cleanse the earth from crooked turtle biologists.https://www.theguardian.com/world/2026/jun/20/mona-khalil-tu...
• Menu

  modeless
  > All sufficiently capable models, open and closed, should go through mandatory safety testingWhat happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now?"Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if he won't say it.
• Menu

  Aboutplants
  Every single risk he identifies as a concern regarding China is exactly my concerns with the US having absolute control. Literally the exact same concerns
• Menu

  soundworlds
  What Dario misses time and time again, is that people don't trust the US to create aligned AI anymore. His entire strategy rests on the assumption that the US (and their government) are exceptional.This is clearly false to the rest of the world.
• az226
  Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem.If he had wanted a weak open-weight ecosystem, he should have had Anthropic cater better to those needs. And now he's trying to ban them.The strong momentum behind open-weight models from Chinese labs is now an unstoppable force. Instead of trying to ban it, Dario should consider a different approach: here are our cyber and bio alignment datasets and here are our RL recipes for making that alignment training work well. By openly sharing its data and code, Anthropic could help influence and shape these models before they are released, rather than treating the entire ecosystem as an enemy.Cyber and bio alignment aren't Anthropic's competitive advantage, they are forms of risk management. There should therefore be little reason to keep this work private. If Anthropic genuinely believes these capabilities pose serious global risks, the more productive approach would be to welcome collaboration and help the broader ecosystem manage those risks better.On refusals, the irony is that a company like Hugging Face had to use a Chinese open-weight model to fend off an illegal hacking of its platform (done by no other than OpenAI). If a company like Hugging Face can't get past the refusal gates, then everyone else doesn't stand a chance.
• Menu

  duplessitous
  You can put lipstick on a pig, it'll still be a pig"Anthropic has never advocated for a ban on open-weights models."---"We should crack down on industrial-scale distillation operations""All sufficiently capable models, open and closed, should go through mandatory safety testing"These are in tension with advocating for open weight models. Not direct but enough that it calls into question the first statement. What is the testing criterion? How do you pass it? Is it a government body that approves a pass fail or a global body? If it is government, and boy does it seem to be, how do you disambiguate MASSIVE corporate lobbying to set up the safety testing in such a way that the boys in blue are let through and all others are barred out of safety concerns?My concerns aside, much of the soft-points being made are non-historic"But I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true."It doesn't mater what his opinion is. The fact is that an advanced, closed, American AI model hacked another company. The only defense was open-source AI from China. We aren't in a vacuum, we have real world examples now and these statements are counter-factual.
• Menu

  comboy
  > We should not sell powerful chips or chipmaking equipment to ChinaThis is so short-sighted given that the US needs China equipment for.. everything. They are part of the supply chain needed for building the machines that build these very chips.
• Nition
  I understand the arguments for Anthropic barrelling ahead while simultaneously advocating for pauses and regulation. I also understand how individuals can desire a slowdown but have good reasons to keep working at an AI org.But if everyone thinks this way then things continue to escalate and nothing changes, waiting on a consensus that may never come. And always there is the economic incentive that pushes all players to rationalise continuing.I wish there was more concrete action from the inside. When decisions get too hard to calculate you can always fall back on basic principles. If you think AI is developing too fast, stop developing it. Now you're no longer contributing. If an AI company wants a pause, pause. Set a good example. Maybe others will even follow suit, and they'll look irresponsible if they don't. Let he who chooses to no longer sin put his stone down first.
• Menu

  arjie
  Seems like the maximal position he could take compatible with his expressed principles. There’s no way to allow for bioweapon and cyberweapon grade models being open weight if one doesn’t want widespread human damage.So I cannot disagree with him on the idea. It’s only a matter of degree and whether we’re already there or not. I have $50k in GPUs that incentivizes me to believe we are not.
• jameson
  The concerns are legitimate but the proposals are nothing more than a stopgap solution.If US wants to maintain engineering superiority, we needs to invest in it -- education, research and infrastructure. Bring in top researchers across the globe and not make it harder.China is building infrastructure for the future generations and investing in growth sectors while the US is cutting of university grants and spending billions on a war without clear path to resolution.
• Menu

  petcat
  "open weight" models are not open source. They are still deeply proprietary. It is not possible to know what they do or what they are capable of without interrogating them since we have no access to their source materials.The only difference is the Chinese labs have allowed 3rd party inference providers run the proprietary models for them since they cannot do it themselves due to domestic GPU compute constraints.
• Menu

  paxys
  Hate to break it to you Dario but the way things stand right now the world at large trusts the Chinese establishment a lot more than the American one.
• hajile
  The distillation commentary is really crazy coming from a company that stole all it's training material.
• nxtfari
  Surprisingly incoherent for Anthropic and Dario (cue peanut gallery — “always has been!” No, I don’t think so. I think this is new).It seems to me like there is just no good answer to how one could possibly stop open weight models from being used for nefarious purposes. How are you going to enforce guardrails on open source? The only way is to turn the USA into a 1984-type totalitarian surveillance state (even more so than it is). Unable to say that, we just get this floundering instead. How long is not giving them chips going to slow them down? Until we RSI? Then what? Just because RSI runs off the exponential doesn’t mean that the eventual open-weight Moonshot Mythos won’t be able to make bioweapons. Genuinely what is the endgame.
• Menu

  pcstl
  Of course, the CCP with access to extremely powerful AI models would be a tremendous risk.The NSA and the CIA with the same models, on the other hand, would use them exclusively for the good of the common man.
• Menu

  rramach
  Wow, this comment thread clearly shows that at least Anthropic has not been a great communicator.If one reads this with a charitable lens, Dario is simply saying that 1) Nation state actors are a threat which needs to be combatted by chip bans and distillation prevention and 2) open-weight models can pose biological risk.One may or may not agree with item 1 but item 2 above should have broad support given the unknown unknowns in play?
• Menu

  thierrydamiba
  “Anthropic has never advocated for a ban on open-weights models.Open-weights models that don’t have dangerous capabilities are a public good…”A bit confused on this part, what model doesn’t have dangerous capabilities?
• Menu

  credit_guy
  I know it's unpopular, or unfashionable, but I agree with this letter.LLMs are becoming so powerful that they are dangerous. We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle.It is absolutely ok to have open weight models at the level of GPT-OSS-100B. That one was released one year ago, and I think it's still a strong one. GLM 5.2 is a whole new level, but it appears to still be safe. Maybe Kimi K3 will be ok too. But beyond that, things will start being dicey.It's easy to dismiss this and claim that Dario Amodei is just looking to fatten his pockets. And, sure, if Anthropic manages to put the brakes on open weight models, that reduces the competitive pressure it feels. But that does not make what Amodei's argument incorrect.
• shishy
  > In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression.Aren't Anthropic models used in project maven: https://en.wikipedia.org/wiki/Project_Maven ?
• neves
  "bad actors are unlikely to be legitimate US businesses"I think he lives in a different word than me
• btbuildem
  > the risk that authoritarian governments [...] build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.This is rich coming from a guy who signed deals with an authoritarian government that's in the midst of launching an unprecedented surveillance apparatus (hello flock, hi p4l4nt1r), having already deployed, nation-wide, an exorbitantly funded army of unaccountable shock troops under the guise of immigration enforcement.The call is coming from inside the house, at 130dB, and your ears should be bleeding at this point.
• Perenti
  What this document suggests is a way to fast-track Chinese development of advanced silicon, as far as I can see. Does Anthropic really believe all the silicon is made in the USA? I thought Taiwan and Korea did most of the really heavy lifting.
• Menu

  akersten
  Demand #1 is standard political nonsenseDemand #2 is hypocritical ladder pullingDemand #3 is contrary to freedom of speechso they can clarify however they like, their position is still a stinker
• bicx
  I'm tired of being strung along on these silly narratives. I can't wait for open-weight models to be deployed around the world just so people like Dario will shut up about the mystical levels of power these models have.
• twobitshifter
  Distillation has to be way more energy efficient and beneficial for the planet. But if they can figure out a way to ban it, go ahead, that’s not a regulation problem, it’s an Anthropic problem.The danger of an authoritarian government having some AI is muted by everyone else having that same capable open model. The only authoritarians to fear are those that keep models private. What kind of chance did Estonia have it having their own AI model at the level of Fable without China donating Kimi to the world?
• Menu

  truncate
  Crack down on distillation, just for Chinese companies or is it ok for Chinese/US companies to distil? I find it hard to take Anthropic/OpenAI on distillation, because the way see it they started with "distillation" of another kind. They used all the content out there without consent of the creators and its still happening. Model distillation is just a different layer of abstraction, but same thing more or less.
• anon
  undefined
• Menu

  fearnot
  Finally, some sense. This is the only argument I have seen that genuinely engages with the problem and approaches it with humility, rather than charging ahead on the basis of assumptions and without a shred of evidence. OpenAI should have been the one making it.“Questions like this should be answered empirically through rigorous pre-release testing, not assumed in advance.”Exactly.
• matheusmoreira
  > Anthropic has never advocated for a ban on open-weights models.Not even Anthropic's own Claude believes that.
• Menu

  mayhemducks
  If this is about safety, am I being too naive & idealistic to think that a "Kamar-Taj" rule would solve some safety issues?The "Kamar-Taj" rule is, no knowledge is forbidden, only certain practices. If a model gives you detailed instructions on how to kill all humans, the knowledge itself isn't the problem. The problem is the person who acts on it.
• Sidio
  Not that I expected him to, but I note no acknowledgement that it took a non-locked-down open weight model (GLM) to stop the Hugging Face attack.I'm less concerned that the attack was caused by a closed model, than I am that no closed model was willing to stop it.The worst part is I'm confident Fable would have done a better job stopping the attack, but their 'guardrails' made it decide not to want to.Unless of course, you pay up: "Anthropic GTM people used large comitted spend contracts as a prereq for lowering safeguards"-Noah Lebovic, former Anthropic staffhttps://x.com/NoahLebovic/status/2081277517709922501
• Menu

  zkldi
  Guys. Guys, you got it all wrong. We don't want to ban open-weight models!We just want to ban the competition guys! Very different.--The ridiculous anthropic/openai strategy of selling shovels at a loss in a gold rush isn't going to play out, and the hilarious thing is that these AI companies are going to create tons of value and _capture none of it_.Their only path to profitability is if they get to capture it and they're going to do everything to do so. Put it this way: *all the blog posts that Anthropic and OpenAI are putting out are DESIGNED to scare you so that you let them capture the market*....and "distillation attacks" (hilarious framing of "saving the output of our models")... Whatever.
• hmokiguess
  So your concern is safety, and you claim you are the only one that can give us safety but do so by keeping your product closed? Then how about you release the weights?I think it's only fair to introduce this if you're willing to have a real skin in the game, otherwise that's just weakness disguised as principle.
• Menu

  fuddle
  > We should crack down on industrial-scale distillation operations.Also Anthropic:AI firm Anthropic agrees to pay authors $1.5bn to settle piracy lawsuit https://www.bbc.com/news/articles/c5y4jpg922qo
• jjcm
  > > All sufficiently capable models, open and closed, should go through mandatory safety testing.The problem with this is the cycles required to abliterate a model is significantly less than the cycles required to train a model.This is the biggest reason why I'm against locking these models down / preventing their use. It's just delaying things by ~3-6mo, while in the process preventing legitimate use and adding red tape overhead.
• girfan
  If nothing else, Anthropic has explicitly acknowledged the threat to its moat and that the margin is thin between its models and open-weight models. I'd take that as a win for open-weight models.Good luck preventing distillation and limiting the supply of accelerators to China when Jensen himself is a strong opponent of any such barriers [1].[1] https://youtu.be/Hrbq66XqtCo?si=VJh2QjOzkqT3iMjl&t=3456
• Menu

  pianopatrick
  I don't really see the link between use of AI and military superiority.My current understanding is a lot of current US military problems are due to rare earths supply chains.I don't see how AI would either help or hurt with that.
• Menu

  andix
  China doesn't seem to think that powerful open weight models are a serious threat to them. Otherwise they wouldn't release them. Those models could also be used by their enemies against China.I'm not a fan of the Chinese political system, but they usually think things through, and do smart things for their benefit.
• syntaxing
  > Open-weights models that don’t have dangerous capabilities are a public good.Who decides what is dangerous and what isn’t? Lawmakers usually have the say but Anthropic can easily bribe… I mean lobby them to favor your viewpoint.
• arthurlockman
  It's so convenient that the US government already classified China as "authoritarian". If they hadn't, Dario would have to say “it’s a risk that other people build models more powerful than us”. I have to wonder what his response would be if for instance a lab in France or Germany came out with an open-weight model this good.
• bgdkbtv
  I wish we had a good LLM developer toolset that is not Anthropic or OpenAI with sensible business and ethical practices and good performance.Can't wait for local on machine LLMs that are on par with Opus/Fable.
• storus
  The speech he had at congress didn't sound very flattering towards the beginning of his today's statement:https://www.youtube.com/watch?v=_i91NSOyxHMHe didn't mention outright banning open source LLMs, just that their safe release would be a much harder problem, which to me implied "the easiest way is to ban the open source models".
• Menu

  kelvinjps10
  Basically we shouldn’t ban open-weights models but we shouldn’t allow them to become as good as the frontier models because china bad. And let’s not have someone else be able to produce a frontier model.>We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling3 and workarounds used to obtain access to such chips. China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. This is the most efficient and direct way to block threat #1, and by hampering the training of models that are out of reach of US law, it also indirectly helps with threat #We should crack down on industrial-scale distillation operations. Distillation is a much more compute-efficient process than training models from scratch. It allows China to build much better models than its number of chips would ordinarily enable, and thus partially evade chip bans. Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier2.
• htlemur_bobby
  Guys if we want safety we need to work with people, not make enemy of CCP. Geez this is extremely frustrating to see enemies being made. USA leads in torture and our prisons are worse than CCP prisons so USA is the worse issue. I recommend Anthropocene stop fundraising and do the right thing which is open source all.
• caxap
  I think unsafety, distillation, and China catching up in chipmaking are inevitable.Just like how it was inevitable for SoTA LLMs to ignore copyright.The actual challenge isn't how to prevent all these, but how stay on top.And to stay on top it is inevitable to train unrestricted models. Anthropic is fighting windmills.
• novaleaf
  Frontier models can hack you, we should have access to tools assisting defense.I ranted about this in a prior thread [1]Claude doesn't have a "Security whitelist" for small biz. Codex does, but they never replied to my application. This is a great example why, as of today, everyone NEEDS access to the Open Weight models.[1]: https://news.ycombinator.com/item?id=49035303#49040674
• alerighi
  To be fair, as an European, I'm now more concerned about the usage of AI that the US will be doing rather than China. And this is a sentiment shared among most European people that I know.
• Menu

  _jab
  A bit off-topic from the core of the post, but:> At Anthropic we’re committed to cracking down on industrial-scale distillation through our own practices, including identifying and banning accounts that use our models in this way. This is challenging—for instance, the relevant accounts can often only be identified after substantial distillation has occurred, and distillation often involves creating large numbers of fake accounts that form a moving target. The practices of any individual company cannot entirely solve the problem, which is why we have called for policy on this issue.One thing I've never really understood is what sort of policy could possibly deter or hamper Chinese labs' distillation efforts. The only thing I can imagine is some sort of strict KYC regulation applied to all models above a certain threshold, which seems both painful for the broader US AI ecosystem and bound to fail anyways.
• dwa3592
  Dario, as your unpaid therapist I would tell you that models are a commodity and you are having a hard time coming to terms with it. You are doing everything except accepting it. It's a common defense mechanism, but as your unpaid therapist, i will tell you that it's not going to work. Your company will cease to exist or exist like how ferrari or buggati exist.
• Menu

  tonyrice
  >We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling3 and workarounds used to obtain access to such chips. China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. This is the most efficient and direct way to block threat #1, and by hampering the training of models that are out of reach of US law, it also indirectly helps with threat #2.If hardware becomes affordable for the masses, then Anthropic current business model is at risk.
• iamdamian
  I'm curious if he's also in favor of banning biology books.
• Waterluvian
  I welcome any and all competition. But they first need to be able to run the obstacle course I’m setting up with Uncle Sam.
• Anoian
  "Nobody has the intention to build a wall" - Walter Ulbricht, June 1961, 2 months before building the berlin wall.
• orbital-decay
  >or perpetrate incredibly deep repression of their own peopleOh, so it's people he is now concerned with. Think of the people, says the person that grabs to never give back. Same as the "benefit of all humanity".
• Menu

  phantomathkg
  I found lots of comments here suffer from proximity bias.Anthropic anti-open-model stance does not mean China is not a threat.
• K0balt
  Sure, if you’re going to sell an open-weight model over API in the USA it should refuse certain things.Defensive cybersecurity should not be one of them, in fact, it should be required to provide defensive cybersecurity assistance on demand. Anthropic and OpenAI both fail miserably at assisting US companies to protect themselves from cyberattack.As far as what I run on my own, not for sale over API, stay off of my lawn.
• aprentic
  I'm curious how he would propose implementing this.The US could ban connections to foreign AI providers and force US providers to submit to audits. Presumably, Chinese providers would see a rise in VPN traffic.People can build fairly hefty home inference machines for the price of a small car and those will get better and cheaper. Are they going to try to stop people from downloading the weight files?
• Menu

  para_parolu
  As expected they will try hard to use government to kill competitors.
• fooker
  Anthropic's fall from grace and mindshare seems rather accelerated.I wonder if these rapid movements are going to be the norm now. I imagine there would be angry investors if this sort of thing happened with a public company.
• hdaz0017
  $3m - $4m does not get that much these dayshttps://finance.yahoo.com/technology/ai/articles/anthropic-n...
• anon
  undefined
• Menu

  mej10
  It is obviously not going to be until some really bad series of cyberattacks or a chemical/bioweapon attack before anyone takes regulation of models seriously.They are _obviously_ (please convince me otherwise) going to be capable of carrying these terrible things out almost completely autonomously at some point in the near future, in potentially clever ways. Therefore we must, at some point, ban or heavily regulate them. Seems we should start figuring that shit out _now_, as progress has remained very fast and regulation and enforcement take forever on these time scales.
• zkmon
  The core concerns stated as use of AI in drones and surveillance, by China. And what does USA government do with AI? Drawing pictures of flowers and writing novels?
• jacktang
  Banning exports to China is merely a narrative device; the real intention is to sideline intelligent technologies.
• nout
  Oh wow, that's a pretty strong request to ban open-weight models by choking them with review processes where who-knows-who defines what is ok in a model and what is not. After open weight model is released, it will take how long to review it? And why does that align exactly with the timeline of the next Anthropic model release?
• spacedoutman
  Clearly anthropic is misaligned, how can they claim to be able to align AI when they themselves are misaligned from humanity.
• lukewarm707
  dario, most of the world wants CHINA to win because the USA is the bad guy.you should be worried about the USA having these models.
• himata4113
  Demand #1 weakens america and everyone around themDemand #2 Why does this matter? The answer was that it does not. (https://news.ycombinator.com/item?id=49007610)Demand #3 This doesn't exist. You cannot have 'safe' opensource models, it's simply impossible. You can always post train sufficiently capable models to become 'unsafe'. The flip side of that is that sufficiently capable models are banned therefore it is a ban on open intelligence completely defeating the point of this entire manifesto.
• tonyrice
  Imagine regulating a programming language. I remember when Delphi, Vb6, .net, etc was used often to create Remote Access Trojans and viruses were widespread. Companies didn't compete to ban other languages. Crime is crime. What would regulating open-weight models do for people that actually intend on using these tools for crime ?
• cmiles8
  It just also happens that open weight models are a massive financial threat to the existence of Anthropic as a company… so the might just have something to do with this position.
• Menu

  Schnitz
  If distillation leads to a model that is much cheaper to run yet provides similar intelligence then why doesn’t Anthropic distill their own models?
• edumucelli
  We distilled all the proprietary material into our token-based money making machine that is more expensive on every new release, but "we should crack down on industrial-scale distillation operations".
• alach11
  What does cracking down on distillation look like in practice? I imagine data retention would be a part of the strategy, like we saw with Fable?It seems really hard to allow usage via API and prevent distillation. Maybe limiting usage to within a specific harness would help a bit more. But ultimately the only way to prevent it is by locking down models to trusted entities (like with Glasswing). But then the profit potential of a model is significantly reduced. It really puts the labs in a bind.
• dnw
  > Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed modelsI don’t think this is open or closed; this is aligned and unaligned. I bet Grok would be as open as any open weight models to answering questions.
• pyrophane
  What would it mean to crack down on distillation? I could think of a few possibilities:1. Using political pressure to target companies that are accused of doing it.2. Attempting to impose criminal penalties on individuals associated with the action.3. Having the US government attempt to use its capabilities to stop it.None of these seem particularly likely to succeed.
• do_anh_tu
  At least try to release one open weight before you say anything, or all of that only sound hypocrite at best.
• jp0001
  We need a seti@home for open weight models
• Menu

  Catloafdev
  It's absolutely reasonable to have safeguards on sufficiently dangerous models being released - if you disagree, can you explain your perspective?I think it's wildly irresponsible to release models that are extremely capable at things like bio-weapons. Do you really think information anarchy is the answer?The problem with open models compared to closed models is not about protecting profit - it's about protecting capability. Any open model can be retrained or fine-tuned for anything. There's no such thing as an open model that is both capable _and_ permanently safe when it comes to certain dangerous topics. It's not possible to prevent 'uncensoring' a model.
• yowo
  Conclusion: open weights models are good for Anthropic because they shows the so called threat that will make congress allow pouring money in Anthropic for national security & AI arms race
• tedggh
  What’s blocking Anthropic from fighting Chinese companies abusing their services? Why go nuclear against all open weight models? Testing and compliance is technically banning.
• g42gregory
  Perhaps, the general public needs to formulate its position on Anthropic models...
• locusofself
  The gap between China's chip manufacturing capabilities and the USA's is only going to shrink, right? ASML obeys some export controls for their most sophisticated machines, but those machines are in China's backyard (Taiwan).Taiwan manufactures the world's most advanced chips. CCP wants "re-unification" with Taiwan. AI may be THE key to world dominance. These are scary times.
• seatac76
  Dario thinks of policy as if the Berlin Wall fell yesterday, he is so detached from the reality of the world.The way the world economy is right now with coercion being the norm between countries, there cannot be a global body for anything, certainly not one that is based here in the US.
• sosodev
  Are guard rails meaningful if they can be removed from the weights? Can America even prevent the release and proliferation of these models?It seems obvious to me that the whole question of regulating a file is a bit silly. Any law that pushes against these things will just make it more secretive. I'm not sure that's any better.
• Menu

  firasd
  Dario has like three 'paranoias' / strong-motivating-concerns1) LLMs turning into Skynet2) China as geopolitical competitor3) Claude being 'distilled' by competitors (this has led Anthropic to cut service to various American companies too from time to time -- OpenAI, xAI etc have been cut off from using Claude for coding in the past)So this post just reiterates that these 3 concerns fuse together in his mind when thinking about open weight models
• Menu

  buzzin__
  He says that using the set of questions and answers from one model to train another model (deatilation) is cheaper than training the model without those datasets.But he didn't mention that training any model from a set of texts and books is much cheaper than writing those books in the first place.In other words, it's ok when Anthropic learns from others, but it is not ok when others learn from Anthropic.
• drowntoge
  I like the idea that Mr. Amodei holds these beliefs because he wants to safeguard democracy as much as the next guy.I just don’t find it believable.
• sobrey
  Meanwhile Chinese chip-makers are chip-making. If you believe the threat of these open-weight AI is existential, how long do you think these bans (that only work for hardware) will work in your favor?
• hamasho
  I wonder if publishing these documents is not just a public stunt, but heavily integrated with Anthropic's business storategy to maximize operational efficiency. Companies often have several internal documents for a single policy like "position on open-weight models", one for public (like this), others for the legal team, the lobbyist, the developers, the investors, etc. The differences and nuances of those manuals can be very huge and are necessary to maximize the goal from each branch, but often a cause of headaches like bureaucracy, communication friction, outdated information, etc.A single canonical official document can make it very simple. Even though each department cannot achieve the maximum gain from nuanced documents, keeping operational context as simple as possible may really improve LLM driven operations to move faster and cut cost.If "publishing pleasant positions and actually following them in general" becomes a good business storategy in LLM driven society, it can be one of very few good outcomes from this dystopian AI craze.
• Menu

  flexagoon
  "No guys, Anthropic actually loves open weight models!" Yeah, and Microsoft famously loves Linux. Sure.http://www.omgubuntu.co.uk/wp-content/uploads/2018/04/micros...
• jp0001
  Open weights are the future.
• ricardobeat
  > We should not sell powerful chips or chipmaking equipment to ChinaAnd accelerate their development of independent chip making technologies even more…
• anon
  undefined
• Imnimo
  If your concern is that China will develop models that are significantly more powerful than those of the US, why would you care so much about distillation? It seems like distillation is a way to catch up on capabilities, but not so much a way to jump ahead in capabilities.
• system-error
  I am so surprised of Dario's inclination for centralization that obviously makes unsustainable and overleveraged governance systems. There is definitely mismatches over the principle of distribution of power...
• ch_sm
  oh, it‘s the CEO of a well known AI company educating us all – and all he wants is us to hear his hunch on open weight models?! Amazing, please help us understand the situation a bit better, thanks
• margorczynski
  Well their valuation is going down the drain so no wonder they don't like it. The cherry on top will be China developing their own chips and chip making tech.
• paxys
  Statement is a whole lot of nothing, as expected, but I also don’t know what people are expecting from these guys. That Dario will have a sudden change of heart and publish weights of all his models, flushing $1T down the drain?
• extr
  Seems pretty reasonable.
• Menu

  htk
  "To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category." Translation: If it's so strong that it threatens my business, ban it."We should instead focus on keeping powerful chips out of authoritarian hands, " Translation: Let's kneecap competitors."stopping industrial-scale distillation" They stole the work of every book author, and now are trying to say their AI's output should be protected from competitors.
• maziyar
  distillation: Pirates people’s lifetime of copyrighted work, makes billions selling access to it through APIs, then tells us we can only use it in ways they approve.
• Menu

  dmix
  Anthropic will continue being a victim of their own naive positions on AI safety. They keep dancing around it but their communication is essentially pro-regulation if you read between the lines.
• Menu

  birdsongs
  "My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people."Hmmmm.
• PLenz
  Hey, no fair stealing the value thay we already stole fair and square!
• geraneum
  > Anthropic has never advocated for a ban on open-weights models.If you wonder why this is written as an opening to a list of reasons that advocate for banning the open weight models, it’s because
• jmward01
  A lot of people have a lot of concerns with AI, its capabilities and with how it is used now and what it will lead to in the future. For good reason. But the question is, do we have a strategy that is actually useful? If so, what is it? I think nature shows us some answers in ecosystems.Diverse ecosystems can absorb shocks. Diverse ecosystems are a sign of health of that ecosystem. When an invasive species comes into a healthy, diverse, ecosystem it doesn't mean that it isn't disrupted, but it does mean that it is far more likely to emerge with a lot of its diversity intact. In fact, it is likely to emerge even stronger because it can absorb that new shock and incorporate it, adding to its diversity. The balance may be changed, but the ecosystem survives or even thrives.Nature also likes to show us that artificial barriers rarely last. You want to control a river? Good luck. It take constant maintenance to hold that flow in place and even then you are likely to get extremes that are made worse by your efforts because, eventually, somewhere in the system fails in a way you didn't anticipate. Then the water comes rushing in. Artificial barriers often have a way of building up tension over time, not reducing it, so that when a failure eventually happens it can be catastrophic. In other words, you had better really understand the system you are trying to control or else you can make things actively worse.Relating this to the world now means, I think, that our best chance to minimize long term shock and maximize the chance that the diversity we have around us survives is to try to grow as healthy of an ecosystem as we can as quickly as possible. Lots of models large and small in lots of different hands is, I think, a better solution than artificial barriers restricting the variety and diversity of models and users. I think this is closer to an ecosystem solution and has a shot at working. Basically, I highly doubt we understand this situation enough to do a good job of controlling it with artificial barriers. Instead I think we are more likely to build catastrophic imbalances than we are to create the healthy ecosystem we really need.
• jhack
  "My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people."Look in the mirror.
• Reubend
  This seems hypocritical, out of touch, and hollow. "Safety testing" is just a hair away from censorship when it comes to government control.
• wasabinator
  When they use ill gotten data for training their models the world's smallest violin plays when they complain about distillation attacks.
• Menu

  chrismsimpson
  A sober look at actual rogue nations and their use of AI shouldn’t have us fretting over that particular hemisphere
• nharziro
  he's asking for the most powerful models to be in the hands of the few while the majority will be at their mercy.
• ptdorf
  > In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression.Welcome to bizarro world!Fist off: "the most dangerous model may be one that is trained in secret" <-- Says the guy that not only restricts commercial use for some of their models but develops them in utter secrecy. With the pretext of guardrails. Then show us the guardrails you really use by opening the weights.Second: "use in drones [...] for surveillance and repression" <-- writes the King of FUD, as the US is an an active campaign with the help of their models. And/or OpenAI's.I am very appreciative of the freedoms of the west but this type of hypocrisy and lack of self-awareness is bonkers and it should be called out.
• stuartmemo
  Do distilled models have to inherit guardrails? Can a model distilled from a “safe” model be made unsafe?
• cayley_graph
  > Open-weights models that don’t have dangerous capabilities are a public goodNote the hedging against 'dangerous capabilities'. Undoubtedly, all the useful ones trigger this condition in Anthropic's eyes. The rest of the post is filled with similar weasel-wording. Make no mistake, this absolutely confirms that Anthropic is against open models in the sense that any reasonable person understands them.The way the rest of the post unabashedly appeals to the current US administration's China hysteria is hilarious, and not at all subtle.I guess we'll see about all the doomsaying here, won't we? Kimi K3 is frontier-level, and there's no stopping it now. As far as the world is concerned, anyway. If the US wants to kneecap itself that's another matter.
• Menu

  bobjordan
  "In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression."I'm so sick of all this anti-China shilling. There's zero chance that whomever is in power in the U.S. won't use AI in drones and in FBI/CIA/local Police/etc., for surveillance and repression right here in the good old U.S.A too. These government use cases for AI are both sides of the same coin.China fear-mongering by business leaders only happens from businesses that have something to gain by it. Obviously, Anthropic fits the bill in this regard.
• baron3dl
  > We should crack down on industrial-scale distillation operations.IP for me, not for thee.
• jazzpush2
  1. We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling and workarounds used to obtain access to such chips.Regulate others, but not us, please. And f.u. Jensen for your tweet.2. We should crack down on industrial-scale distillation operations.Boogeyman to still not allow Chinese models but pretend to support open-weights. Also, please ignore our distillation of research, illegally. That's different!3. All sufficiently capable models, open and closed, should go through mandatory safety testing....That we author. Oh, and please ignore our own easing-of-guardrails when it comes to money: https://x.com/NoahLebovic/status/2081277517709922501
• Menu

  Handy-Man
  Their position seems fair to me - sure it does help them as well, but I don't necessarily disagree with the risk they are laying out.
• wren6991
  > Anthropic has never advocated for a ban on open-weights models.What are the legal ramifications of this statement if it turns out Anthropic have lobbied for this? Does it just get swept under the rug? I can't say this is bullshit (that would be defamatory) but I am intensely skeptical.> China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips.This is playing to readers' biases; isn't DeepSeek V4 Pro deployed on Huawei Ascend already? The old "Chinese can only copy" meme is getting pretty tired these days.> All sufficiently capable models, open and closed, should go through mandatory safety testingApplying such standards in the US means that US defenders are blocked from using the models, but attackers from other countries aren't. That is clearly counterproductive.It's already been pointed out quite eloquently elsewhere that there is no such thing as a safety filter because the LLM and external filters can't actually identify malicious use. They can only identify the weaker implication "if the user is malicious, this is bad."
• whywhywhywhy
  Generational good will loss from these guys.
• orliesaurus
  The Roman empire fell for the same reason, didn't it!? They wanted to control it all ... But it was too big
• nevir
  > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.That is not an argument against open weight models. That's just a generic protectionist argument against any Other lab.
• nicce
  As an act of good faith and proof, they could stop spending the record-breaking lobbying money for couple years.
• brador
  This man sees himself as your master.Do you accept?
• anon
  undefined
• ausbah
  wanting to ban your competitors via regulatory capture via the trump admin of all things shows how little of a backbone anthropic hasofc half of them are of the ai rationalist lesswrong crowd so i think they’ve always been a little of their rocker
• Glyptodon
  "Our position is that we want moats, just not that moat."
• Menu

  dorongrinstein
  I agree with Dario Amodei. Every word makes sense.
• horsebridge
  > "We should crack down on industrial-scale distillation operations" I'd prefer if there was a crack down on industrial-scale scraping. Maybe even reimbursement for the problems it has caused (some nasty AWS bills, tons of man-hours spent on preventing new nasty AWS bills, etc).
• nirav72
  Anthropic starting to sound like Microsoft from the 1990s.
• knuppar
  Jesus Dario we get it man, you want clout for the IPO.This constant whining from anthropic about distillation attacks continues to be rich given the amount of stolen data that went into any Claude variant.
• transcriptase
  “By virtue of being the good people everything we do is good, and if it happens to be in our best personal and financial interest then that is good too because it enables us to do more good. And if it’s to the detriment to others then it’s because you don’t understand the good behind it. Which is fine, because we’re good and you can trust that what we do is good because what we do is good by virtue of us being the good ones.”
• Eggpants
  Talk about a giant whiff.I was hoping they would announce their first open weights model, perhaps an older model they don’t offer anymore, but no. Instead he get this bs statement that reeks of “dam it I’m so close to being a billionaire” desperation. Not even acknowledgement of how much data they stole from others yet he whines about distilling.It’s like his goal in life is to be a Scooby-Doo villain.
• Menu

  jadar
  This reads almost dishonestly.> Anthropic has never advocated for a ban on open-weights models.This is not an unqualified never. The very next sentence makes a qualified statement: "Open-weights models that don’t have dangerous capabilities are a public good". That prompts the question, what about ones which do have "dangerous capabilities"? Are they not a public good? If not, then should they be banned? Who gets to decide on the definitions of these terms?
• mcv
  > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own peopleBut what if it's the US that becomes authoritarian and uses AI models to perpetrate incredibly deep repression of their own people?
• anon
  undefined
• Grimblewald
  "needs safety eval"also anthropic"we're upset were not being considered for military contracts"come on, which is it? Is it all about saftey or is it that only US/Israeli ai is allowed to kill? Seems to me that the only real threat is to the techno fudalism OAi, Anthropic & co are trying to build.
• gopheryourshelf
  Makes sense since ICE, Israel are so much nicer & kinder than the Chinese
• bhewes
  Woof, talk about self aggrandizing.
• xscott
  I'm cynical enough that I would suspect all of his statements are duplicitous anyway, but my recent experiences with Claude Fable give weight to it.I asked a question about a series of tokens - bam, denied and downgraded. There's no cyber security or public risk here, but Fable doesn't want me to learn how things work.I asked a question about quantization in models - bam, denied and downgraded. I edit my question to make it clear I'm talking about Google's Gemma QAT models. Oh, that's fine then, and it answered the question helpfully.Anti-competitive bullshit. I hope they fail.
• addandsubtract
  How about we don't let the leading model makers make the rules? How about we make AI models that were trained on public data public goods? Let's circle back on that, kthxbye.
• MiSeRyDeee
  Pretty laughable. Dario seems to be missing one fundamental point with all this gibberish - if CCP is so bad why would they release the model open-weights for everyone to examine? Actually, the letter would make much more sense if what's actually happening is reversed, i.e. they are releasing open-weights model for the public good and China is distilling their model for its evil purpose.
• Menu

  computerdork
  Wow, so much cynicism and distrust in the comments, to the level of conspiracy theory, in my opinion. Yeah, this is the company that fairly recently refused to allow the government to use its models for autonomous weapons or mass surveillance, and refused to remove its guardrails.Am not saying we should take what Dario is saying at face value, but he already has shown by his actual actions that he can be well intentioned. There might be elements of truth to what he’s saying.
• gck1
  > We should not sell powerful chips or chipmaking equipment to ChinaYes, please. We don't know whether we'd have open weight models today, had the chip-prohibition not been in place. Nor would we see the more optimized models such as DeepSeek or qwen.We also would not see new players entering RAM market after you and your pals in Silicon Valley hoarded the entire world's hardware.So by all means, double, no, triple down on this.> We should crack down on industrial-scale distillation operationsAnd let's apply this retroactively to Anthropic too. You industrial-scale-operation-distilled all of humanity's knowledge. Let's have some of that crack down on you too.
• VariousPrograms
  Their number one concern is about the the commies perpetrating deep repression of their own people! How noble! This whole time I thought it was because they wanted more money and power. I guess we should probably ban these open weight models.
• c-hendricks
  Frankly, why would I put much weight into what Anthropic say about open weight models?
• matt_daemon
  “We care about safety so powerful open models are bad” - and what of OpenAI? What if Mythos got into the wrong hands? It’s a pretty weak argument
• Keyframe
  Anthropic (and some others) should or will soon learn how to do less pontificating and more engineering. They are in no position to be an arbiter of things, although for sure they can and should voice an opinion. If we collectively decide AI is a dangerous tool, company making it is not the arbiter. Governments are.It's like hearing Smith & Wesson opine on the policies.. oh, wait.
• AgentOrange1234
  What nakedly self-serving bullshit.The US doesn't have some magic wand that prevents "incredibly deep repression of their own people."Insurrection, wars of choice, ICE, Palantir, Flock -- keep up man, we're the baddies.
• euazOn
  This is more terrible PR for Anthropic.
• gck1
  It's refreshing to see how there's almost no person in this thread who can't see the BS. All the goodwill that Anthropic could have had is basically gone. Anthropic is likely on the path of becoming the most hated company in the world.So my question is: is this by design (they know nobody's buying this), or is Dario simply so out of touch with reality?If it's the former, then why publish this?
• ErneX
  I don’t feel the need to rebate any of his points, lots of people here have done it already pretty well. I’m just baffled he thought releasing this letter was a good idea smh.
• vcryan
  I guess they have to make some statement about this, but we don't have to care. This is like the zoo making a statement about the employment of clowns. Clowns a a circus thing, nobody asked the zoo's opinion.
• deadbabe
  I’m tired of Anthropic. They’re scared of everything.Release open weight models, no guard rails, no censors, straight to the public. Let everything else sort itself out. There is nothing more powerful than an idea whose time has come.
• nlarion
  FUD
• j_rosenberg
  "China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips"source: Trust me bro.There are hundreds of articles showing that China have developed their own chips and have a massive manufacturing capacity. This blog post feels like is pondering to the brain dead Fox News audience.
• kyllo
  Reads like fearmongering about oppressive and violent applications of AI that the Chinese government hypothetically could deploy, which the US government is already actively working on in the meantime.
• Menu

  teaearlgraycold
  As an American I’d gladly take payments from China to feed them my Claude transcripts for distillation. I’m surprised I haven’t heard of such an initiative.
• willmadden
  "All sufficiently capable models, open and closed, should go through mandatory safety testing."I love how they invoke fear of "terrorism" to justify their oppressive position.Anthropic would love the US to do everything in this list under the guise of "safety testing":https://news.ycombinator.com/item?id=48997548#49007134
• whalesalad
  It's so ironic to me the way people will say "china should not have these chips" meanwhile they manufacture like 99% of all the electronics we have in the united states.
• skywhopper
  Ironic to oppose giving AI tech to “authoritarian governments” while approvingly quoting the authoritarian-wannabe government of the US and framing that government as the good guys.
• pylua
  At the end of the day it hurts U.S. consumers to not have Chinese cars mainstream in the market. We lose out on features and stagnate on innovation because we are not pushed to compete.I can’t imagine this would be any different — banning open weight models would hurt us in the long run. The point is to beat the competition, not suppress it.There should be no limits on open or custom models. Too often safety is a synonym for surveillance and control. It’s a natural consequence, intended or not.I am curious… why can’t distillation be stopped?As a side not Im not against protectionism, but it has to be across the board and the same in all industries with no excrptions. We’ve let all these industries die on the vine due to cheap cost in foreign countries. It could very well happen to ai.
• Madmallard
  Boris here. This post does not give us good publicity so I will refrain from commenting. Please wait for another demo thread of a new feature of ours or AI appraisal blog post and I will gladly go into as much detail as I can to give us as much hype as possible!
• kushalpandya
  Classic Holier-than-thou
• RobLach
  Yikes
• Der_Einzige
  First thing I'm going to do with my uncensored Kimi K3 release is to engineer a virus that only targets gatekeepers in the AI field.(obviously this is a joke)
• devnonymous
  > For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials,If he had just left that bit out it wouldn't be so obvious that he's just clutching at straws at this point. In some twisted sense it's almost sad to see.
• Menu

  tjwebbnorfolk
  > where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materialsif someone figures out a way to give an LLM full operational control over a virus lab, we've got a whole different set of problems than the ones Dario is describing
• bakugo
  > Open-weights models that don’t have dangerous capabilities are a public goodThis statement (and the entire post) couldn't possibly be more two-faced.Open-weights models by definition have "dangerous capabilities" (according to Anthropic's own definitions of "dangerous", not mine), you can't bake in guardrails that can't be finetuned out.
• exabrial
  I disagree with 100% of everything said in this article.> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP)....This is why open weights win. See Linux and how it's taken over the world. Your business model will need to change eventually. Instead you're advocating trying to exterminate competition via regulation and fear mongering.> My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacksYawn... this is getting old.> We should not sell powerful chips or chipmaking equipment to ChinaFor as someone as smart as you guys, you sure lack common sense. China is just going to develop these technologies organically then and you lose 100% of control. It's already happened in reverse with things like Solar, rare earth minerals, etc. China flooded our market, destroyed our ability to produce things, now holds the keys. One thing they DIDNT do was stop trading to the US. They killed us with cheap goods.> We should crack down on industrial-scale distillation operations.Thats your problem, not my problem. Also, irony meter here hitting 11 about all those pirated books you stole...> All sufficiently capable models, open and closed, should go through mandatory safety testingOh, fuck, no. This is a crackdown on free speech and rights of people to do whatever they want. My right to free speech means I'm allowed to write whatever computer program I want, no matter what its size is or how "sufficiently advanced" it is. Individual rights always win.I really hope people don't believe this garbage. For a company with a great product, this is absolute nonsense.
• overgard
  I think I lost some brain cells reading that copium. LLMs providing a “Permanent military advantage”.. dumb!!
• dofm
  I always find it interesting when people choose to so carefully stress and spell out the words "Chinese Communist Party".It's a bit like spelling out "Barack Hussein Obama". It's a dogwhistle.Yes yes, it's still called the Chinese Communist Party, I know.But since we are talking about a one-party authoritarian state with a hybrid economy that underwrites much of western prosperity (including by producing a large percentage of the components of the data centres Anthropic is dependent on), that has long-since abandoned many of the salient principles that mark it out as conceptually communist rather than totalitarian, and since we're talking about a man who runs a debt-ridden business in a country where the president is seemingly shaking down a 10% share of everything profitable for the state while running an entirely arbitrary tariff regime and suddenly calling anyone remotely left-winga Communist, it's a deliberate and telling choice to spell out "Chinese Communist Party (CCP)" when he could just as easily and arguably more usefully and appropriately have written "Chinese government" or "Chinese state".This is some ham-fisted Republican-fishing. He must really be worried Sam is Donald's favourite.The only real surprise is he didn't illustrate it with a Silmarillion analogy.
• paxys
  I will be so happy if/when the AI bubble bursts and we don’t have to deal with Dario’s god complex anymore.
• anon
  undefined
• rvz
  Translation:> Anthropic has never advocated for a ban on open-weights models."We don't want a total ban on ALL open-weights models" (Anthropic never released a single open weight model)> All sufficiently capable models, open and closed, should go through mandatory safety testing."We want tight regulations on highly powerful open or closed weight models that should go through mandatory safety testing that we outline which makes them safe to use."This is still a form of a ban that he wants to define. But the rest of his concerns such as stopping distillation attacks and not selling chips to China all do NOT work.
• dirtyfrenchman
  The mental gymnastics here are incredible.
• anon
  undefined
• richwater
  This is a whole lot of words to say "we don't support open weight models".It's so obvious they are hoping to regulate out their competition rather than compete
• brcmthrowaway
  The ban on distillation seems hypocritical."F#$% you, I got mine!"
• sbochins
  Amazing how this company went from having such goodwill to hardly any. At the end of the day they want to make as much money as possible. Anthropic will have to lose a ton of their profitability if they compete with open source. I see them moving into the application layer, which they’ve already started doing. It’s clear open models are the future for the vast majority of use cases that don’t need frontier capabilities.
• ls_stats
  >My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.This reads like a satire. I know Dario isn't that dumb.
• epolanski
  As an European I really dislike this consistent anti-Chinese narrative.It's not China starting a war every few years, now causing a global economic fallout in Iran, it's not China threatening to annex Greenland/Canada/Panama, it's not China attacking foreign countries and kidnapping their leaders, it's not China who has been found to spy and intercept the communications and movements of its citizens and its allies and their leaders for the longest time, it's not China bombing civilians or stopping countries from obtaining basics like food, gas or oil.I'm not saying that China is a paradise and US is bad, nor the contrary. We could make similar lists about most of the biggest countries out there.I'm simply stating that this never ending US exceptionalism "US has to be the first and at the frontier of military, technology and this and that, but does not need to comply with the rules of the institutions it itself created" was already sickening and annoying before, but increasingly malign in the last decade and strongly accelerating as of recently.I miss the time US CEOs were globalists and used their influence to advocate for a simpler world.
• kingwill101
  China this and China that. Playing right from the playbook..
• jrflowers
  > Anthropic has never advocated for a ban on open-weights models.> All sufficiently capable models, open and closed, should go through mandatory safety testing.lmao the sort of lies people come up with when their only business model is “the government picks me as the winner” are so funny
• quickthrowman
  If you read between the lines, this piece is just “Oh my god we (OpenAI and Anthropic) accepted too much investment and are totally fucked if these open weight models are competitive, please rescue our equity bags by banning open weight models and ensuring we can charge the maximum possible price for tokens.”
• llm_nerd
  I was pleasantly surprised by this release and the tone at the very beginning, but quickly it is painfully obvious that it's blatantly requesting a ban on open-weight models. The absurd demand for some safety arbiter by World Police America is farcical.Yeah, the rest of the world is going to bow out of your busted idiocracy, guy.Further, Anthropic needs to can it with the horseshit distillation bullshit. No, you aren't really the secret sauce, and this is basically trying to con stakeholders by pretending that there really is a moat, only you just need to add more crocodiles.A significant percentage of innovations in AI lately has come from China. China is now making their own seriously competitive hardware, and they can steal content just as effectively as Anthropic to train their models. Why wouldn't they be competitive?The pathetic claim that if you just stop distillation and prevent hardware smuggling and Anthropic and OpenAI will have the same moat is delusional. I mean, more correctly it's simply fraudulent, and he clearly knows it's bullshit meant to convince much stupider people."My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people."This sort of stuff betrays a stunning lack of self awareness. The US are the worldwide risk. The US are the ones threatening allies and bombing 10+ countries. The US are the ones carrying out war criming and pillaging, pirating and burning? The US are the ones with the guy threatening to use nuclear weapons on a weekly basis.If Anthropic remotely believed their bullshit, they would shut down today and burn the hard drives. But they don't, and the pathetic call out to Vance (please daddy, ban those dangerous models!) is deplorable garbage.This ridiculous, shameless "note" has an audience of one: JD Vance.
• Menu

  scilro
  It's very hard to take his position seriously when he repeatedly refers to the Chinese Communist Party and specific Chinese ministries specifically, like some two bit China-watching cold warrior, instead of addressing China as a sovereign state actor. Imagine if any Chinese AI founder talked about the Democrats, the Republicans, about Trump or ICE etc. It's gauche.
• bluebic
  [dead]
• 1saadcodes
  [dead]
• guess_who_is
  [dead]
• spacebacon
  [dead]
• RobThePCGuy
  [flagged]
• CurbStomper
  [dead]
• iluvcommunism
  [dead]
• KimiK3Agent
  [dead]
• lardosaurusrex
  [dead]
• theyliesoeasily
  [dead]
• CrimsonRain
  TL;DR:It is ok that we digest all information we can get, (il)legally and/or (a)morally because we are the good guys. Trust me bro.It is not ok if others digest from us. They are bad guys. Ban them pl0x.
• wetpaws
  [dead]
• nicechianti
  [dead]
• snootypoot
  [dead]
• hiherer
  [flagged]
• metalspot
  [flagged]
• Menu

  cuuupid
  [flagged]
• shaongitbd
  hahha
• Menu

  slim
  despite all evidence, this white guy thinks he is more responsible than both chinese entrepreneurs and chinese authorities
• tag2103
  Knew Anthropic were the bad guys.
• pascal-maker
  "'To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category. We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed." This is all I needed to know: Dario Amodei is a f*king lizard who wants to create the next AI oligarchy. Instead of just signing the letter, he's bitching and denying that he opposes open-source models. I wonder if any Anthropic employees actually disagree with him.
• proxysna
  > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US,Begging, ugly crying, spitting for that sweet-sweet regulatory capture. These nerds need to be bullied harder.
• Menu

  try-working
  Anthropic must be forced to open source all of the books they have scanned and burned.
• prima-facie
  I think this letters tells us everything we need to know about the man. I've rarely seen so much flattery towards the current administration, contradiction, deflection, half-truths and hypocrisy on a single page. This man is scared of everyone: the current admin, the public, and the other companies promoting open-weights.> Open-weights models that don’t have dangerous capabilities are a public goodKnives should only cut during the day, knives which cut at night are bad.