HN

Need help?
<- Back
Is Opus 5.5 nerfed? Thinking patterns changed on November 3

Is Opus 5.5 nerfed? Thinking patterns changed on November 3

rzk

Comments (154)

  • benwills
    I noticed Fable decline in quality a couple weeks ago. And I've noticed Opus drop over the last week, even after bumping to a default of Extra High reasoning for almost everything.Tonight has been maddening with the kinds of absurd mistakes it's repeatedly making. It's so frustrating.I'm the kind of person that wants something to be consistent and reliable. And Anthropic has been great, except for in this way. Which is enough of a recurring frustration (about every couple months this year) that I genuinely can't wait for other options to catch up. Then I'll be leaving as soon as I can.I just don't know how long that will be.
  • rzk
  • Zambyte
    It blows my mind how people still build their workflows around depending on services like these, with how easy it is to just self host and forget about all of this. You can buy a GPU that can run Qwen 3.8 flash next for about 9 months worth of the highest tier Claude Code subscription. You'll never have to worry about token limits, the model changing out from under you, getting banned, etc.
  • ricardobeat
    I noticed the same, and posted about it three days ago.Opus is suddenly fumbling basic tool calls and failing to finish tasks even at high, where I had been permanently using it on low effort with success since it came out. It is very frustrating.My guess is they quantized it and/or reallocated DC capacity to train a new model, this has happened before.
  • mstaoru
    We use Azure Foundry hosted Opus and it seems as capable as ever.I run a small local GPU cluster for Qwen 3.8 Flash and while it's "free" (you should check electricity prices in EU...) unfortunately it doesn't even compare to Opus 5.5 or Astra in capabilities. It's like GPT 6 vs GPT ~4, but imagine GPT 4 is also 100x slower and sounds like an airplane taking off.In the end it's totally possible that I would pay more for electricity running Qwen for 24 hours churning on some complex problem vs. Astra one-shotting it in 15 minutes.I hope we get there, but it's not there yet, and I'm sure never will be with consumer GPUs. Maybe if you spend $20k+ on a local AI rig. There's just no market force to give access to AI to everyone at fair price.
  • shikck200
    Token use exploded a week or two ago. I worked for a few hours and hit my 5 hour limit. Did only some simple bug fix. Somethings definitely broken.This is the #1 thing i HATE with LLMs. They are not predictable like a compiler is.
  • Festro
    Definitely noticed some anecdotal declines this past week or so. Missing steps it didn't previously, needing reminders of tool calls and skills that it should auto apply.Lots of "this is a potential issue, I didn't check into it though" which is super frustrating to see. The model just points out major flaws in logic or barriers but goes no further than a comment on it.I've got a Qwen coder model and a flash next model locally now, so it's not the end of the world. But it does need to be remedied - are we seeing them prep for a new model deploy when this happens? Or are they experimenting with leaner hardware to drive down costs ahead of IPO?
  • Retro_Dev
    As one user pointed out:> I noticed this too as did many others. Had to bump my default reasoning but it’s not the same. Seems to often happen before new model dropsI heard that Qwen4 was dropping some time soon, I wonder if Anthropic is trying to announce a new model immediately after Qwen4 - both for their IPO, and to kill the competition.
  • gherkinnn
    I moved from Claude to Codex and am happier. The prose is less grating and results comparable. As much as I don't like Altman I am tired of Anthropic's antics.
  • ghoul2
    Definitely a significant change. I noticed it in just daily work. Quality of work down, mistakes up, repetitive errors in scripts - rare earlier, now a few every session.
  • prymitive
    Some teams I know started moving all linting and validation into LLMs and markdown files, the reasoning was that they asked agent to review changes that were later found to be buggy and agents flagged these bugs. And that’s great but my immediate worry was that: it finds it now, but will you check every new model for all these bugs? And what if you need to switch providers? Or your provider silently tweaks the model to improve their margins? The answer was “well also build deterministic linters for all these problems”, but you know that’s just wishful thinking - if you can get 99.9% done with just a little bit of fuzzy text and the next 0.1% is a lot of work it won’t happen. So you have model behaviour lock-in that will bite you suddenly when you least expect it. Or maybe LLMs will just keep getting better and cheaper? It’s all a casino right now.
  • xscott
    With open weights, YOU get to decide when the model changes.
  • jarym
    Yes. That was the weekend my token usage blew up and used up my weekly allowance in a matter of hours. It was so stark.After I figured out, like this chap on X, that it wasn't ME, I diversified my model usage and now a lot more work has shifted to Astra/Sol and I'm learning to make my workflows more resistant to the whims of each model.
  • Haven880
    The reverse happen with Deepseek. Getting smarter and faster especially if you dont mind the China API. Gemini Argon getting dumber compare to older 3.8 and significantly slower too.
  • lukax
    I guess it's just capacity problems. They probably have a fixed amount of compute and they need to make space for new models that will be released. So they quantize existing models e.g. in half so that can use 50% of current capacity for new models. They have such massive scale that you just cannot dynamically get that amount of new compute online so they need to work with what they have. No provider has thousands of GPUs idle.
  • orbital-decay
    This is not real model's reasoning chain, they might have just replaced or tuned the model that summarizes it. If you want to track actual degradation you need to run evals. Trackers like this do exist, e.g. https://aistupidlevel.info/
  • DavCreator
  • __natty__
    We need a consistent way to tell level of service for LLM services. Without they can play with us and steal money by providing different/quantized models than we paid for. I believe this may be a good next AI company if they will be reliable.
  • PostOnce
    We are on the cusp of an intellectual obesity epidemic.
  • terminalcommand
    I am using Opus 5.5 high almost exclusively. I definitely felt this and looked for nerfing watch sites. It felt like downgrading to the old version of deepseek. Things have improved a bit now.
  • aurad
    I've seen big drops in Opus 5.5 (on high/x-high) & Fable. Both in terms of following instructions / workflows execution and overall code quality. Also the token burn-rate increased a lot, I've reached limits of 3 Ultra subscriptions during last week & had to spend 1.5k on API usage during the weekend just to be frustrated with the stupidity of the output.Weirdly enough, Sonnet quality seems unchanged.
  • closetheloopdev
    If Anthropic indeed nerfed Opus 5.5 on November 3, it could be because they wanted to be prepared for inference cost going up due to the data center moratoriums and the election results. I really hope this is false, but I don't see how inference cost can go down in the near future.
  • xbmcuser
    From what I understand they are picking and choosing who is running the latest models. So nerfed yes but for you not for everyone
  • setnone
    this is the reason i cancelled claude after 4.6, non-determinism aside, anthropic just doesn't seem care about maintaining sane baseline for their products
  • r_lee
    I'm also noticing the model seems to be a lot more sloppy now and less reliable like compared to a week ago or so?
  • Topfi
    > "One Bug. Two effort levels."Sounds like analysis fully driven by a model, which can be fine, if you know what you are getting into and can verify the output. Just last week had my dad ask Claude Opus 5.5 on High about watermarking after I informed him. The model told him that watermarking text was not implemented in Anthropic models, whereupon he showed me and I then asked him to just ask the model to check again, ideally online. These models do still "hallucinate", they still have major gaps in their output not easily caught if you don't know the topic and interestingly, anything connected to LLMs, model providers, etc. tends to be where they do worst.The annoying thing is, they still sound immensely authoritative across most any topic, but especially LLM stuff, which is such an easy way to get tricked. I have been for sure in the past and will be in the future, it's why I recommend having some incredibly niche hobby (like sewing with less known materials, obscure motorcycles, etc.) to check how quickly the model makes a mistake which no one in the know would make. Just to keep yourself disillusioned and inoculated.In this case, looks like the LLM did not consider that these are (and have been for months now) just summaries and the actual reasoning can not be analyzed for encryption reasons.If in doubt, please do some research independent of the model output. LLMs not having any internal "understanding" that'd give them anything akin to an internalized "feel" for what they "know" accurately enough is one of their major flaws.Funnily enough, once did the opposite, made a manual analysis of a Github outage, then had the model just write an artifact to quickly share. Was lazy, could have just put something on Pages, but taking summary changes and extrapolating that to a "nerf" is less ideal.On the regression front, I run my own, private evals and have only once encountered consistently reproducible, over run-to-run deviation, degraded performance, which was purely present when using Claude Opus 4.8 in April after some system prompt changes in Claude Code specifically which were addressed soon thereafter. These were not present with OpenCode via API and any other time the public on Reddit and co. reported regressions, I could not reproduce it, even when using Claude Code as the harness and a subscription.What I have encountered and been able to reliably reproduce is that certain labs do very clear teach to the public benchmarks and underperform them commonly, whereas others do not. And that certain, especially cheaper and higher throughput, neoclouds very clearly take some quality affecting, tool call misformatting shortcuts.For what it's worth, currently measuring, but haven't seen anything so far with Opus 5.5, especially nothing connected to reasoning that'd make the assessments some of the commenters draw from this apparent.
  • Asmod4n
    Hm, I’ve seen a degradation of 5.5 yesterday. I’ve had a session running for around 50 days and it just started to generate garbage and ignored all rules. A fresh one feels the same as always
  • cft
    Ever since I started using Claude at its inception this was the biggest breach of trust that I've experienced.I'm afraid to have touch my code base because a nerfed model could slip in stupid bugs.This is also why a weaker local model might be preferable because it allows me to calibrate trust, and my expectations don't change.I have had Astra analyze my long running Opus conversations and the degradation definitely happened on November 6th and 7th.
  • jonathanlydall
    Anecdotal, but I tend to get by with medium thinking on Opus and I noticed that as of sometime last week it really did feel like it had become noticeably slower to think about things.I wouldn't say I've especially noticed its effectiveness having been reduced, it's just slower.
  • jwpapi
    Wasn’t there a platform that kept testing that ?
  • sunaookami
    It's just a new summarizer, the model and performance itself is unchanged.
  • anon
    undefined
  • tammyselvi
    not as bad as sol 6.1. im getting better results out of glm 5.3
  • throwaw12
    LLMs are such a great business, they are non-deterministic, hence you can do whatever you want, because customers can't do anything about it:* nerf them based on supply and demand, when you have a high demand, nerf it, but keep charging same price.* do you want to show revenue growth to investors? increase thinking tokens, customers will keep paying, because again, this is non-deterministic system, sometimes it thinks more* do you want to increase profits? start serving 4-bit, 5-bit quantizations and keep charging same price* want even more? publish a paper "Advanced Auto Super Smart Prompt Router - AASSPiRation", and start routing requests to Qwen3.8-27B or GemmaE2B from time to time on the edge* Not enough? introduce "Complex Chained Workflows - CCW" in your coding harness, which spawns multiple deterministic workflows (which you can't stop or control) and they will use Haiku to write the code, then use Opus 5.5 high to verify the code (which is obviously wrong) and Opus 5.5 produces recommendations on how to fix the code, delegates to Haiku, which again fails writing proper code, and keep doing it 5-6 times, you made lots and lots of money (don't forget, all should run in parallel, to pick one best result among 10 outputs).
  • deniska
    People who never tried outsourcing before are discovering that there are risks associated with outsourcing.
  • peri-cl
    Eagerly awaiting Artificial Analysis to redo their Anthropic benchmarks and kick it down 15-20 index points, in the interests of accuracy. Those guys love accuracy. They moved Heaven and Earth to kick Kimi off the #2 spot, out of their deep, abiding passion for having the most accurate benchmarks in the industry.I have no doubt they will ensure their users get an accurate picture of Opus 5.5 within hours.
  • seqizz
    Pretty nice business model tbh. Whenever you need/feel like it, just tune it down.Oh, even psychological effect would be nice if you slightly tune down the current frontier 1-2 weeks before releasing the new one.
  • walthamstow
    I'm skeptical about nerfing. Are we saying they are instructing AWS to nerf the version of the model they're running on Bedrock?If they did, there's no way that remains a secret.If they didn't, Anthropic would be making their own service noticeably shitter than AWS Bedrock
  • Cthulhu_
    This'll likely be the reality moving forward - they allocate more compute at launch to get good test scores, reviews, and news coverge, they get people excited and used to it, then they cut costs.I'm reminded of the video game industry where they would launch a game without any in-game purchases at launch, get the reviews to make note of it and give it a higher rating because of it... then add in-game purchases anyway. I think that stopped again though.
  • bandrami
    Each new model version hits your dopamine receptors very hard when it first comes out, but in the nature of things it stops hitting after a while and you need novel models to fire them off again. That's why people think the models are getting nerfed after a couple of weeks; it's actually their endocrine system getting nerfed.
  • skylurk
  • andai
    The quantization will continue until version number increases.
  • etcetcetcetceta
    I suppose they can achieve a benchmark then nerf the model to save compute once they've got the headline.
  • 0x70dd
    Ok, I'm not going crazy then. Noticed that it is slower and results were of a lower quality.
  • igleria
    as a user I believe it: they are probably reducing assigned compute to prepare to offer new models.That would be the less bad reason.
  • thewhitetulip
    I am curious. How do people figure this out?I use Sonnet 5.5 daily and Opus rarely. My usage is also a little involved as in I do all the design and strategy and get Sonnet to just write the codeFor that, Sonnet has been consistentI use Opus or Fable when it's a dumpster fire situation where we have no time to think evenSo far I've never had an issue with Sonnet even.
  • anonzzzies
    I guess it’s AB testing and we are not on the bad side (yet).
  • amelius
    Nerf means something else in deep learning parlance.
  • listedb
    It's dumbing down a little bit but I don't think its a lot worse. I think the change in display adds to the feel
  • PUSH_AX
    Fable 5.1 quality took a significant dive for me last week. I'm not imagining it.
  • perks_12
    Welcome to the world of post-IPO AI. Now the 20% growth needs to come somewhere, and profitability would be a nice starter.
  • tsukikage
  • gib444
    Does nerfing usually come before a new model release?
  • hebelehubele
    They're preparing to release a new model, so it has to look better than the current one.
  • davidd_1004
    Seems fine to me
  • hugodan
    all good as long as it keeps killing young girls in the middle east
  • Giorgi
    Yes, I noticed it too, dumbing down continues, there goes their IPO.
  • BiteCode_dev
    People are already so used to have genies and unicorns around they now feel infuriated when they are a bit slower than before.Dude, you have a mythical creature at your disposal at retail, chill, it's going to be ok.
  • Madmallard
    Can we get a self-hostable distilled opus 5.5 yet please?This company does not deserve to be successful and has no moral high-ground.
  • 486sx33
    [dead]
  • FastDeep
    [flagged]
  • 1STORM3
    [dead]
  • squidmaster
    [dead]
  • novoreorx
    Is there a practical reason this link uses twitter.com rather than x.com, or is it a nod to the old name? If it’s the latter, I don’t really see the point.
  • fractalf
    5.5 is useless below max effort, just frustrating and plain old wrong