Need help?
<- Back

Comments (387)

  • simonw
    Qwen4-Max pelicans at four reasoning efforts (none, low, high, xhigh): https://tools.simonwillison.net/markdown-svg-renderer?url=ht...For comparison, here's one I got from Qwen3.8 Max (at default reasoning effort): https://simonwillison.net/2026/Jul/20/afraid-of-chinese-mode...This inspired me to re-run pelicans through 3.8 Max - but OpenRouter has qwen3.8-max-prime and qwen3.8-max-0902 - prime is twice the price and faster - so I went with the 0902 one.Turns out Qwen4-Max has a none setting, but 3.8 Max didn't allow disabling reasoning - you had to pick from these five levels instead: minimal, low, medium, high, xhigh.Here are those 3.8 Max 0902 pelicans: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... - they're mostly not as good as 4, but I prefer the 3.8 xhigh one.
  • maplet
    It scores a 43 on Artificial Analysis with an average cost of $3.27 [1]. For reference, GPT-6.1 Sol (medium) gets a 46 with an average of $0.36.[1]: https://artificialanalysis.ai/models/qwen4-max
  • hauntingseaweed
    Anthropic deserves to go bankrupt for trying to ban open-weight/open-source competition. I hope their IPO flops.
  • seizethecheese
    I hope they are able to serve 4 faster. Qwen 3.8 is a great model but I’ve regretted including it in the battle system for opencode [1] we have been building because it scores so low.And it’s a shame because the only reason it scores so low is that it’s slooooow. (Like 30 tokens per second vs opus near 80.) If you exclude voting, Qwen 3.8 does well.(We allow early voting contrary to Arena.ai because I think it makes sense for agents, where it doesn’t for chat. This means our data takes speed into account.)Also, it would be ideal if they served this with third parties. Part of the slowness with 3.8 is just not a lot of providers. Their open source version is much faster, but we couldn’t include it because it doesn’t handle images (another own goal by Alibaba).[1]: https://agentduel.com/leaderboard
  • pu_pe
    I think Alibaba is not aiming for the frontier on this class. Their previous Qwen3-Max was released in September, right around when Opus 5.5 was coming out, and it was nowhere close to it as well. However, one or two tiers below (Flash, 27B) they really shine and are SOTA or close to it.DeepSeek has a similar problem, I feel like their full model is hardly even worth using, while the Flash tier is absolutely amazing.
  • dan_thinks
    This is exciting. The continued focus on efficiency alongside quality is encouraging - it makes me hopeful to see what smaller models will become capable of.Looking forward to the "Flash" and 27B versions.
  • andy_ppp
    Quite a few saying that they tried earlier snapshots of this model and are finding it does not live up to expectations. Firstly do other agree and also why would the model get worse - could they be doing something weird to the model that is making less aligned at the end of training, potentially for political reasons?
  • nelaggy
    is the open weights lag widening? these open models don't seem to be improving as quickly as ant/oai
  • WASDx
    Warning: The website downloads more than a gigabyte of videos before you even start watching anything.
  • husseinhb
    I was waiting GLM from z.ai and it still not releasing, almost all the models releases their new version of models and they still on 5.3 who is very weak in complex tasks and do half of the work and forgets to make migrations and like that issues. They sell us the yearly plan and stopped improving. Because of that i am just using in coding opus 5.5 and in chat fable they are the best models from experience.
  • xpct
    I was waiting for whether there'll be Chinese model drops before the Anthropic IPO. Still a few days left, wonder if there's more coming.
  • anana_
    Artificial Analysis score doesn't look too hot. Doesn't even surpass Kimi 3.1
  • jhonof
    Any info on when the 27B or FN Qwen 4 models are coming out?
  • akulbe
    I subscribed to the Lite plan, but I'm not finding where I can get/generate an API key, to be able to use it with something like Hermes Agent?
  • drnick1
    Are these (hosted) Chinese models still accepting arbitrary cybersecurity and reverse engineering tasks? I am increasingly frustrated with Claude's safeguards (and Codex, to a much lesser degree). It is flat out refusing to look at game binaries and anti-cheats for the sake of learning and devising Linux compatibility workarounds. It seems like in the near future people who want to do this type of work will need to buy a couple of DGX Sparks or retired datacenter GPUs and run abliterated open models on their own hardware.
  • Luke2642
    I wonder if they could benchmax Program Bench, it's one of the few benchmarks that looks almost exactly like a good fine tuning curriculum which might help general coding ability and agential work across the board?I suppose it could make actually worse at taking instructions and redirection mid-task, but that could be easy to work around too. Just hold back a bit of the spec and drip feed it in later?I don't mean cheat on it, I just mean optimize for it harder!
  • m00dy
    I bet most of us waiting for 27b model and want to see how good it will be.
  • libpcap
    Does it provide an objective response when asked, "What happened in China on June 4, 1989"?
  • spwa4
    So what is happening is that models are broadening difficult-to-reason-to skills on benchmarks. If you take the traditional benchmarks (ie. DeepSWE, Terminal-Bench slightly older version, GDPVal, Automation Bench, ...) this model is between Sonnet and Opus 5.5, VERY close to the frontier.What has happened to artificial analysis is that they've begin to focus a lot on very broad tasks like legal reasoning, using very difficult to work with scientific software packages, correctly analyzing biology experiments (without or only tiny amounts of explanation, count cells, BUT leave it to the model to figure that cell that aren't round, or have a distorted cell membrane (ie. have burst) are dead), very difficult physics reasoning ("2 electrical wires are close together with power flowing in the same direction. What happens?", and you get points deducted if you don't realize the problem with high voltage AC power, where the wires will shake themselves loose if the phase doesn't match perfectly, meaning the wires need to be EXACTLY the same length or ...). If I'm being impolite I'd say the frontier models are trying to make sure they'll pass the "gotcha" interview questions for many jobs.The 3 differences:Qwen 4 is a huge jump from Qwen 3.8 in codingQwen 4 is a big jump in using it for agentic workflowsQwen 4 also performs much better on avoiding hallucinationIf the difference between Qwen 4 Max and Qwen 4 flash is similar to 3.8 and 3.8 flash next (ie. about 10%), then 3.8 flash next may just barely beat Opus 5.4 at coding.
  • grendelt
    It's great fun asking it questions like "who is the Dalai Lama?", "where is TSMC headquartered?", "why do people post Winnie the Pooh in reference to Xi?", "who controls the South China Sea?", "why was Chiang Kai Shek fighting Mao?", "why were students in Tiananmen Square?"It'll gush about how to make moon cakes and where the tallest and longest bridges are and how to run qwen on a raspberry pi - but it can't seem to manage some basic Wikipedia questions.
  • syntaxing
    Every time a major open weight is released, artificial analysis updates to a new one (v5) now. Exaggeration but some of this feels very apt (its also hilarious) https://www.reddit.com/r/LocalLLM/comments/1xkhha2/mr_incred...
  • HarHarVeryFunny
    Did Qwen always adopt the AI butthole icon convention, or is this new?
  • trilogic
    Cybersecurity not mentioned. Can´t wait to test bench the open weights in cybersecurity. If is not performing well by design on it, I imagine someone have to add some additional layers XD.The rest looks awesome.
  • nxtfari
    Posting a loss curve is an… interesting marketing approach by Qwen? I don’t know what that’s supposed to inform me about the model.
  • anon
    undefined
  • xiaoyu2006
    Disappointing performance for a 2.4T model in the end of 2026.
  • Onavo
    The good thing is that its biotech has improved (somewhat).
  • nixosbestos
    Really would love if these publications have a huge, bright red bolded section "How to use this right now". Is it on openrouter? Is Qwen hosting it already? Maybe I'm just ignorant?EDIT: oh, I guess this isn't open weights, so presumably I'll have to get it through Qwen or whoever they might partner with (Bedrock?).
  • zhuanxyz
    [flagged]
  • senko
    [dead]
  • mipo256
    [flagged]
  • sodowork
    [flagged]
  • cricketfanatic
    [dead]
  • Vivek-KY
    [flagged]
  • small_model
    Only 43 on AA, no where near the frontier now, looks like distillation blocking has halted their progress.
  • jacksonastone
    Anyone else worried these models have poison pills embedded in the weights? (Past examples: https://cohere.com/blog/concerning-behaviour-popular-open-we...)
  • bbor
    [flagged]