Qwen 3.8 27B

<- Back

Qwen 3.8 27B

erdaltoprak

Comments (601)

CMay
Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it.Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks.The VRAM usage seems way less efficient than Gemma 4 or Glimmer though, with 32K of context taking 2.5GB of VRAM. With those, even with MTP or a DFlash model loaded, you could still fit 256k-768k of context. With Qwen 3.8 27B I can't even fit 128k if I quantize V to Q4_0. Maybe with some trial and error I can find some settings that perform well enough with a larger context window that it's still useful for longer tasks.Lots more testing to do, though I was getting some decent results out of Muse Glimmer which was more than twice as fast and supported huge context windows, managing to solve some bugs that Gemma 4 struggled with. I can't even begin to throw that task at Qwen, because just the prompt alone would use the entire context window and then it would reason for probably that same amount.If you've got a 32GB card, it should be a decent model even if it really is memory hungry.EDIT: Tried a few kv cache quantization settings, but it failed with those. I designed this benchmark to be pretty brutal in the face of KLD and any reasoning quality loss, so it's not too surprising. Gemma 4's QAT held up pretty well, at least and could consistently complete it.
simonw
Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare.(No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a small chainring.")I ran that on an M5 Max MacBook Pro using LM Studio and their 17GB GGUF: https://lmstudio.ai/models/qwen3.8It took 21 minutes(!) and used 22,276 reasoning tokens to produce 3,223 tokens of output.(For the "they're training on your benchmark now" crowd, all of that cheating didn't prevent it from spending 20 minutes thinking about the task first! You can see the reasoning trace in the link I shared.)For comparison, here's one I got from qwen3.8-2.4t-a95b on OpenRouter, which is pleasingly animated: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
dofm
There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for"."Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman.I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which seem to perform poorly.Other notes: it uses the trick of repeating the prompt in the thinking trace.It also worries about hidden chain of thought appearing in the final answer. It talks about "desired oververbosity 9", which is new. A bit GPT-ish.It is being extraordinarily thorough in thinking through one of my code requests, but I don't know if the net result will be any better than the 35B MoE.I asked it to ask me clarifying questions — it did, and it offered me a list of defaults I could simply agree to.I don't think it is necessarily overthinking in the looping sense, but it is in the being exhaustive sense. I need to explore how it does with a tighter reasoning budget.I am impressed but I am definitely in Camp Please-35B-A3B-When? here, because on an M1 Max this isn't really practical. I hope they do one, though I think they may not.
kimsey0
If anyone else is running this on an RTX 5090, https://github.com/Neroued/ninfer as inference engine gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup.
satvikpendem
As usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc.[0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
svdr
Wow. This model is so good, and we have GLM 5.3 (seems great voor security related work) and Deepseek. In a few months we'll have Fable/Sol-like capabilities that are not coming from the big US companies. I feel as a programmer that that is more than enough. How wil OpenAI and Anthropic survive when frontier model intelligence becomes commoditized?
onlyrealcuzzo
If the benchmarks don't lie, this is getting very close to Opus 4.6 capability - which was the turning point for me for when AI was "good enough" that it became very hard to justify not using it.I'm sure there's some benchmaxxing going on, and some things you get only with a a larger model.But I'm feeling pretty confident if not by Gemma 5 than by mid 2028 we'll have local models that are almost always as good as Opus 4.6 was and in many cases far better.
Casteil
One thing a lot of people don't seem to factor when hyping Qwen is how much models like this tend to 'overthink' with seemingly endless 'second guessing'. 3.8 seems no different from what I've tried thus far.As capable as it is, it's hard to justify using it when a competing model (e.g. Gemma4:26b-a3b) can consistently achieve the same or similar response with only 1/10th as many 'thinking' tokens, achieve much higher tokens/second, and take a small fraction of the time. I suppose 'YMMV' depending on your use case.Also, I haven't used it enough yet to see if it's prone to infinite looping, but its predecessors sure were.
LeBit
https://xcancel.com/Alibaba_Qwen/status/2088280182356611304
jjcm
Image->html test for this.Original images: https://image.non.io/neonRamenDesigns.webpQwen 3.8 build: https://html.non.io/neonRamenQwen3.8-27bOverall I'm very impressed with how well this did. It's a big improvement over 3.6, and it feels on-par with some much, much larger models. I think this one is on-par with Gemini 3.7 Flash.One thing to note - the build for this on my RTX 6000 pro blackwell took a long time. Easily one of the longest builds I've done. It took around 2 hours to build the site. Obviously we'll have some quants for this soon that will accelerate things, but I was still surprised with how long it took.Comparison builds from this week:https://html.non.io/neonRamenGemini3.7 https://html.non.io/neonRamenGLM5.3 (note: non-multimodal)
ramon156
People will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowadays are "good enough". I cannot complain because I'd rather spend that time improving my prompts and docs. Opus might be a _slight bit better_ at picking up vague hints, but it's also extremely expensive, and I hit the 5 hour limit way too quick.I care a lot about speed and efficiency right now. For my setup I would like to have 2-3 different model families. I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting, Deepseek V4 Pro 0813 for developing, and Gemini flash lite (any recent cheap model) for repo scouting. I'll add another one in the mix for reviewing (in this case Gemini 3.7) and that's all I need.I've tried most models except Grok.Qwen is too expensive IMO (Alibaba Cloud subscriptions are hard to come by and I'm not spending 50 euros a month for a tool, so 18 euros it is). If it ever becomes efficient enough to run locally I will definitely look back.Claude is slow and expensive (the cache hit prices are absurd).OAI is pretty good, I might add it to my arsenal seeing how cheap it is.These opinions change every day. Last week I would've never picked Deepseek until I read about the pricing. even post aug 16 it's worth it (although it's getting close to gemini pricing).Right now my costs are 12 euros a month (z.ai) + whatever deepseek consumes. This typically isn't more than 8 euros a week. 44 euros a month and I have a setup that is doing pretty well.
xlayn
The unsloth Q8kxl https://huggingface.co/unsloth/Qwen3.8-27B-GGUF for some reason is looping and going crazy on the think part (I tried to search for an email to let the guys know but didn't find one)... I used the bartowski one and that one doesn't have that issue https://huggingface.co/bartowski/Qwen3.8-27B-GGUF?show_file_...that's using llama.cppllama-server \ -m ~/somePath/Qwen3.8-27B-UD-Q8_K_XL.gguf \ -np 1 --kv-unified \ -fa on --no-cache-idle-slots --reasoning-preserve \ --temp 0.2 \ --spec-type draft-mtp,ngram-mod --spec-draft-n-max 3 --spec-draft-n-min 1 \ --cache-type-k f16 --cache-type-v f16 \ --chat-template-kwargs '{"preserve_thinking": true}' \I tried playing with all the recommended parameters from the unsloth page with no luck...in one of the high fever ramblings it ended with amen... lol
KronisLV
I hope really badly that we'll get a new 35B A3B or similar MoE model!I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model <100B would be the sweet spot (when you have the VRAM but not the TDP or compute power). Heck, I'd gladly take A5B or A8B or even A10B as a sort of middle ground.Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304
jedbrooke
I hope the bonsai team makes another 1bit quant of this model (or releases code/instructions on how to do it), using the Qwen3.6 27B on my 16GB mac mini has been wild . The 1bit quant feels like opus level… for the first couple turns. Then it has trouble eg switching from plan mode to act mode. This is mostly mitigated by starting a new session. (tbf this limitation is called out on the hf page)I saw unsloth has 1bit quants too so I might check that out, anybody have experience with those?
swalsh
WOW, my first try running on my 2 3090's, it was a bit slow... but it FEELS like opus 4.5, i gave it an image and a broad overview of what I wanted it to build, and it built the whole thing from beginning to end.
hypfer
Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM.llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spec-default --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 --threads 24 --jinja --reasoning on -fit offIdentical to the qwen3.6 config. With a prompt like "svg owl" (which can reuse quite a lot compared with creative writing or similar, so ngram-mod shines), I get about 70-80t/s like this, with a memory overclock of about 1.5GHz
scrlk
Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch.Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
T0mSIlver
Unsloth Q4_K_M on a single 3090, llama.cpp "Generate an SVG of a pelican riding a bicycle" first try https://www.reddit.com/r/LocalLLaMA/comments/1voa3ch/comment...
TomGarden
Any tips on the best approach at running this at an M4 Max 128GB? Token throughput was a bit slow with the last 27B one (MLX), ended up using the A3B variant but if I could get this one to reasonable speed I'd much prefer it.
Almondsetat
The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help
spijdar
I wonder how this practically compares with Muse Glimmer, especially quantized.I've got an RX 7900 XT (20GB of VRAM) and I can run glimmer with a full 128k context window with the draft model at 65-80 tok/s.This model, on the other hand, I get about 30 tok/s with a 30k context. Raising the context or loading the draft layers for MTP drops performance to 9-15 tok/s.So I wonder how big the "real world" delta between Glimmer and Qwen is here. I can already run 3-bit DSv4-flash at 9-15 tok/s with 100k~ context, and I suspect it would outperform 4-bit Qwen 3.8 27B here.I'll have to experiment and see if I just made a stupid mistake somewhere, but it looks like Glimmer might make more sense for the comically specific niche of "20GB VRAM".
xlayn
The file "Just loads" on llama.cpp, the Unsloth https://huggingface.co/unsloth/Qwen3.8-27B-GGUF is an MTP file, I see mostly the same speed on pp and generation. There has to be something wrong with those benchmarks, I find extremely hard to believe a 27B model can work similar or exceed opus 4.6.
monkmartinez
Qwen3.6-27B has been the main LLM powering my little agentic stack. I have adopted the test and verify approach to any models allowed to run on my machine. When the "heretic" version drops, I will fire up the harness and test. Super excited to see how it stacks up against Qwen3.6!!!
z_rho_one
Beating or comparable to Opus 4.6 in benchmarks. Opus 4.6 was released in February, 2026. So if we still want to talk about a "6 month difference" between Chinese and American AI, the sentence should now be:Chinese (small model) AI is 6 months behind American (largest model) AI.
erdaltoprak
This is one of the most important model releases since most use cases don't need SOTA/FrontierIf you want Qwen3.8-27B Serving Configs for the DGX Spark vLLM NVFP4 and RTX 4090 llama.cpp GGUF I added the setups here https://x.com/ErdalToprak/status/2088299678085308761?s=20
singingtoday
Played with this on my Mac a bit today. Not bad!
NorwegianDude
If the benchmarks are a real indication, we now have a local model that is runnable on a high-end personal PC that trades blows with the leading model Claude Opus 4.6 Max from half a year ago.Insane if that is the case. Downloading now!
jhonof
Yeah this is the first model I have been able to run locally that actually feels useful, this is unreal I am considering cancelling my claude sub and going to just api (maybe GLM?) for really hard tasks.
tosh
27b dense model at Opus 4.6 levelOpus at homeI hope there also will be a new ~10b variant
c7b
For those commenting on the long reasoning, it may be interesting to know that the reasoning effort is set to xhigh by default [0]. Other possible values are medium, low and none. Flag for changing it in llama.cpp below, but note that the long reasoning seems to contribute a great deal to the quality. --chat-template-kwargs '{"preserve_thinking":true,"reasoning_effort":"medium"}' [0] https://unsloth.ai/docs/models/qwen3.8#thinking--preserve-th...
Balinares
I wonder if Anthropic and OpenAI possibly missed the window to go public. A 27B open-weight model trading blows with the SOTA from just half a year ago is not great news for trillion-dollar investments...
Anonyneko
Is there any way to turn off thinking if I'm using Ollama? In my particular case, the Ollama API (the software I want no-think for is tied to Ollama's bespoke API). If not, I'll stick to 3.6 for the time being...
chvid
These are massive improvements - and something you can actually run on a laptop.
minimaltom
Architecture thread! Afaict they continue to use gated attention + delta net, which was also adopted+adapted by K3, but im surprised theres no improvements to the residual stream (deepseek are using manifold hyper-connections, kimi have attention residuals) ?Perf improvements seem to all come from training?
seanmcdirmid
I'm struggling to figure out what to use this for. From the intelligence benchmarks in OMLX. If only they would release another MoE model. Intelligence Benchmark Comparison --- Detail --- Model: scottlowry--Qwen3.8-27B-oQ4e-mtp Benchmark Accuracy Correct Total Time(s) Think -------------------------------------------------------------- GSM8K 93.3% 28 30 282 No MATHQA 46.7% 14 30 26.3 No HUMANEVAL 96.7% 29 30 156.5 No MBPP 83.3% 25 30 71.5 No LIVECODEBENCH 43.3% 13 30 1040.4 No Model: stamsam--Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-MLX-oQ4-MTP Benchmark Accuracy Correct Total Time(s) Think -------------------------------------------------------------- GSM8K 96.7% 29 30 51.9 No MATHQA 60.0% 18 30 9.1 No HUMANEVAL 83.3% 25 30 82.9 No MBPP 80.0% 24 30 29.6 No LIVECODEBENCH 36.7% 11 30 283.7 No
ThouYS
I am so happy right now, qwen3.6-27b was an absolute game changer. To see another one in the same league.. phew
natch
Apart from model performance, what harness are people using to come close to Claude Code or Codex workflow styles with tool use, conversations, loops, remote control, etc.?
sheepscreek
Better than Opus 4.6 at computer use? Comparable with it for SWE? Am I reading this right?I’ve heard rumours about AI shops optimizing for benchmarks. I also don’t think Qwen/Alibaba would be crazy enough to claim something unless there is some truth in it. Would love to see a side-by-side with Opus 4.6 on categories where Qwen 3.8 27B aces it.
rcarmo
Hmm. No MoE or active params weights means this will run _slow_
mickeyp
Model benchmarks are useful, to a point, but it is the long tail of things you do with the model that determines if it's good at a wide range of activities. Ant/OAI, to their credit, build their models -- even the small ones -- so they follow instructions and do tool calling well, without the system prompts confusing them. This is especially important for long-horizon tool calling.So one open weight model might "meet" Opus or whatever on benchmarks, but then fail to follow a simple answer format and also tool call correctly. The models are whipped to within an inch of their lives to strictly adhere to their post training quality gates.
kanemcgrath
I think I am going to buy a second rtx 3060, as 27B has been just outside of my range for to long, and this looks like the parameter count tipping point
jacquesm
The default reasoning is set to 'xhigh', if you want to compare with the past or reduce the time (if you can take the hit in output quality) then you can pick 'high' or 'medium' as well.
theanonymousone
I'm wondering whether any provider can offer this for cheaper $/token than the new DSv4 Flash, which is both cheaper and smarter :/Completely local use is a different story, of course.
syntaxing
Would I be surprised there’s bench maxing happening? Yes. But some users also use Q4 quantized and complain how dumb local models are.
kristopolous
q4km is about 48 tps on a 4090. my llama.cpp params are --flash-attn on --parallel 1 --load-mode mmap
cloudengineer94
Having some really good fun with it so far with System Design (ERP) mainly SAP.I did notice if you go beyond Medium he starts overthinking like hell as per usual for a Qwen model.
mraza007
Man what a week, We just had GLM 5.3 that came out and then we had smaller local model Qwen3.8-27B from QwenJust tried using Pi Agent and looks very promising
simonw
Anyone seen this show up in any APIs yet? I'd love to try it out faster than my Mac can run it.
piyh
Qwen 3.6 is ~$2/m tok, 3.8 should be drop in replacement. Gemma 31B is $0.34/m tok. The price differential on these models is massive on openrouter.
cloudengineer94
Been trying out Qwen3.8-27B-Q5_K_S_20GB and it's quite interesting it's behaving very well.Going to give it some coding tasks and see how it goes.We been eating good at LocalLlama this week.
Valdior
I am waiting for Qwen 3.8 MoE - last time 3.6 MoE was better on codding that just dense 3.6.
chillaranand
"Generate an SVG of a pelican riding a bicycle" - generated a promising image at first shot.https://avilpage.com/qwen-3.8-27b.html
TomGarden
Really excited to see what people do with this. 3.7 27B was probably the best compromise between size and intelligence to run on consumer hardware
anon
undefined
anon
undefined
synergy20
I wish this can run directly on my RTX 4090, seems like 30B is the sweet spot for dense model to run locally, sadly RTX 5090 is very expensive and I need a new PC and new power supply(and UPS) to run that, adding a second RTX 4090 is another option, but not sure if my PC can do that yet.
altruios
remember to let llama.cpp catch up to anything new in this model. Save your judgment until about 2 weeks of use.
maherbeg
Does anyone have a https://tenstorrent.com/hardware/cards to try it on?
kunver
Looks like a pretty significant improvement on the DeepSWE benchmark compared to the previous 27B model.
potus_kushner
hopefully for us mere mortals without $4k+ hardware a 35B MOE model will be released. or a new prism ternary bonsai model based on this one.
bertili
Wow. Speed improved as well. 200t/s on a RTX 5090!https://x.com/sgl_project/status/2088281320422322413
webbrain
it is amazing folks! it works like charm with 5090
g023
All this performance at such small model sizes, why are the API fees so high for the AI monopolists on this side of the world?
arjie
I use the Qwens as a vision model for my DeepSeek V4 Flashes to handle. But the Qwens run on old RTX A6000 Ampere. Does anyone know if there's any news about INT4/AWQ quants for the RTX A6000?
yassa9
Can anyone who has that specific personal test he tries on different models , and tries this model , to tell us here if possible , how good or bad is this new model ? compared to others ?I only trust those users genuine personal tests
ThouYS
3.6-27B on little-coder was already mind blowing. looking forward to this guy!
jlkivey
Note: on the model card the comparison to Opus is Opus 4.6 Max, not 4.7
esotericsean
Need to upgrade to a second 3090! Slowly building up my local models with Krea2, MiniMax H3 (and their new Music3), and now Qwen 3.8
tosh
also cool: Qwen 3.8 27b is multi modal!
irthomasthomas
Why don't qwen/alibaba host the model themselves? I was looking forward to trying it on their coding plan. Google are the same way with their Gemma models.
anana_
Monstrous benchmarks! Hoping it is not benchmaxxed.
amazingamazing
With my 5080 laptop so close yet so far to using this stuff
walrus01
Is it just me or is 3.6 27B Q8 K XL (Unsloth) holding up better in sustained token/s rate as the context fill increases over time? The token/s rate seems to be much higher for a time period deeper into context than previously seen.At least as compared to 3.6 27B in the same quantization.
filup
https://news.ycombinator.com/item?id=48403639my prediction was way too far out. 4.6 at home! Woo.
pu_pe
Seems to be SOTA for its size. Hopefully independent benchmarks will come soon.
kunver
Welcome deepseek flash flash!
anon
undefined
expedited123
Kinda was expecting to see Gemma 4 26B in benchmark comparisons :(
crazyemeraldcod
Its so smart!
webbrain
wow
fr2029
Will there be an A4B MoE?
brcmthrowaway
Is oMLX or MTPLX supported?
btbuildem
O joyous day!
gaigalas
Waiting for the MTP version to pop up on Unsloth. Speculative decoding makes a huge difference.Been running quantized 3.6 at 110t/s on a cheap 5060Ti and quite happy with it. If 3.8 improves on it, it would be awesome.
imagetic
Yes.
davidw
I don't know much about the production of these models. How hard would it be to 'fork' something like this and have it not be full of CCP indoctrination?
lossolo
Why weren't the points merged again from the "dupe" thread that had 289 points?https://news.ycombinator.com/item?id=49299684What a weird mechanism. If someone is judging a thread/topic/event impact by the number of points it got, then doing this unfairly degrades that thread.It should have deduped by user and combined the 168(at the time of writing this comment) + 289 points. Just add the twitter link from the previous thread as an additional link in the description, like you normally do, move all the points over, and remove the old thread.
naasking
Can anyone confirm whether this new Qwen release is any more concise when thinking? Overthinking was the biggest (only?) downside of the Qwen models.
tristor
I'm hoping to see folks distill this with current generation Opus / Fable reasoning traces. I have had my best results locally so far from Qwopus (Qwen 3.6-27B w/ Opus 4.6 reasoning distilled). This looks GREAT and I am definitely setting this up later today.
Mr_Eri_Atlov
This is the homelab model hands down
brcmthrowaway
This with ddg mcp to fill in world knowledge. Are local models the future when computer architectures catch up?
brcmthrowaway
My Strix Halo is about to go overdrive!
cmrdporcupine
I found this kind of amusing while running it (using Pi as the harness). Don't know if this is evidence of intense fine tuning from Claude but it smells like it..." The user wants me to explore the repository at XXXX and report back. Let me start by understanding the project structure, reading the CLAUDE.md file, and getting a general overview of what this repository is.Let me start by reading the main project documentation and exploring the directory structure.I'll take a look around this repo. Let me start by getting a lay of the land.read resource CLAUDE.md (ctrl+o to expand)ENOENT: no such file or directory, access 'XXXX/CLAUDE.md'"
ramon156
need another fable uncensored merge with 3.8, really curious what it can deliver
dude3
[flagged]
fintuner
[flagged]
RobertasTa
[flagged]
anon
undefined
anon
undefined
anon
undefined
steffi_oliver
[flagged]
alpha_trion
NICE, i've been waiting for this drop, thanks for posting this
anon
undefined
literoldolphin
Why is anyone even using video cards these days? You may as well be burning cash.This is the perfect candidate for just splattering it on your nvme and then reading it off there and into memory. All of these run perfectly fine on simple m4 silicone:https://github.com/drumih/turbo-fieldfare https://github.com/leonickson1/Swiftlet https://github.com/sqliteai/warp