HN

Comments (103)

• Menu

  ford
  You're better off going directly to artificial analysis, this is a feature-poor/misleading/outdated repackagingEx. this type of price estimation is quite naive - some models can require 2-3x the number of tokens to achieve the same level of intelligence. Artificial Analysis' own cost per task is a more fair estimation of cost.
• Menu

  Xeoncross
  If you have a 24-64GB mac, consider running Qwen3.8 27B locally at night. It's a bit slower to run locally, but if you're sleeping it's less of a problem.Depending on your memory, you'll need to use the weaker Q4 versions but they still perform well.It ranks higher than GPT-5.3 Codex (xhigh) or Claude Opus 4.6 (max) so is great for pairing with https://github.com/kunchenguid/gnhf for nightly experimentation, cleanup, or recommendation lists for in the morning.
• Menu

  jrflo
  Does anyone actually pay API costs out of their own pocket? It's about 10x cheaper to just get a codex or chat gpt subscription, it's so heavily subsidized compared to the API that I'm sure it would be cheaper to use frontier models on a subscription plan rather than paying API prices for deepseek flash.
• Menu

  Brendinooo
  I'd really like something that's more oriented around subscription fees.If I want to spend $100 on LLMs next month, what should I do? Get Claude because Opus 5.5/Fable 5.1 are scoring well? Get Grok because 4.7 is supposedly a good mix of competence and cost? Try out a Chinese model? Don't do a subscription at all like this site is saying?
• Menu

  greggh
  I've been running a quant/tune of Qwen3.8 27B on my M1 Max 32gb MacBook. That plus a good pi setup is having great results. I've used a full q8 of the model before and I dont see a real difference other than how slow it is. But leaving it running overnight on tasks is working great. It is currently debugging some issues in a native Mac Swift application and getting through the list of issues just fine.This is the one that works good for me on 32gb:https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUFSpecifically this one: Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf
• Tepix
  1. In real life, most of us use token packages like OpenCode Go etc.It would be handy to have a site like this one that takes into account the various deals and attempts to calculate the number of tokens per monthly fee for a chosen model. I realize this makes the task a lot more difficult.2. It would be handy to have a chart like that for the AI hardware that people own. It helps you decide which model to run (resulting in different levels of intelligence and speed). Also difficult to please everyone (preprocessing vs token generation for example) and to keep updated!I found https://llm-list.com/ yesterday and when I had a detailed look, I quickly found outdated entries, for example looking at GLM 5.3 flash it listed several providers as "free" that weren't free any longer.
• floppyd
  As the time goes on it only becomes harder to differentiate between model capabilities with just one or two numbers. I would love to see some kind of multi-axis placement of all the models on less objective attributes, like wordiness, willingness to give up, an ability to "think ahead" and pre-solve possible problems in code, for example, that I didn't think of or didn't think of talking about, etc etc etc.For example I've been really enjoying Deepseek v4.1 Flash, it's very "straightforward" to the point of being almost dumb sometimes, but it's absolutely relentless and would solve almost any problem no matter how inefficient the solution is.No idea how to measure all that, just average CoT length per task is probably a good approximation for some things, but not others.
• qwerty2020
  Cost per token is an extremely naive way to index cost, renders this chart essentially meaningless.
• rmi_
  I wrote (i.e. let AI write) something like this for Opencode Go: https://6bj.de/aaogo/go_value_report.htmlIt compares Artificial Analysis scores against usage limits on Opencode Go, so I can see where to waste my quota most efficiently. Updated whenever I feel like it.
• Menu

  khalic
  More benchmark noise. They’re already problematic per se, and the error bars only get bigger by aggregating and adding costs on top of it.Create your own, private evaluation system that reflects your use and your constraints. Focus on hard cases you encountered, find cases that break on one tier and not on another.
• jwolfe
  This definition of cost is not particularly useful. You want cost per task, not cost per 1m tokens. Artificial Analysis does a good job of this.
• Menu

  pelagicAustral
  Is there a cheaper model than Gemini 3.8 Flash (High) that maybe/kind-of is on-par with it? For me it works really good but hit the limit in two hours tops... last week was the first time I hit the weekly limit and had to wait 4 days... Claude patches OK, but that is also getting drained really fast these days...
• israrkhan
  Some of the charts are not uptodate.For example "Raw capability: Math Index", shows GPT-5.2 as highest. It does not even mention astra, of fable.
• swingboy
  I guess it’s hard to actually calculate, but most people are probably on subscription plans with Anthropic or OpenAI and there’s not any good resources to estimate which models are the most “usage limit” efficient. I suppose you can sort of correlate raw API costs with how much usage a model would probably drain, but it would be nice if there was a reliable place to get that info.
• newsy-combi
  Coding and math graphs are very interesting. Extremely cheap models make it into the upper echelon, delivering 90% of the performance for 1% of the price compared to the #1.
• Menu

  radial_symmetry
  The coding and math tabs seem to be missing the latest models...
• barrystaes
  Fun, but this seems to assume all LLM run on SAAS subscriptions. I would like to compare this to local LLM costs by converting my usage load x hardware costs into a token price. In addition if it games the LLM so often, these eventually are optimized and basically cheat on the score.
• rdsubhas
  > "$0.23 to under $0.24 - Qwen3.8-Flash-Next"There are only 6 categories in the site, but a category entirely created out of thin air for Qwen3.8-Flash-Next?Literally "$0.23 to under $0.24" means EXACTLY $0.23.
• Menu

  ggcr
  What's impeding a lab from releasing its own new model, pricing it really low for the beautiful Pareto plot, accompanied by phrases VC love like "establishing a new frontier in cost", to just then raise prices back up?
• Menu

  jeremysalwen
  It's missing Opus 5.5 which was released over a day ago (and also is clearly on the pareto frontier).
• lasky
  I won’t touch Meta models. I simply don’t trust or like Zuck and especially Wang
• daft_pink
  I would really like something similar for local ai where the x axis is vram.
• verytrivial
  I find it interesting that with the given metric comparison, for coding at min 50 strength, every frontier model brand is from a distinct vendor: Ling, Qwen, Gemini, Muse, Grok, GPT, and Claude in increasing value.
• real0mar
  Updated daily and yet the frontier for math is listed as gpt-5.2
• TomGarden
  If you maintain this over time, maybe include other sources than just AA and update the design to look less like zero-shot claude styling (I know that font! I know that color! Lol) it's genuinely useful :)
• Menu

  aslkalska
  all I ever wanted is an updated website where I can see the best models I can run on my different devices locally, I don't get why people are throwing money at these companies
• Tycho
  Interested to see the Coding and Math scores for Grok 4.7 when they’re available. Grok 4.6 is apparently already on the frontier for Coding.
• FrostKiwi
  The page looks Claude generated and ranks Opus at the top.
• yu3zhou4
  I’d love this but for every hardware
• Menu

  hsnewman
  I'm sure that local LLM will be far cheaper
• someguynamedq
  Astra worse than 5.6 Sol is crazy work
• 7777777phil
  I tried to build a very similar “frontier” [1] for myself and also include writing (which is highly debatable)[1] https://philippdubach.com/posts/jev-model-router-for-pi/
• zephiel7
  Luna at the bottom of intelligence is laughable...
• jeffbee
  When a clanker is slopping up a webpage, why does it have to be #f4f3ee? Curious.
• CurbStomper2
  [dead]
• isurajmisra
  [flagged]