HN

Need help?
<- Back
Mercury 2.5 LLM hits 770 tokens per second

Mercury 2.5 LLM hits 770 tokens per second

Retro_Dev

Comments (53)

  • jjcm
    I'm still sad that we haven't seen a new Taalas style chip a la https://chatjimmy.ai/. Smaller models are good enough now to make that insane burst of tokens so useful.
  • bearjaws
    If you care about speed Cerebras gpt-oss-120b is 1400tk/s and "just as smart" in ranking.I've used it on a few for fun projects and its decent but the speed is crazy to watch.
  • the_arun
    Chat Jimmy clocks at 17K tokens per sec burning LLM into the Chip - https://chatjimmy.ai/ - Source: https://theashishmaurya.medium.com/taalas-the-startup-that-p...
  • nextaccountic
    At some point the bottleneck becomes tool calling.. and as such, it's preferably if the model is co-hosted (in the same datacenter, at least) with your code repository and all other reference/context it needs (full documentation for most ecosystems, maybe even a copy of common crawl to minimize web fetch usage, etc)
  • walrus01
    Pricing at $0.25 and $0.75 already puts its cost well above reasonably reputable inference providers for deepseek v4 flash or qwen 3.8-flash-next or similar class of open weight LLMs that fit in under 170GB of RAM, so I don't see the point. I think this is probably also stupider than laguna s 2.1 which can also be very cheap to serve.
  • freakynit
    I have tried using Mercury 2.5 for a lot of my tasks.. but this model just isn't there. It seems to be on par with any 14B model at max. Even GPT-OSS-20B performs way better than this in my own attempts to use it.I really really wanted to use this because it offers incredible speeds and pricing combinations. But nop.. I still am not using it.. not even for basic tasks.
  • nylonstrung
    I honestly think the diffusion LLM approach is a dead endIt's telling that frontier labs like Google toyed around with it but didn't invest further even for their most speed and cost sensitive small modelsStill unclear for what, if any use cases this is pareto frontier
  • seduerr
    Cerebras is fast…?
  • sharktheone
    this feels like "we got the same benches as gpt-oss-120b but are also potentially slower while saying it is great"
  • whalesalad
    I used this a few days ago and thought something must be wrong with how fast it was responding. "Mercury 2.5 is below average in intelligence, but well priced when comparing to other models of similar price." this is so funny. So when you have a stupid model that is fast - what do you use it for?
  • low_tech_punk
    it's stupid fast!
  • entrope
    > Mercury 2.5 is below average in intelligence, but well priced when comparing to other models of similar price.Well priced when compared to other models of similar price, eh?Are we allowed to call this slop, even if the output is not directly from an LLM?
  • pushpendraw
    [dead]
  • rvz
    The speed means absolutely nothing when it is finishing almost dead last when compared to the frontier AI companies.