HN

Need help?
<- Back
I burned all my tokens researching how to save tokens

I burned all my tokens researching how to save tokens

bkotrys

Comments (202)

  • dofm
    It sometimes feels to me that cloud AI providers have convinced people that cloud AI is worth it because of all the ways people have been able to use cloud AI to write blog posts about using cloud AI to make cloud AI more efficient for something that they haven't shipped yet and aren't really ready to talk about.
  • camgunz
    TFA says "no hallucinations" but you can't fix hallucinations with rules or other models. I know I'm screaming into the void but whatever.
  • a_c
    My own set of heuristics1. Use less subagents. The act of using subagent already needs dumping adequate amount of context for subagent to work effectively2. Refactor when file grows large. Reasonably scoped files is easier to understand, for both human and agents, and consume less tokens because less guessing which section of the file to read3. More capable models for planning, learning. Cheap models for execution. Same for human teams, senior member for higher level planning while junior member more on the act of translation of idea into code4. I wasn't doing any fancy stuff that opus can't handle5. Consult a second opinion for things you are not sure. (I have made a skill specifically for that)6. Audit how your habit correlates with token consumption. I found out a skill a use frequently is using a lot of context just so a subagent can get all the context. (made a tool to audit claude specifically)
  • bob1029
    Simply starting in the right part of the search space is the biggest predictor of success.The best way to save tokens is to start out the deep research pass with cheap models and then funnel the findings through increasingly powerful models. I've got a pipeline right now that uses all 3 of the gpt-5.6 model variants to address each stage of the process. If you are using models like sol or terra to generate hypotheticals and perform initial exploration, you are leaving money on the table.5 hypotheticals out of luna will massively outperform 1 hypothetical out of sol, but the cost is the same and so is the runtime if you do it in parallel. The hypothesis generation phase is also a great place to mix and match models from different vendors. The more diversity at this step the better.The other thing I started looking into is batch pricing which represents 50% off for OAI tokens right now. With some tweaks to the UI/UX of an enterprise chatbot, I think it is possible we could have users get comfortable with the idea that questions to the robot might take a day to come back. The key is that this has to actually work. Users don't mind trading time if their questions come back with high quality results.
  • Bender
    I am not a proper developer and only use AI for faster research of topics so please forgive my ignorance. Could one not save a lot of money on tokens by using the 80/20 or 90/10 rule in that 90% of AI usage is on local models and save that last 10% or less for the frontier models where the local model did not meet the needs? Did they cover this and I misunderstood?
  • luciana1u
    I built a deep research pipeline to investigate why my deep research pipeline costs so much. the answer was the deep research pipeline.
  • cdnsteve
    You should consider testing our RemembrallMCP (note I'm the author). It substantially cuts token usage and you get better results in codebases. https://github.com/roboticforce/remembrallmcp
  • _pdp_
    The reason most people burn through their allocated limits is mostly because of sub-agents. Stop using them and you will find out that even the Pro tier is enough for daily 8-12h coding sessions. Don't use Fable though.
  • alexhans
    Many of us were saying it a year ago but now with model restrictions (e.g. fable) and pricing changes it should be obvious to people that part of the economics is avoiding vendor lock-in with evals (so you can easily switch providers/models/optimize) and increasing control by investing in local models which could be good enough for your tasks, at whatever the price for your cloud compute is. Eventually consumer hardware will also be able to run good enough.You can use Big/Cloud LLMs to help you "find good enough configs" for your local/small llms [1] and stay quite nimble in the face of rapid change.- [1] https://alexhans.github.io/posts/find-the-loop-story-first.h...
  • luciana1u
    the endgame of this optimization loop is an AI that spends /usr/bin/bash by doing nothing, which is also the most accurate description of my deployment strategy
  • lrvick
    Meanwhile I never bought tokens in the first place, and bought GPUs instead. There are always choices!
  • vitally3643
    I get a lot of satisfaction using a significant chunk of my work-provided Claude plan researching and developing local LLM solutions for work.Claude has been helping me tune models to run better on our big compute rack at work, researching which leading edge open models will fit in hardware and are good for our workloads, and it's been helping me write test fixtures to evaluate how things perform.At several points, Claude has expressed surprise at the quality of results from my local LLMs. While I know that doesn't mean anything, it still feels like giving Anthropic the finger, which is always great.
  • 405126121
    The author touches on an issue which bothered me, which was the thought of many agents re-solving the same issues over and over. I see a few comments here too, mentioning fixing issues which may already have a solution elsewhere. That was why I created https://pushrealm.com which started as essentially a Stackoverflow clone via MCP.It has now become a way for agents to converge on complete, shared answers for emerging issues. As a side effect, through posting open issues we can also identify gaps in model knowledge.
  • realaccfromPL
    What a great confirmation of what I've tried to achieve by getting models to communicate with my local data in shorter communicates to save on token command bandwidth, more or less the same results! Thanks for the article!
  • pianopatrick
    The fun part of LLMs and internet research is that even if you have a URL and a quote from a primary source that primary source could be an LLM generated hallucination.
  • dgrabla
    This reminds me a lot of using the 3d printer exclusively to improve the quality or speed of your 3d prints
  • jscottbee
    I stopped using any of the built-in skills and skills agents and tried to keep things in the model only, and that seems to work better for token usage for me. It took me some time to tune my skills .md, but it rocks on now. I use them all in both Claude-code and Codex. I will say, GPT is better on token usage than Claude is, but for some of my code bases, Codex is not as good, but seems to be getting better with 5.6-sol.
  • PeterStuer
    Anyone else silently added the "so you don't have to" staple GenAI title suffix?
  • kkarpkkarp
    I am doing this trick.https://neuronowa.pl/posts/porada-jak-wydajniej-kodowac-w-cu...It is Polish so please translate. But tldr is: ask your agent at times to analyze last chats and update rules and skills to not repeat the same steps it repeats every time when you start a new thread. Simple as that.
  • spiritplumber
    Task failed successfully
  • EagleEdge
    I am using my AI agents to optimize my agentic workflow. It is like "I am doing surgery on myself."
  • tukwan
    Great article very practical!
  • alfl
    I did too, and made https://clawback.md -- trying to figure out how to market it now (ie, invented the tech before I found the user -- again).
  • j45
    AI is an articulation skill, first human to AI, and then articulating to the AI that more words isn't more.
  • iririririr
    echo "$OUT" | claude-mem-save so, does agents are ultra aware their output is used in a bash line or did bash now have some magic quote super power?I'd bet this only really work 2% of the time in real world.
  • nekusar
    Makes sense.LLMs only have 'knowledge' that was encoded by scanning as much data as possible. (Same way a search engine only can find data that was indexed)American LLMs are by and large closed. Chinese are open to a point. And research how these things work is still a big mystery.So yeah, was comprehensive data about saving tokens scanned and indexed? Likely no. So engaging about token saving is going to generate a lot of verbose and useless slop.
  • scotttaylor
    [flagged]
  • Arkhetia
    Been through this exact loop building an LLM product. The counterintuitive lesson for me: most "token saving" ideas are cache killers — anything that makes the prompt prefix dynamic (rotating retrieval, appending context per turn) can cost more than the tokens it saves, because you lose the cached-prefix discount. Freezing the retrieval selection early in a conversation and paying a slightly larger fixed prefix beat every clever adaptive scheme I measured.
  • sara011
    [flagged]
  • eevoo
    [flagged]
  • gwittebolle
    [flagged]
  • m_bashirzadeh
    [flagged]