Cache your repeated prompt prefix first, since Claude cache reads cost as little as 0.05x to 0.1x normal input. Then route easy tasks to a cheaper model like Haiku 5.5, and push non-urgent work through the 50 percent off batch API. Together these can cut a typical bill by half or more.
LLM bills do not grow slowly. One feature launch, one long system prompt, and the invoice doubles. The good news is that most of that spend is waste you can remove without touching quality.
This guide covers the three levers that matter most: caching, routing and batching. Prices are current for October 2026, and rupee figures use roughly ₹88 to the dollar.
Where does your LLM API money actually go?
Three things drive almost every LLM bill: how many tokens you send, how many you get back, and which model you pick. Output is the expensive part. On Claude Sonnet 5.5, input costs $2 per million tokens while output costs $10, so a chatty answer costs five times what the same number of input tokens would.
Most apps also repeat themselves. A support bot might resend the same 8,000 token system prompt and knowledge snippets on every single request. You pay full price for that identical text again and again.
The third leak is model overkill. Many teams send classification, tagging and extraction jobs to a flagship model because it was the default during prototyping.
Before you optimise anything, log tokens per request by feature. One afternoon of logging usually shows that two or three endpoints cause most of the spend. Fix those first.
How does prompt caching work, and how much does it save?
Prompt caching lets the provider reuse the processed version of a prompt prefix you send repeatedly. You pay a small premium once, then a steep discount every time the same prefix comes back.
On the Claude API you mark the content to cache. A 5 minute cache write costs 1.25x the base input price and a 1 hour write costs 2x. A cache read costs 0.1x on most models, 0.05x on Opus 5.5 and Sonnet 5.5, and 0.025x on Fable 5.1. Sonnet 5.5 reads recently dropped to $0.10 per million tokens. The 5 minute cache pays for itself after one read, and the 1 hour cache after two.
The OpenAI API caches automatically on recent models, needs a prefix of at least 1,024 tokens, and offers a prompt cache key to improve hit rates. Newer models bill cached input at around 10 percent of the normal rate. Google's Gemini API has implicit caching, plus explicit caching that adds a storage fee per token per hour.
Here is a worked example. Assume a support bot on Sonnet 5.5 handling 50,000 requests a month. Each request carries an 8,000 token fixed prefix, 500 tokens of user text and 300 tokens of output. Without caching, the bill is about $1,000, or around ₹88,000.
Cache the prefix and assume 95 percent of requests hit it. Reads cost $38, the occasional writes $50, the fresh user text $50 and the output $150. The total drops to about $288, roughly ₹25,000. That is a 71 percent cut from one change.
Why is your prompt cache hit rate so low?
Almost always because something changes near the top of the prompt. Caching matches an exact prefix, so one different token early on breaks everything after it.
Put stable content first: system prompt, tool definitions and reference documents. Put anything that changes last, such as the user message, a timestamp or the user's name. A single timestamp at the top of a system prompt is the classic way to land on a zero percent hit rate.
Check the minimums as well. The smallest cacheable prefix on Claude depends on the model, from 512 to 4,096 tokens. A short prompt will simply never cache.
Watch the clock. The default Claude cache lasts 5 minutes and refreshes on every hit. If your traffic arrives in longer gaps, the 1 hour option is worth the 2x write, but only if you expect at least two reads.
Finally, verify instead of assuming. The API response reports cache read and cache write token counts. If reads stay at zero, you are paying the 25 percent write premium for nothing.
How does model routing cut your LLM bill?
The price gap between model tiers is huge. Anthropic lists Haiku 5.5 at $0.10 input and $0.50 output per million tokens, Sonnet 5.5 at $2 and $10, Opus 5.5 at $4 and $20, and Fable 5.1 at $10 and $50. Haiku is a twentieth of the Sonnet price.
There is one quirk. Haiku 5.5 steps up to $0.50 and $2.50 once a prompt passes 100,000 tokens, counting cached tokens, and the higher rate applies to the whole request. Long document jobs can erase its advantage.
Start with static rules, not clever machine learning. Send classification, extraction, tagging and short summaries to a small model. Keep hard reasoning and complex code for a bigger one. If 70 percent of your traffic can go to a model that costs a twentieth as much, blended input cost falls by roughly two thirds.
For automatic routing, RouteLLMRouteLLM is an open source router from the LMSYS team. Its peer-reviewed results hit 85 percent savings on MT Bench at 95 percent of GPT-4 quality, calling the strong model on only 14 percent of queries. Those numbers are benchmark specific, so treat them as a ceiling, not a promise.
LiteLLM is an open source gateway with routing rules, caching and spend tracking across 100+ providers. OpenRouter is the managed alternative with access to many models through one API, but check its fee structure before committing.
My verdict: start with static rules in LiteLLM and skip learned routers until those stop saving money. Then watch your escalation rate. A cascade that quietly escalates 90 percent of traffic costs more than no routing at all.
When should you use the batch API?
Anthropic, OpenAI and Google all give 50 percent off input and output for work that can wait up to 24 hours. Results usually return much faster than the window, though nothing is guaranteed.
On Claude, Sonnet 5.5 batch pricing is $1 input and $5 output per million tokens, and Haiku 5.5 is $0.05 and $0.25. Anthropic states that caching multipliers stack with the batch discount. On OpenAI, Flex processing gives the same 50 percent with variable latency and fuller caching support than its Batch endpoint.
Take a nightly job that classifies 100,000 documents, each 2,000 tokens in and 100 tokens out. On Sonnet 5.5 at standard rates that is about $500, or around ₹44,000. In batch mode it is $250, around ₹22,000. Move the same job to Haiku 5.5 in batch mode and it is $12.50, around ₹1,100, a drop of about 97 percent, provided Haiku handles the task well enough.
Good batch candidates are evals, backlog summaries, tagging, data enrichment and nightly reports. Chat is not one of them. If no human is waiting for the answer, it belongs in a batch queue.
Is semantic caching worth it?
Semantic caching stores past answers and returns one when a new question looks similar enough, so the model is never called. Chatbot workloads often land around a 25 to 35 percent hit rate, and every hit is a skipped API call.
The risk is a wrong answer delivered confidently. Two questions can look almost identical and need different answers, especially when they involve a specific account, date or price.
Use it for public, repeatable questions such as FAQs and documentation lookups. Skip it for anything personalised, and tune the similarity threshold on real traffic before trusting it. Gateways like the one above include a semantic cache, so you do not need to build your own.
Do this last. Provider caching is simpler, safer and usually saves more.
What smaller tweaks trim your token bill further?
Cap output length. Since output costs five times input on most models, a max token limit and an instruction to answer briefly can save more than you expect.
Summarise old conversation history instead of resending it. After 10 turns, a short summary does the job of thousands of tokens.
Trim retrieved context. Sending the top 3 chunks instead of the top 10 often keeps quality the same and cuts input sharply.
Watch long-context surcharges. Some models charge a higher rate once a prompt crosses a threshold, as Haiku 5.5 does at 100,000 tokens. Check your model's rules before you stuff a whole document into the prompt.
In what order should you cut LLM costs?
Measure first, then cache, then batch, then route, then add semantic caching. That order follows effort against return.
Prompt caching needs little more than reordering your prompt, and in the worked example it took the bill from $1,000 to $288. Batch moves are a configuration change for anything asynchronous. Routing takes more work because you must prove quality.
Build a test set of 100 real requests before you move any traffic to a cheaper model. Compare the outputs side by side. If the cheap model matches on at least 9 of 10, route that task. If not, leave it where it is.
Frequently Asked Questions
What is the fastest way to reduce LLM API cost?
Turn on prompt caching and put static content at the start of every prompt. It often cuts input cost by 70 to 90 percent on repeated context.
Does the batch API stack with prompt caching?
On Anthropic, yes, the multipliers stack. On OpenAI, caching in Batch works on newer models, and Flex processing is the fuller option.
Is semantic caching safe to use?
Only for public, repeatable questions. Skip it for personalised or account-specific answers.
How much can model routing save?
Published benchmarks show 40 to 85 percent, but real savings depend on how much of your traffic is easy.
Do cache writes cost extra?
On Claude, yes. A 5 minute write costs 1.25x and a 1 hour write costs 2x the base input price.
