How to Estimate and Cut Your LLM API Bill

Related: WhatsApp Chatbots for Business: How They Work, What They Cost, and the Rules

AI features have an unusual cost profile. Building them is a one off. Running them is a meter that ticks on every request, forever. Teams that skip the arithmetic tend to meet it for the first time on an invoice.

This guide covers how pricing works, how to estimate a bill before you build, and the changes that reduce it without making the product worse.

How the pricing works

Almost every provider charges per token. A token is a chunk of text, roughly three quarters of an English word. A 1,000 word document is around 1,300 tokens.

Three things set the price:

• Input tokens. Everything you send: instructions, conversation history, documents, tool definitions.
• Output tokens. Everything the model writes back, including any reasoning it does before answering.
• The model. Within one provider, the smallest and largest models often differ by around ten times in price.

Output is the expensive side. It commonly costs four to five times as much per token as input. That ratio shapes a lot of the advice below.

The unit that matters is cost per completed task

Not cost per token, and not cost per request.

A cheap model that gets the answer wrong still bills its tokens. Then it bills the retry. Then there is whatever the wrong answer cost downstream, such as a support ticket, a refund, or a person fixing it by hand. A more expensive model that succeeds first time can be the cheaper option.

Keep that in mind whenever a change looks like a saving on the price list.

Estimating a bill before you build

The formula is simple:

requests per month x (input tokens x input price + output tokens x output price)

The hard part is getting honest token counts. Here is a worked example with illustrative prices. Check your provider's current price page for real numbers.

Example: a support chatbot

• 2,000 conversations a day, 6 turns each
• A 3,000 token system prompt with instructions and product details
• Each turn adds about 150 tokens from the customer and 300 from the bot
• The full history is resent on every turn, which is how chat works by default
• Illustrative prices: $3 per million input tokens, $15 per million output

Across one six turn conversation:

• Input: about 25,650 tokens, because the history grows every turn
• Output: 1,800 tokens
• Cost: about $0.077 of input plus $0.027 of output, so roughly $0.10

At 2,000 conversations a day that is about $6,200 a month.

Now look at where it goes. The system prompt is sent six times per conversation. That is 18,000 of the 25,650 input tokens, and about half the entire bill. The customer's actual questions are a rounding error by comparison.

This pattern is common. In most chatbots the static instructions cost more than the conversation.

The levers, in order

Start with the free wins. They lower cost without lowering quality. Only move to tradeoffs afterwards.

1. Cache the static prompt

Most major providers offer prompt caching. If the start of your request is identical to a recent one, the repeated part is billed at a steep discount.

On Anthropic's API, as one concrete example:

• A cached read costs about a tenth of the normal input price
• Writing to the cache costs a quarter more than normal input for a 5 minute cache, or double for a 1 hour cache
• So a 5 minute cache pays for itself from the second request
• Every read refreshes the timer, so steady traffic keeps it warm all day

Applied to the example above, caching the system prompt alone cuts the bill from about $6,200 to about $3,300 a month. That is a configuration change, not a rewrite.

Two traps:

• Caching is a prefix match. Change one character near the start and everything after it misses. The classic silent killer is a timestamp or request ID in the system prompt. Put anything that changes per request after the cached part.
• There is a minimum size. Many providers will not cache a prefix below a certain length, and on some models that minimum is a few thousand tokens. A shorter prompt simply does not cache, and nothing warns you.

Check the usage numbers in the API response. If cached tokens stay at zero on repeated requests, something is breaking the prefix.

2. Stop resending what the model does not need

• Trim history. Long conversations resend everything, every turn. Keep the recent turns and summarise the older ones.
• Retrieve, do not attach. Send the three relevant paragraphs from your documentation, not the whole manual. See production RAG architecture.
• Cut unused tools. Tool definitions are input tokens on every call. Twenty tools the model never uses still get billed.
• Audit the prompt. Prompts accumulate instructions over months. Many are duplicates, contradictions, or fixes for problems that no longer exist.

3. Control output length

Output is the expensive side, so verbose answers are expensive answers.

• Say exactly what shape you want back, ideally with a short example.
• Ask for structured output such as JSON when a program will read the result.
• Treat the maximum output setting as a safety net, not a length control. The model does not see it. Hitting it just cuts the answer off halfway, and a cut off answer is a failed task.

4. Batch the work nobody is waiting for

Several providers offer a batch tier at around half price. You submit a set of requests and get results back within hours instead of seconds. On some providers the batch discount stacks with the caching discount.

Good fits: overnight classification, bulk summaries, backfilling data, evaluation runs, scheduled reports. Anything where a person is not sitting there waiting.

5. Only then, the tradeoffs

• Smaller models for simple jobs. Classification, extraction and routing often work well on a small model at a fraction of the cost. Test it on real examples first.
• Reasoning settings. Many models let you set how much they think before answering. Lower settings cost less and are fine for simple tasks. Hard tasks need the higher settings.
• Routing between models. Send easy requests to a cheap model and hard ones to a strong model. Note that caches are usually per model, so splitting traffic can reduce your cache hits. Measure the whole effect, not just the price per token.

These change what the model can do. That is why they come last, and why each one needs testing against real examples before it goes live.

Agents cost more than chatbots

An agent that loops through tools resends its growing context on every step. A task with 30 steps can easily use a hundred times the tokens of a single question. Caching matters even more here, and so does a sensible limit on how many steps a task may take.

Measure it

• Log input, cached input and output tokens for every request, tagged with the feature and the customer.
• Track cost per completed task per feature, weekly.
• Set a billing alert and a hard monthly limit on the provider account. A bug in a loop can spend a month's budget in an afternoon.
• Change one thing at a time, and check quality as well as cost after each change.

If your AI bill is growing faster than your usage and you are not sure why, send us the details. Cost reviews are a regular part of our AI work.