Skip to content
ADVIQ LABS
All insights

AI3 min read

What an LLM feature really costs to run, and how to keep it predictable

Token pricing looks tiny until real users arrive. A simple way to estimate monthly cost before you build, plus six techniques that keep the bill under control.

ALBy the Adviqlabs team

Language model costs are small per request and easy to ignore during development. They add up quickly in production, especially with chat interfaces and retrieval, where every message carries a lot of context. The first real invoice is often the moment an AI demo becomes a business decision.

The good news: LLM costs are very predictable once you model them, and there are well-known ways to reduce them.

How pricing works

Most providers charge per token, with separate prices for input (what you send) and output (what the model writes). A token is roughly three quarters of an English word. Output tokens usually cost several times more than input tokens.

So the cost of one request is:

cost = (input tokens × input price) + (output tokens × output price)

Estimate before you build

Let's model a support assistant using retrieval. The numbers below are illustrative, so replace them with your provider's current prices.

Assumptions

  • System prompt and instructions: 800 tokens
  • Retrieved passages: 5 × 400 = 2,000 tokens
  • Conversation history: 1,200 tokens (average)
  • User question: 50 tokens
  • Answer: 350 tokens
  • Example prices: $1.00 per million input tokens, $4.00 per million output tokens

Per message

  • Input: 4,050 tokens × $1.00 / 1,000,000 = $0.00405
  • Output: 350 tokens × $4.00 / 1,000,000 = $0.0014
  • Total ≈ $0.0055 per message

Per month

  • 2,000 active users × 15 messages each = 30,000 messages
  • 30,000 × $0.0055 ≈ $165 per month

Now change one assumption: conversation history grows because you never trim it, averaging 6,000 tokens. Input jumps to about 8,850 tokens and the monthly bill more than doubles. History and retrieved context, not the question or the answer, drive most of the cost.

Build this spreadsheet before you ship. It takes twenty minutes and answers the "can we afford this?" question early.

Six ways to keep costs down

1. Trim and summarise conversation history

Keep the last few turns verbatim and replace older turns with a running summary. This alone often halves input tokens in chat features.

2. Retrieve less, but better

Sending ten passages "just in case" is expensive and can make answers worse. Improve ranking so three or four strong passages are enough.

3. Route by difficulty

Not every request needs your most capable model. Use a smaller, cheaper model for classification, routing, short answers and extraction, and escalate to a larger model only when needed.

4. Use prompt caching

Many providers discount repeated prompt prefixes such as long system prompts or shared documents. Put stable content first and variable content last to benefit.

5. Cache whole answers where it's safe

For common, non-personal questions ("what's your refund policy?"), cache the final answer for a period. A Redis cache keyed on a normalised question is often enough.

6. Cap output and set budgets

Set a maximum output length that fits the feature. Add per-user and per-workspace rate limits, and alert when daily spend crosses a threshold. A runaway loop or an abusive user should trigger an alert, not a surprise invoice.

Track cost like any other metric

Log tokens in and out, model used and latency for every request. Tie it to a user or workspace. Then you can answer the questions that matter:

  • What does an active user cost us per month?
  • Which feature or customer drives most of the spend?
  • Did last week's prompt change make things cheaper or more expensive?

Price your product with it in mind

If AI is central to your product, include usage in your pricing: plan limits, fair-use caps or credits. Unlimited AI on a flat low price is a common way to end up with your best customers being your least profitable.


Adding AI and want predictable costs from day one? We design LLM features with cost tracking, limits and caching built in. Book a free call.

Start a project

Have a product in mind? Let's scope it together.

Book a free call. We'll talk through your idea, suggest the leanest path to launch and follow up with a written proposal.