How to Forecast Token Spend Before an AI Project Starts

Your sponsor wants one number on day one: what will the model cost per month? You can forecast token spend before anyone writes a line of the product, using token counts from real inputs, an expected volume and honest multipliers. By the end you will have a counting script and a low, expected and high range for the AI project budget.

1. Collect 50 real inputs before you forecast token spend

Pull about 50 real inputs from the process the AI will take over: tickets, emails, contracts, call transcripts. Pick them at random and keep the ugly ones, because long inputs drive the bill.

Save each as a .txt file in samples/. Save your draft system prompt as system.txt, and include anything else sent on every call, such as tool definitions. That fixed prefix is input you pay for each time.

2. Count tokens before launch with the provider's tokenizer

Count with the tokenizer of the model you plan to use, because tokenizers differ. Anthropic's token counting endpoint is free, accepts system prompts, tools, images and PDFs, and returns input_tokens. Its docs call the count an estimate that can differ slightly from billing. For OpenAI models, tiktoken counts locally with no API key.

# count_samples.py
import argparse
import statistics
from pathlib import Path


def make_counter(engine, model, system):
    if engine == 'anthropic':
        import anthropic

        client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

        def count(text):
            response = client.messages.count_tokens(
                model=model,
                system=system,
                messages=[{'role': 'user', 'content': text}],
            )
            return response.input_tokens

        return count

    import tiktoken

    enc = tiktoken.get_encoding('o200k_base')
    return lambda text: len(enc.encode(system + '\n' + text))


parser = argparse.ArgumentParser()
parser.add_argument('samples', help='folder of .txt files, one real input each')
parser.add_argument('--system', required=True, help='file with the system prompt')
parser.add_argument('--engine', choices=['anthropic', 'tiktoken'], default='anthropic')
parser.add_argument('--model', help='model ID you plan to use (anthropic engine)')
args = parser.parse_args()
if args.engine == 'anthropic' and not args.model:
    parser.error('--model is required with --engine anthropic')

system = Path(args.system).read_text(encoding='utf-8')
count = make_counter(args.engine, args.model, system)

prefix = count('.')  # system prompt plus one filler token
sizes = [count(p.read_text(encoding='utf-8')) - prefix
         for p in sorted(Path(args.samples).glob('*.txt'))]

cuts = statistics.quantiles(sizes, n=20)  # 19 cut points, 5% apart
print(f'samples:      {len(sizes)}')
print(f'fixed prefix: {prefix} tokens')
print(f'input mean:   {statistics.mean(sizes):.0f} tokens')
print(f'input p50:    {statistics.median(sizes):.0f} tokens')
print(f'input p90:    {cuts[17]:.0f} tokens')
print(f'input max:    {max(sizes)} tokens')

The script subtracts a one-token baseline to split the fixed prefix, which can be cached, from each task's own input. Run it like this:

pip install anthropic tiktoken
export ANTHROPIC_API_KEY=<your-api-key>
python count_samples.py samples/ --system system.txt --model <model-id-you-plan-to-use>

# no API key needed, but only a sanity check for non-OpenAI models
python count_samples.py samples/ --system system.txt --engine tiktoken

Use the mean for your expected case, because cost is linear in tokens and a few long inputs pull the mean above the median. Use p90 for the high case.

3. Estimate output tokens separately

The counting endpoint counts input only, and output tokens usually cost several times more each. Before a build, write ten reference answers by hand in the format you expect and count them with the same tokenizer. If you plan to use a reasoning mode, measure with real calls, because thinking tokens bill as output.

4. Add multipliers for retries, agent loops and cache hit rate

Three multipliers do most of the damage:

• Retries: the share of calls repeated after a timeout, bad JSON or a failed validation. Nobody can measure this before a pilot, so label it an assumption.
• Agent loops: each step re-sends everything before it, so six steps cost far more than six times one step.
• Cache hit rate: prompt caching bills cache reads at a fraction of the input price (0.1x on most Claude models) but charges a premium on a miss (1.25x for the default 5-minute cache). Entries expire after 5 minutes without use, so a tool called a few times an hour mostly misses.

# forecast.py
from dataclasses import dataclass
from typing import Optional

IN_PRICE = 3.00    # USD per 1M input tokens (example, use your model's price)
OUT_PRICE = 15.00  # USD per 1M output tokens (example, use your model's price)
WRITE_MULT = 1.25  # 5-minute cache write, relative to the input price
READ_MULT = 0.10   # cache read, relative to the input price (varies by model)


@dataclass
class Scenario:
    tasks: int                  # tasks per month
    prefix: int                 # fixed tokens per call: system prompt, tool definitions
    variable: int               # the task's own input tokens
    output: int                 # output tokens per call
    steps: int                  # model calls per task (1 = no agent loop)
    step_growth: int            # tokens each step adds to the history
    retry: float                # share of calls repeated after a failure
    cache_hit: Optional[float]  # share of calls reading the prefix from cache, None = off


def monthly_cost(s: Scenario) -> float:
    if s.cache_hit is None:
        prefix_mult = 1.0
    else:
        prefix_mult = s.cache_hit * READ_MULT + (1 - s.cache_hit) * WRITE_MULT
    # call k re-sends the history of the k-1 calls before it
    history = s.step_growth * s.steps * (s.steps - 1) // 2
    input_tokens = s.steps * (s.prefix * prefix_mult + s.variable) + history
    output_tokens = s.steps * s.output
    per_task = (input_tokens * IN_PRICE + output_tokens * OUT_PRICE) / 1_000_000
    return per_task * s.tasks * (1 + s.retry)


scenarios = {
    'low':      Scenario(12_000, 2_400,   650, 120, 1,   0, 0.02, 0.90),
    'expected': Scenario(20_000, 2_400,   900, 150, 3, 700, 0.05, 0.70),
    'high':     Scenario(35_000, 2_400, 1_900, 250, 6, 900, 0.15, 0.30),
}

for name, s in scenarios.items():
    print(f'{name:>8}: ${monthly_cost(s):,.0f} per month')

Take prefix and variable from the counting script, output from the reference answers, and steps from your workflow diagram. The history term is conservative because it bills the growing context at the full input price. Replace the example prices and cache multipliers with those on your provider's price page.

5. Present a low, expected and high LLM cost forecast

With the example inputs above, the script prints roughly the following:

| Assumption | Low | Expected | High | | --- | --- | --- | --- | | Tasks per month | 12,000 | 20,000 | 35,000 | | Task input tokens | 650 | 900 | 1,900 | | Output tokens per call | 120 | 150 | 250 | | Model calls per task | 1 | 3 | 6 | | History added per step | 0 | 700 | 900 | | Retry rate | 2% | 5% | 15% | | Cache hit rate | 90% | 70% | 30% | | Monthly cost | $65 | $646 | $5,486 |

Give your sponsor one sentence: plan for about $650 a month and approve a ceiling of $5,500. The high case stacks every bad assumption at once, so it is a ceiling, not a prediction.

Then change one input at a time to see what moves the number. Raising agent steps from 3 to 6 alone takes the expected figure from about $646 to about $1,690, the biggest single swap from expected to high. That is where the design conversation belongs.

Gotchas

• Token counting cannot predict your cache hit rate. That stays an assumption until a pilot.
• Prompts shorter than the model's minimum cacheable length (512 to 4,096 tokens, depending on the model) are not cached at all.
• Recount whenever the model changes, since the tokenizer can change with it.
• Fifty samples will miss the rare giant input. Set a maximum input length in the design and budget for that cap.

A forecast that shows its range and names its assumptions is the only budget likely to survive first contact with production traffic.