How to Build a Grafana Dashboard for Token Usage and Cost

Model providers bill by tokens, and invoices arrive weeks after the spending. By then, a prompt that doubled in size or a loop that retried all night has already cost real money. Exporting token usage as metrics lets you see costs by route and model as they happen.

Step 1: Record usage as metrics

Every model response includes token counts. Turn them into a Prometheus counter with a few labels:

from prometheus_client import Counter

TOKENS = Counter(
    "llm_tokens_total",
    "Tokens used by model calls",
    ["route", "model", "kind"],
)

usage = response.usage
TOKENS.labels(route, model, "input").inc(usage.input_tokens)
TOKENS.labels(route, model, "output").inc(usage.output_tokens)

If you use prompt caching, add cache reads and cache writes as separate kinds, since they're priced differently.

Step 2: Keep labels low in cardinality

Label by route, model and environment. Don't label by user ID or request ID, which would create a separate time series for every user and overload Prometheus.

Step 3: Convert tokens into money

Keep the price per million tokens for each model and kind in one place, such as recording rules or a dashboard variable. A cost query multiplies the token rate by the price:

sum by (route) (rate(llm_tokens_total{kind="output", model="claude-sonnet-5"}[1h])) * 3600 * 10 / 1e6

This gives dollars per hour of output tokens for a model priced at $10 per million. Use recording rules to combine the kinds into one cost series.

Step 4: Build the panels that matter

• Cost per hour by route.
• Tokens per request by route, which reveals prompts that keep growing.
• The share of input read from cache, if you use caching.
• The five most expensive routes over the last day.

Step 5: Alert on changes

Alert when any route's hourly cost stays well above its usual level for more than 30 minutes. Relative alerts catch runaway loops without constant threshold tuning.

Step 6: Compare with the invoice

Once a month, compare the dashboard total with the provider's invoice. Small differences are normal. Large ones mean a missing route, a wrong price or an uncounted kind of token.

Things to watch

• Price changes. Keep prices in one place and update them when providers change pricing.
• Batch discounts. Batch usage costs less. Label it so the dashboard doesn't overstate costs.
• Retries. Count tokens from failed attempts too, because you pay for them.

Start with the cost-per-hour panel. It's usually enough to spot the first expensive surprise.