Cost-Optimizing a Claude-Based Pipeline Without Downgrading Quality

When AI costs rise, the first suggestion is often to switch to a cheaper model. That can be the right move, but most applications have savings available before model quality needs to change at all.

Step zero: Measure

Log the usage data from every response, grouped by route or feature: input tokens, output tokens, cache reads and cache writes. A single monthly total doesn't tell you where to act. Usually one or two routes account for most of the spend.

Benefit 1: Caching reduces repeated input costs

If requests share a long system prompt, tool definitions or reference documents, cache them. Cached reads cost a fraction of the normal input price. For agents with large, stable prompts, this is often the biggest single saving.

Benefit 2: Batching halves the cost of non-urgent work

Work that nobody is waiting for, such as overnight classification, report generation or backfills, can go through the Message Batches API at 50% of standard prices. Results are returned asynchronously.

Benefit 3: Smaller inputs

• Retrieve relevant passages instead of sending entire documents.
• Summarize or compact long conversation histories.
• Return short, relevant tool results instead of full API responses.
• Remove prompt sections that no longer change behavior, and confirm it with evaluations.

Benefit 4: The right amount of reasoning

Set the effort level for each route. Classification and extraction rarely need deep reasoning, while planning tasks might. Lower effort produces fewer output tokens, and output tokens are the more expensive kind.

Benefit 5: Appropriate output length

Ask for only what you need. "Answer in three bullet points" costs less than a long response you shorten in code. Set max_tokens to stop unusually long responses, but leave enough room for complete answers, because a cut-off answer that has to be retried costs more.

After that: model routing

Once these savings are in place, test whether some routes work well on a smaller model such as Claude Haiku 4.5, and whether any need a larger one. Base the decision on evaluation results.

Compare cost per completed task

A cheaper request that fails more often or needs extra turns may cost more overall. Track the cost of each successful outcome and compare options using that figure.

When optimization isn't worth it

• Small workloads, where the engineering time costs more than the savings.
• High-stakes routes, such as legal or financial outputs, unless evaluations show the change is safe.

Do caching, batching and input trimming first, then start comparing models.