Benefits of Idempotency Keys for AI Agent Tool Calls

Your agent calls refund_order, the network stalls, the client times out, and the retry logic does exactly what it was told. The customer gets two refunds. Idempotency keys fix this with one header and one small table: the agent attaches the same key to every attempt at the same action, and the server runs that action once.

Below are the wrapper that builds the key, the server-side store that enforces it, five benefits, and the cases where you should skip all of it.

Derive an idempotency key for each agent action

The key has to stay the same across retries and change between distinct actions. Building it from the run ID, the step number and the tool name does both.

import time
import uuid

import requests

# Any fixed UUID works; it only namespaces the derived keys. Generate your own.
KEY_NAMESPACE = uuid.UUID('6f1c2a0e-3b7d-4c55-9a1e-2d8f0b7c4e11')
RETRYABLE = {409, 429, 502, 503, 504}


def idempotency_key(run_id: str, step: int, tool: str) -> str:
    return str(uuid.uuid5(KEY_NAMESPACE, f'{run_id}:{step}:{tool}'))


def call_tool(url: str, payload: dict, *, run_id: str, step: int, tool: str,
              max_attempts: int = 4) -> requests.Response:
    key = idempotency_key(run_id, step, tool)
    for attempt in range(max_attempts):
        try:
            resp = requests.post(url, json=payload, timeout=10,
                                 headers={'Idempotency-Key': key})
        except (requests.ConnectionError, requests.Timeout):
            resp = None  # we cannot know whether the server ran it, so resend
        if resp is not None and resp.status_code not in RETRYABLE:
            return resp
        time.sleep(2 ** attempt)
    raise RuntimeError(f'{tool} at step {step} failed after {max_attempts} attempts (key {key})')

A retry, a restart and a replay all produce the same key. Two rules make that true: generate run_id once per run (a uuid4() is fine), and save the tool arguments when the model emits them. Never ask the model to regenerate them on retry.

Add a dedupe store on the tool server

Stripe's idempotent requests docs describe the model to copy: save the result of the first request for a key and return it on repeats. This FastAPI version (Python 3.10+) does the same with SQLite.

import hashlib
import json
import sqlite3
import time

from fastapi import FastAPI, Header, Request
from fastapi.responses import JSONResponse

app = FastAPI()
db = sqlite3.connect('idempotency.db', check_same_thread=False, isolation_level=None)
db.execute('''CREATE TABLE IF NOT EXISTS idempotency (
    idem_key TEXT PRIMARY KEY,
    request_hash TEXT NOT NULL,
    state TEXT NOT NULL,
    status_code INTEGER,
    response TEXT,
    created_at REAL NOT NULL)''')


def issue_refund(args: dict) -> dict:
    ...  # call your payment provider here, passing the same key downstream


@app.post('/refunds')
async def create_refund(request: Request, idempotency_key: str | None = Header(default=None)):
    if not idempotency_key:
        return JSONResponse({'error': 'Idempotency-Key header required'}, status_code=400)
    body = await request.body()
    request_hash = hashlib.sha256(body).hexdigest()
    try:
        db.execute('INSERT INTO idempotency (idem_key, request_hash, state, created_at) VALUES (?, ?, ?, ?)',
                   (idempotency_key, request_hash, 'in_progress', time.time()))
    except sqlite3.IntegrityError:
        saved_hash, state, status, response = db.execute(
            'SELECT request_hash, state, status_code, response FROM idempotency WHERE idem_key = ?',
            (idempotency_key,)).fetchone()
        if saved_hash != request_hash:
            return JSONResponse({'error': 'key reused with a different payload'}, status_code=422)
        if state == 'in_progress':
            return JSONResponse({'error': 'original request still running'}, status_code=409)
        return JSONResponse(json.loads(response), status_code=status,
                            headers={'Idempotent-Replayed': 'true'})
    try:
        result = issue_refund(json.loads(body))
    except Exception:
        db.execute('DELETE FROM idempotency WHERE idem_key = ?', (idempotency_key,))
        raise
    db.execute('UPDATE idempotency SET state = ?, status_code = ?, response = ? WHERE idem_key = ?',
               ('done', 200, json.dumps(result), idempotency_key))
    return JSONResponse(result)

The primary-key INSERT is the lock: only one request can claim a key. A repeat with the same payload gets the saved response, a repeat while the first is still running gets 409, and a repeat with a different payload gets 422. Those codes match the IETF Idempotency-Key header draft, which is an expired Internet-Draft (version 07), not a standard, but a sensible contract to copy. With several server instances, move the table to Postgres or Redis; the unique constraint does the same job.

Five benefits of idempotency keys for agent tool calls

1. Retries stop being a gamble. After a timeout the agent can't know whether the call ran. With a key it doesn't need to: it resends, and the server either runs the action or returns the saved result. Stripe describes the same behavior: it saves the status code and body of the first request for a key and returns them on repeats, 500 errors included.

2. Duplicate agent actions stop at the tool. A looping orchestrator, a queue message delivered twice, two workers grabbing one job: they all send the same key, so one side effect happens. One limit: step-based keys won't catch a model that decides at step 7 to refund what it refunded at step 4, because that is a different step. For once-per-run actions, build the key from the business object instead, such as the run ID plus refund:order-123.

3. Resumes are replay-safe. Say the orchestrator crashes at step 9 of 12 and restarts from the top. Steps 1 to 8 send the same keys, get saved results back, and execution carries on at step 9 without resending the customer email. This only works if step numbering is deterministic and the saved arguments are reused.

4. Audit logs read like a story. One key means one logical action. Log the key on every request and note when the response was a replay (the Idempotent-Replayed header above), and "what happened to this refund" becomes a single search. A key with six attempts and one execution tells you that you have a timeout problem, not a billing problem.

5. Mistakes fail loudly. If the model quietly changes the amount from 40 to 45 on a retry, the payload hash no longer matches and the server answers 422 instead of doing something different. Stripe does the same: it compares incoming parameters to the original request and errors if they differ.

When idempotency keys are not worth it

Skip them for read-only tools. get_order and search_docs are harmless to repeat, and a stored response can hand a replay stale data. Stripe's docs say the same about GET and DELETE: a key has no effect because they are idempotent by definition. Writes that are naturally idempotent, like setting a ticket's status to closed, also don't need one.

Limits to plan for

• Retention. Stripe prunes keys once they are at least 24 hours old. Keep yours longer than your longest pause, or a run that resumes after a three-day approval will hit an empty table and act twice.
• The crash gap. If the server dies after the side effect but before the UPDATE, the row stays in_progress and retries get 409. Pass the same key to the downstream provider so it dedupes the real side effect too, which also makes clearing stale rows safe.

An agent that can retry safely is one you can leave running overnight without someone watching for duplicate refunds.