How to Add a Fallback Chain to Bedrock AgentCore Gateway

Your agent's model gets throttled at 2 a.m., and every client retry just joins the queue. An AgentCore Gateway fallback chain fixes that with one rule: when the primary model fails, the gateway tries the next destination for you and your client code stays the same.

By the end you'll have a rule that routes a model, falls back on 429s, 5XX errors and stalls, and uses a circuit breaker to stop hammering a destination that is already down. It uses the InvokeModel wire format, so your existing Bedrock SDK code keeps working. Fallback applies to inference targets, which the AWS fallback docs cover in full.

You need a gateway with inbound auth already configured.

Create a Bedrock Runtime target for InvokeModel

The bedrock-runtime connector needs no endpoint configuration. The gateway resolves the regional endpoint and discovers models from the Bedrock catalog.

aws bedrock-agentcore-control create-gateway-target --cli-input-json '{
    "gatewayIdentifier": "<your-gateway-id>",
    "name": "bedrock",
    "targetConfiguration": {
        "inference": {
            "connector": {
                "source": {"connectorId": "bedrock-runtime"}
            }
        }
    },
    "credentialProviderConfigurations": [
        {"credentialProviderType": "GATEWAY_IAM_ROLE"}
    ]
}'

The target only reaches Ready if the gateway's execution role can invoke models and run the catalog scan. Attach this to the role, and narrow Resource for the invoke actions in production:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:ListFoundationModels",
        "bedrock:ListInferenceProfiles",
        "bedrock:ListProvisionedModelThroughputs",
        "bedrock:ListCustomModelDeployments",
        "bedrock:ListImportedModels",
        "bedrock:ListMarketplaceModelEndpoints",
        "bedrock:ListPromptRouters"
      ],
      "Resource": "*"
    }
  ]
}

The Classmethod walkthrough found the bedrock-runtime connector missing from the console's connector options and created it with the CLI instead. If you don't see it, use the command above.

List the models your gateway can serve

Rule conditions and fallback destinations must use model IDs the gateway actually knows, so ask it:

awscurl --service bedrock-agentcore --region <region> \
    "https://<your-gateway-id>.gateway.bedrock-agentcore.<region>.amazonaws.com/inference/v1/models"

IDs come back prefixed with the target name, such as bedrock/us.anthropic.claude-sonnet-5-5. The InvokeModel docs say you can use a listed ID with or without the prefix. Pick a fallback model that accepts the same request body as your primary (more on that in the gotchas).

Create the AgentCore Gateway fallback chain rule

The rule has two parts. A matchModels condition says which requested models it applies to. A routeToTarget action with a staticRoute pins the target, and the fallback block inside it holds the chain.

aws bedrock-agentcore-control create-gateway-rule \
    --gateway-identifier <your-gateway-id> \
    --priority 100 \
    --description "Sonnet first, fall back to a second Anthropic model" \
    --conditions '[
        {"matchModels": {"anyOf": [{"value": "us.anthropic.claude-sonnet-5-5"}]}}
    ]' \
    --actions '[
        {
            "routeToTarget": {
                "staticRoute": {
                    "targetName": "bedrock",
                    "model": "us.anthropic.claude-sonnet-5-5",
                    "fallback": {
                        "circuitBreaker": {"mode": "ENABLED"},
                        "triggerOn": {
                            "statusCodes": ["429", "5XX"],
                            "errors": ["CONNECTION_ERROR", "FIRST_BYTE_TIMEOUT", "UNROUTABLE"]
                        },
                        "firstByteTimeoutMilliseconds": 5000,
                        "targetFallback": [
                            {"targetName": "bedrock", "model": "<fallback-anthropic-model-id>"}
                        ]
                    }
                }
            }
        }
    ]'

The gateway walks the chain in order and attempts each destination once. If every destination fails, the caller gets the last error. targetFallback takes one or two entries, and each entry can add its own triggerOn to decide when the chain moves on from that entry.

Choose fallback triggers and a first-byte timeout

statusCodes takes 1 to 10 exact codes or classes like 5XX, limited to the 4xx and 5xx ranges. errors takes 1 to 3 of these:

• CONNECTION_ERROR: the gateway never reached the provider.
• FIRST_BYTE_TIMEOUT: the destination connected but stayed silent past firstByteTimeoutMilliseconds.
• UNROUTABLE: the target is missing or doesn't support the request path.

A response matching none of your triggers goes back to the caller unchanged, so a 400 from a malformed body won't silently hop to another model.

The timeout covers providers that stall instead of failing cleanly. The docs example uses 5000 ms, but set yours from real latency numbers. On a non-streaming call the answer normally arrives in one piece, so too short a timeout cuts off slow but healthy responses.

Tune the circuit breaker

The breaker is on by default. Closed, requests go to the primary. Open, the gateway skips the primary and goes straight to the next destination, which saves latency and a billed attempt. Half-open, a small share of requests probe the primary to see if it recovered.

Turn it off when failing fast would move you to something pricier than a flaky primary:

"fallback": {
    "circuitBreaker": {"mode": "DISABLED"},
    "triggerOn": {"statusCodes": ["5XX"]},
    "targetFallback": [
        {"targetName": "bedrock", "model": "<fallback-anthropic-model-id>"}
    ]
}

With the breaker off, the chain still walks in the order you declared. UNROUTABLE advances the chain but never feeds the breaker.

Call the gateway and test the failover

Point the stock bedrock-runtime client at the gateway's /inference path. Callers need bedrock-agentcore:InvokeGateway, while the gateway's role holds the Bedrock permissions.

import boto3
import json

client = boto3.client(
    "bedrock-runtime",
    region_name="<region>",
    endpoint_url="https://<your-gateway-id>.gateway.bedrock-agentcore.<region>.amazonaws.com/inference",
)

body = json.dumps({
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Write a haiku."}],
})

response = client.invoke_model(
    modelId="us.anthropic.claude-sonnet-5-5",
    body=body,
    contentType="application/json",
)
print(json.loads(response["body"].read()))

Keep contentType set: on translated routes a missing Content-Type fails with a 400. The caller sees no sign of which destination answered, so confirm routing in the gateway's routing logs, which record the serving destination and any destination the breaker skipped. To force a failure in a test gateway, put a stub behind the primary that you can flip between ok, error and slow, as the Classmethod walkthrough did with a Lambda function URL.

Gotchas

• No fallback once a stream starts. The caller gets the partial response plus an error and must retry.
• Every attempt is billed as a gateway invocation. A failed primary plus a successful fallback counts twice, and Bedrock bills the model cost of each attempt separately. A skipped destination isn't billed.
• Destinations must be distinct. Each target and model combination can appear only once in a rule, and the fallback block needs at least one targetFallback entry.
• The gateway doesn't rewrite request bodies. An Anthropic Messages body sent to an OpenAI-family fallback model is rejected with a 400, so keep a chain inside one payload family.
• Guardrails differ on translated routes. A request with guardrailIdentifier set is rejected there; AWS recommends AgentCore Policy for guardrails that must work across providers.

A fallback chain turns a provider's bad afternoon into a slower, slightly pricier answer instead of an outage, which is what keeps an agent from paging you in production.