How to Canary a Model With AgentCore Gateway Weighted Routes

Swapping a model for 100% of your traffic at once is a coin flip with a pager attached. With AgentCore Gateway weighted routes you can send 10% to the new model instead: one rule that splits a model's requests 90/10, a fallback on each leg, and a second rule that pins your QA role to the canary.

This builds on the October 22, 2026 update that added model fallback to Amazon Bedrock AgentCore Gateway. If you haven't built a plain fallback chain yet, read "Amazon Bedrock AgentCore Gateway Adds Model Fallback" first.

You need a gateway with inference targets in Ready state. The examples use four: chat-stable, chat-stable-backup, chat-canary and chat-canary-backup. The gateway rejects a rule that repeats a destination (a target and model pair), so each leg gets its own backup.

1. Create the AgentCore Gateway weighted route

A weighted route takes exactly two legs. Weights run from 1 to 99 and must sum to 100, and leg names must be unique (rules reference). The matchModels condition scopes the rule to one requested model, so callers keep sending the model ID they use today. The canary leg pins model to the new one; the stable leg serves whatever was requested.

aws bedrock-agentcore-control create-gateway-rule \
    --gateway-identifier <your-gateway-id> \
    --priority 100 \
    --description "Canary: 90/10 split for <current-model-id>" \
    --conditions '[
        {"matchModels": {"anyOf": [{"value": "<current-model-id>", "operator": "StringEquals"}]}}
    ]' \
    --actions '[
        {
            "routeToTarget": {
                "weightedRoute": {
                    "trafficSplit": [
                        {
                            "name": "stable",
                            "weight": 90,
                            "targetName": "chat-stable",
                            "fallback": {
                                "circuitBreaker": {"mode": "ENABLED"},
                                "triggerOn": {
                                    "statusCodes": ["5XX", "429"],
                                    "errors": ["CONNECTION_ERROR", "FIRST_BYTE_TIMEOUT", "UNROUTABLE"]
                                },
                                "firstByteTimeoutMilliseconds": 5000,
                                "targetFallback": [
                                    {"targetName": "chat-stable-backup"}
                                ]
                            }
                        },
                        {
                            "name": "canary",
                            "weight": 10,
                            "targetName": "chat-canary",
                            "model": "<new-model-id>",
                            "fallback": {
                                "circuitBreaker": {"mode": "ENABLED"},
                                "triggerOn": {
                                    "statusCodes": ["5XX", "429"],
                                    "errors": ["CONNECTION_ERROR", "FIRST_BYTE_TIMEOUT", "UNROUTABLE"]
                                },
                                "firstByteTimeoutMilliseconds": 5000,
                                "targetFallback": [
                                    {"targetName": "chat-canary-backup", "model": "<current-model-id>"}
                                ]
                            }
                        }
                    ]
                }
            }
        }
    ]'

Replace <current-model-id> with the ID callers request today and <new-model-id> with the model you're trialing.

Each leg carries its own fallback block, because the weighted route has no route-level fallback. The canary's backup pins the current model, so a sick canary degrades to a known-good answer. I listed 429 on purpose: omit triggerOn and the CLI reference says the gateway falls back on 5XX and the three error types, which leaves throttling out. The targetFallback entries omit triggerOn, so they use those defaults rather than inheriting the leg's.

2. Pin your QA role to the canary

Give this rule a lower priority number, which means higher precedence.

aws bedrock-agentcore-control create-gateway-rule \
    --gateway-identifier <your-gateway-id> \
    --priority 50 \
    --description "Pin QA role to the canary model" \
    --conditions '[
        {
            "matchPrincipals": {
                "anyOf": [
                    {
                        "iamPrincipal": {
                            "arn": "arn:aws:iam::<your-account-id>:role/QARole",
                            "operator": "StringEquals"
                        }
                    }
                ]
            }
        },
        {"matchModels": {"anyOf": [{"value": "<current-model-id>", "operator": "StringEquals"}]}}
    ]' \
    --actions '[
        {
            "routeToTarget": {
                "staticRoute": {
                    "targetName": "chat-canary",
                    "model": "<new-model-id>"
                }
            }
        }
    ]'

The gateway resolves each action type on first match in ascending priority, so QA requests settle their routing at 50 and never reach the split at 100. Both conditions must match. There's deliberately no fallback here: testers should see the canary's raw errors, not a polite answer from the backup.

3. Confirm the rules and read the routing logs

aws bedrock-agentcore-control list-gateway-rules \
    --gateway-identifier <your-gateway-id>

aws bedrock-agentcore-control get-gateway-rule \
    --gateway-identifier <your-gateway-id> \
    --rule-id <weighted-rule-id>

Both rules should report ACTIVE. Send traffic, then read the routing logs the gateway vends: they record which destination served each request and when the circuit breaker skipped one. Callers get no indicator of which destination answered, so these logs are the only place you'll see the canary's true failure rate. Your client dashboard will look green even while the backup carries the canary.

4. Ramp the canary weight, then promote or roll back

Raise the weight with update-gateway-rule, which accepts --actions. Keep the same actions JSON in a file and change only the two weights.

aws bedrock-agentcore-control update-gateway-rule \
    --gateway-identifier <your-gateway-id> \
    --rule-id <weighted-rule-id> \
    --actions file://actions-75-25.json

The docs don't say whether the list replaces or merges, so send the complete array, read it back with get-gateway-rule, and wait for the rule to leave UPDATING.

Weights can't be 0 or 100, so the endgame is a different action. To promote, swap in a staticRoute to the canary (add a fallback block back if you want protection). To roll back, point it at chat-stable instead.

aws bedrock-agentcore-control update-gateway-rule \
    --gateway-identifier <your-gateway-id> \
    --rule-id <weighted-rule-id> \
    --actions '[{"routeToTarget": {"staticRoute": {"targetName": "chat-canary", "model": "<new-model-id>"}}}]'

Deleting the rule is a poor exit: if several targets match the model, default routing can pick any of them. Session stickiness is documented for AgentCore Runtime and HTTP passthrough targets only, so don't assume a caller's session stays on one leg.

Gotchas

• No fallback, no safety net. A leg without a fallback block returns the error to the caller. See the fallback guide.
• Streams can't fall back midway. Once a stream starts, the caller gets the partial response plus an error and must retry.
• Every attempt is billed. Each outbound attempt is a gateway invocation, so a failing canary costs about one extra invocation per request until the breaker opens. Model charges come separately from Bedrock or the provider.
• 20 rules per gateway. A canary uses two, and priorities must be unique, so leave gaps and delete the QA pin when you finish.
• Gateway throttling doesn't trigger fallback, per the CLI reference.

Putting the model swap in a gateway rule means the commands that start a canary also end it, with no client redeploy, which is what you want when a new model misbehaves in production.