How to Roll Back a Bad Model or Prompt Change in Minutes

Sooner or later a prompt edit or model upgrade makes answers worse in production. How long the problem lasts depends mostly on how quickly you can switch back.

If rolling back means reverting commits, rebuilding and redeploying, expect an hour. With some preparation, it takes a minute.

Step 1: Separate configuration from the build

Keep the active prompt version, model name and parameters in runtime configuration, such as a parameter store or a feature flag service, instead of hard-coding them in the image. The deployed code should be able to run the previous version as well as the new one.

Step 2: Keep the previous version deployable

Don't delete the old prompt files when you release a new version. Keep at least the last two available, so switching back is a configuration change:

support_agent:
  active_version: "2026-09-10"
  fallback_version: "2026-08-28"

Step 3: Put risky changes behind a flag

Release model upgrades and large prompt changes behind a flag. Turning the flag off sends every request back to the old path immediately, with no deployment.

Step 4: Decide the trigger in advance

Write down what triggers a rollback: error rate above a threshold, latency above a limit, a spike in escalations or complaints from a key customer. When the criteria are agreed beforehand, the on-call engineer can act without waiting for permission.

Step 5: Practice

Run a rollback in staging once a month and time it. If it takes longer than five minutes, find the slow step and fix it.

Step 6: Communicate

After rolling back, tell the affected people what happened and what comes next. Record the change, the symptom and the time to recover in the incident log.

Things to watch

• Provider-side changes. If a provider updates what a model alias points to, you can't roll that back. Pin specific model versions where the provider offers them.
• Data written by the bad version. Rolling back doesn't fix records or messages it already produced. Know how to find them.
• Caches. Cached responses from the bad version can keep appearing. Include the version in cache keys.

Schedule the monthly drill. A rollback nobody has practiced is always slower than expected.