Why Acceptance Criteria Should Be Tests, Not Tickets

Your ticket says "free shipping for large orders". The developer reads that as over 50. The AI coding agent reads it as 50 or more. QA checks whatever the Slack thread said. Everyone ships something, and everyone can point at the ticket to prove they were right.

A ticket can't fail. A test can. That is the whole argument for writing acceptance criteria as tests: a spec that goes red when someone gets it wrong is worth more than a spec that reads well.

What acceptance criteria tests look like in pytest-bdd

Gherkin is a plain-text format built on three keywords: Given (the starting state), When (the action) and Then (the outcome you can observe). pytest-bdd runs those files as ordinary pytest tests. Here is one story, saved as tests/acceptance/features/free_shipping.feature:

Feature: Free shipping on large orders

  Scenario: Cart just under the threshold pays shipping
    Given a cart with a subtotal of 49.99
    When the customer checks out
    Then shipping costs 5.00

  Scenario: Cart exactly at the threshold ships free
    Given a cart with a subtotal of 50.00
    When the customer checks out
    Then shipping costs 0.00

A product manager can write and review that file without touching Python. A developer, or an agent, binds it to code in tests/acceptance/test_free_shipping.py:

from decimal import Decimal

from pytest_bdd import given, parsers, scenarios, then, when

from shop.checkout import Cart, checkout  # your code, not pytest-bdd's

scenarios("features/free_shipping.feature")


@given(parsers.parse("a cart with a subtotal of {subtotal}"), target_fixture="cart")
def given_cart(subtotal):
    return Cart(subtotal=Decimal(subtotal))


@when("the customer checks out", target_fixture="order")
def when_checkout(cart):
    return checkout(cart)


@then(parsers.parse("shipping costs {amount}"))
def then_shipping(order, amount):
    assert order.shipping == Decimal(amount)

scenarios() turns every scenario in the file into a pytest test, and the path is relative to the test module by default. target_fixture hands each step's return value to the steps after it. shop.checkout stands in for your own code.

pip install pytest-bdd
pytest tests/acceptance -v

Run this before the rule exists and at least one scenario should fail. That red run is your proof the test can catch a bug.

Why a failing scenario beats a clearer ticket

It forces the boundary question early. To write the second scenario you have to pick a number, and picking 50.00 makes you ask whether exactly 50.00 qualifies. The ticket never asked. The product manager answers it while writing the spec, not in staging.

Done becomes a command. pytest tests/acceptance is the same check for a teammate, a reviewer and a coding agent. An agent given a ticket can only claim it finished. An agent given a failing scenario can run it, read the failure, fix the code and run it again.

Specs stop rotting. When someone changes the threshold to 75, the scenario goes red until the sentence is updated. The ticket from March stays closed and quietly wrong.

Run it in CI and guard the feature files

A spec that only runs on one laptop is a suggestion. Run it on every pull request:

# .github/workflows/acceptance.yml
name: acceptance
on: pull_request
jobs:
  acceptance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pytest pytest-bdd -r requirements.txt
      - run: pytest tests/acceptance

Agents optimize for green, and the quickest route to green is editing the test. Put the feature files behind code owners so a product manager must approve any change to them:

# .github/CODEOWNERS
/tests/acceptance/features/  @<your-org>/product

In GitHub, also turn on "Require review from Code Owners" in the branch protection rule, or the file does nothing.

Isn't BDD brittle overhead?

Sometimes, yes. Scenarios break constantly when they narrate clicks ("When I press the blue button"). Keep them to business rules and outcomes you can observe, which is what the Gherkin reference asks of Then steps, and they survive refactors because they never mention the implementation. Step definitions get reused, so the second scenario costs a few lines of text.

The cost is real in two cases. One is stories with no crisp rule, like visual polish or exploratory work. The other is a team where nobody outside engineering will ever read the file. There, plain pytest with good test names gives you most of the benefit with less machinery. The Gherkin layer earns its keep only when a non-engineer writes or reviews it.

Convert one open story this week

Pick one open story that has a number or a state change in it. Write two or three scenarios, including the boundary, and open them as a pull request before any code exists. Watch them fail in CI, then hand the branch to a developer or an agent with one instruction: make this green without editing the feature file.

When an agent writes the code, the spec is the only thing separating "it compiled" from "it's right", so make it something that can fail.