Benefits of GPT-6 Astra Flex Processing for Overnight Jobs

Your nightly eval run doesn't care whether a response takes four seconds or forty. Your invoice does care that you paid for the four-second version. GPT-6 Astra Flex fixes that mismatch: add service_tier="flex" to a request and tokens are billed at Batch rates, which the GPT-6 Astra model page lists at 50% of Standard. In exchange you get slower responses and the occasional capacity error.

Below is what that buys you for evals, backfills and report generation that run while everyone sleeps, a Responses API snippet, and the cases where you should skip it.

GPT-6 Astra Flex pricing next to Standard

Per 1M tokens, for prompts up to 272K input tokens, from the OpenAI pricing page:

| Tier | Input | Cached input | Output | | --- | --- | --- | --- | | Standard | $10 | $1 | $50 | | Flex | $5 | $0.50 | $25 |

Four benefits for overnight jobs

1. Half the bill for the same model. Flex runs gpt-6-astra, so your grades, extractions and summaries come from the model you tested, not a cheaper stand-in. As an illustration, a nightly eval of 2,000 prompts at 20K input and 2K output tokens each costs about $600 on Standard (40M input tokens at $10, 4M output tokens at $50) and about $300 on Flex. Same run, $300 back, every night.

2. One parameter instead of a new pipeline. Batch gets the same 50% rate, but it means writing a .jsonl file with a custom_id per line, capped at 50,000 requests and 200 MB per file, one model per file, then collecting results inside a 24-hour window (Batch guide). Flex is an ordinary responses.create call. Your loop, tracing and error handling keep working, and answers arrive as the job runs, so you spot a broken prompt at 1 a.m. instead of at 9.

3. Failed attempts cost nothing. When Flex lacks capacity it returns 429 Resource Unavailable, and OpenAI's Flex guide says you aren't charged when that happens. Retry with backoff, or fall back to service_tier="auto" for the stragglers. A busy night costs you time, not money, until you choose to pay Standard rates.

4. Caching stacks with the discount. The guide says prompt caching gives additional discounts on top of Flex, and cached input is listed at $0.50 per million tokens against $5 uncached. Grading jobs reuse the same rubric on every item, so keep that identical text at the front of the prompt and let the per-item content follow it.

Call GPT-6 Astra Flex from the Responses API

import time
from openai import OpenAI, RateLimitError

# Flex is slower, so raise the 10-minute SDK default to 15 minutes
client = OpenAI(timeout=900.0)

RUBRIC = "<your grading rubric, identical on every call>"

def grade(item: str, flex_attempts: int = 4):
    delay = 10
    for _ in range(flex_attempts):
        try:
            return client.responses.create(
                model="gpt-6-astra",
                instructions=RUBRIC,  # same prefix every call, so it can be cached
                input=item,
                service_tier="flex",
            )
        except RateLimitError:
            time.sleep(delay)
            delay *= 2
    # Out of patience: pay Standard rates rather than miss the morning deadline
    return client.responses.create(
        model="gpt-6-astra",
        instructions=RUBRIC,
        input=item,
        service_tier="auto",
    )

for item in open("nightly_items.txt"):
    print(grade(item.strip()).output_text)

This retries Flex with doubling delays, then falls back to the project's default processing, which is the fallback the Flex guide recommends. The Python SDK already retries a 429 twice with a short backoff, so this loop sits on top of that. RateLimitError also covers ordinary rate limits, which backoff handles too. If you would rather fail the job than pay Standard rates, replace the last call with a raise.

When GPT-6 Astra Flex is not worth it

• Anything a person waits for. OpenAI positions Flex for non-production or lower-priority work. A chat reply that takes minutes is a bug report.
• Hard deadlines. If the report must land by 6 a.m., add a clock check that switches the loop to auto after a cutoff. Otherwise backoff can eat your window.
• Contracts that assume stability. Flex is in beta with limited model availability, so behavior can change. Re-read the guide before putting it under an SLA.
• Jobs nobody needs until morning. Batch costs the same and has its own rate-limit pool, separate from your standard limits. Check how Flex traffic counts against your limits before running it beside live traffic.
• Very long prompts. Requests over 272K input tokens bill at long-context rates on every tier. On Flex that is $10 input and $37.50 output per million tokens.

For anyone running GPT-6 Astra in production, Flex lets the unwatched overnight work cost half as much, so the Standard budget goes to the requests users actually wait on.