How to Call GPT-6 Astra in the Responses API With Python

Add 30K tokens to a 270K-token prompt and a GPT-6 Astra call roughly doubles in price. Better to learn that here than from an invoice. By the end you'll have a working GPT-6 Astra API call in Python through the Responses API, with explicit reasoning effort, structured output, function calling, a cost check for the 272K cliff, and the cheaper Flex and Batch routes. The facts come from OpenAI's GPT-6 Astra model page and API guides.

1. Install the SDK and confirm GPT-6 Astra API access

The model page lists the ID gpt-6-astra but doesn't say who may call it, so check first.

python -m venv .venv && source .venv/bin/activate
pip install --upgrade openai pydantic
export OPENAI_API_KEY="<your-api-key>"

from openai import NotFoundError, OpenAI, PermissionDeniedError

client = OpenAI(max_retries=5)  # the SDK retries 429s and timeouts for you

try:
    model = client.models.retrieve("gpt-6-astra")
    print("Access OK:", model.id)
except (NotFoundError, PermissionDeniedError) as err:
    raise SystemExit(f"This key can't use gpt-6-astra yet: {err}")

A 404 or 403 means your organization can't use the model yet.

2. Make your first OpenAI Responses API call in Python

Use Responses, not Chat Completions: OpenAI's latest-model guide says tool calling on Astra requires it. The same guide says to drop temperature, top_p and top_logprobs.

response = client.responses.create(
    model="gpt-6-astra",
    reasoning={"effort": "medium"},
    max_output_tokens=25000,
    input=[
        {"role": "developer", "content": "You are a terse senior SRE. Answer in five lines or fewer."},
        {"role": "user", "content": "Why would a Kubernetes HPA flap between 3 and 9 replicas every few minutes?"},
    ],
)

print(response.status)
print(response.output_text)
print(response.usage.output_tokens_details.reasoning_tokens)

max_output_tokens includes hidden reasoning tokens, and OpenAI's reasoning guide suggests reserving at least 25,000 while experimenting. A status of incomplete means the budget ran out, possibly before any visible text, and you still pay for the reasoning.

3. Set GPT-6 Astra reasoning effort per task

reasoning.effort takes low, medium, high, xhigh or max. The model page states no default, so set it on every call. none returns an HTTP 400; if you used minimal elsewhere, start at low. Reasoning tokens bill as output at $50 per million, so effort is also a cost dial.

PROMPT = "Review this plan: migrate a 2 TB Postgres database to a new region with a 10 minute downtime window. List the three biggest risks."

for effort in ["low", "medium", "high", "xhigh", "max"]:
    r = client.with_options(timeout=900.0).responses.create(
        model="gpt-6-astra",
        reasoning={"effort": effort},
        max_output_tokens=60000,
        input=PROMPT,
    )
    u = r.usage
    print(effort, r.status, u.output_tokens_details.reasoning_tokens, u.output_tokens)

Run this on one representative task and pick the cheapest level whose answer you'd ship.

4. Return structured output with responses.parse

Structured outputs are supported. responses.parse builds the JSON Schema from a Pydantic model and hands back a typed object.

from typing import Literal

from pydantic import BaseModel


class Triage(BaseModel):
    severity: Literal["low", "medium", "high"]
    component: str
    next_action: str


result = client.responses.parse(
    model="gpt-6-astra",
    reasoning={"effort": "low"},
    max_output_tokens=25000,
    input=[
        {"role": "developer", "content": "Triage the incident report."},
        {"role": "user", "content": "Checkout latency tripled after the 14:05 deploy; error rate unchanged."},
    ],
    text_format=Triage,
)

triage = result.output_parsed
if triage is None:
    print("No parsed output, status:", result.status)
else:
    print(triage.severity, triage.component, triage.next_action)

If the model declines a request you get a refusal instead of data, so check output_parsed for None.

5. Add function calling to the loop

Declare the tool, run what the model asks for, return a function_call_output with the matching call_id, and repeat until it stops asking.

import json


def get_deploy_status(service: str) -> str:
    # Stub: call your real deploy API here.
    return json.dumps({"service": service, "status": "healthy"})


tools = [{
    "type": "function",
    "name": "get_deploy_status",
    "description": "Get the current deploy status of a service.",
    "parameters": {
        "type": "object",
        "properties": {"service": {"type": "string"}},
        "required": ["service"],
        "additionalProperties": False,
    },
    "strict": True,
}]

input_list = [{"role": "user", "content": "Is checkout-api healthy right now?"}]

for _ in range(8):  # hard cap on tool-call rounds
    response = client.responses.create(
        model="gpt-6-astra",
        reasoning={"effort": "low"},
        max_output_tokens=25000,
        tools=tools,
        input=input_list,
    )
    input_list += response.output
    calls = [item for item in response.output if item.type == "function_call"]
    if not calls:
        break
    for call in calls:
        args = json.loads(call.arguments)
        input_list.append({
            "type": "function_call_output",
            "call_id": call.call_id,
            "output": get_deploy_status(**args),
        })

print(response.output_text)

Appending response.output carries earlier items into the next turn.

6. Check GPT-6 Astra pricing on every call

Past 272K input tokens, input and cache rates double and output rises 1.5x for the whole request, not only the overflow.

| Per 1M tokens | Standard | Over 272K input | Batch or Flex | | --- | --- | --- | --- | | Input | $10 | $20 | $5 | | Cached input | $1 | $2 | $0.50 | | Output | $50 | $75 | $25 |

A 270K prompt with 5K output costs about $2.95. At 300K it costs about $6.38.

PRICE = {"input": 10.0, "cached": 1.0, "write": 12.50, "output": 50.0}  # USD per 1M tokens
CLIFF = 272_000


def estimate_cost(usage, discount=1.0):
    """USD for one call. Use discount=0.5 for Flex or Batch under the cliff."""
    d = usage.input_tokens_details
    cached = d.cached_tokens or 0
    written = getattr(d, "cache_write_tokens", 0) or 0
    fresh = usage.input_tokens - cached - written

    over = usage.input_tokens > CLIFF
    in_mult, out_mult = (2.0, 1.5) if over else (1.0, 1.0)
    if over:
        print(f"WARNING: {usage.input_tokens:,} input tokens is past {CLIFF:,}")

    cost_in = (fresh * PRICE["input"] + cached * PRICE["cached"] + written * PRICE["write"]) * in_mult
    cost_out = usage.output_tokens * PRICE["output"] * out_mult
    return (cost_in + cost_out) / 1_000_000 * discount


STABLE_PLAYBOOK = open("playbook.md").read()  # long, unchanging text goes first

response = client.responses.create(
    model="gpt-6-astra",
    reasoning={"effort": "medium"},
    max_output_tokens=25000,
    prompt_cache_key="support:customer_123",
    input=[
        {"role": "developer", "content": STABLE_PLAYBOOK},
        {"role": "user", "content": "<this ticket's text>"},
    ],
)
print(f"${estimate_cost(response.usage):.4f}")

Per the prompt caching guide, a prefix needs at least 1,024 tokens to be cached and reads cost a tenth of the input rate, while writes cost 1.25x. Put stable instructions first and reuse one prompt_cache_key per customer.

7. Halve the bill with Flex or Batch

Both cost 50% of Standard. Flex is the same call with service_tier="flex". It's in beta, so expect slower responses and occasional 429 Resource Unavailable errors (not billed), and fall back to auto for work that must finish.

import time

from openai import RateLimitError


def flex_call(**kwargs):  # same arguments as responses.create
    for attempt in range(4):
        try:
            return client.with_options(timeout=900.0).responses.create(service_tier="flex", **kwargs)
        except RateLimitError:  # includes 429 Resource Unavailable
            time.sleep(5 * 2**attempt)
    return client.responses.create(service_tier="auto", **kwargs)  # Standard rates, but it finishes

Batch is a file job with a 24-hour window, and the model page lists it as supported.

import json

documents = ["<doc 1 text>", "<doc 2 text>"]

with open("batch.jsonl", "w") as f:
    for i, text in enumerate(documents):
        line = {
            "custom_id": f"doc-{i}",
            "method": "POST",
            "url": "/v1/responses",
            "body": {
                "model": "gpt-6-astra",
                "reasoning": {"effort": "low"},
                "max_output_tokens": 25000,
                "input": text,
            },
        }
        f.write(json.dumps(line) + "\n")

batch_file = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=batch_file.id,
    endpoint="/v1/responses",
    completion_window="24h",
)

# Later: poll, then download the results once status is "completed".
batch = client.batches.retrieve(batch.id)
if batch.status == "completed":
    print(client.files.content(batch.output_file_id).text)

Results can come back out of order, so match them on custom_id.

Gotchas

• Endpoints: Chat Completions on Astra is plain text only. The model page also lists Realtime, Assistants, fine-tuning and embeddings as unsupported.
• Rate limits (Standard): Build gets 5,000 RPM and 1M TPM, Launch 10,000 RPM and 4M TPM, Grow 15,000 RPM and 40M TPM. Tiers rise with usage and spend.
• Cyber safeguards: Launch coverage reports that Astra declines advanced offensive security tasks, such as building proof-of-concept exploits. The model page doesn't document this, so treat a refusal as a normal outcome, log it, and don't retry in a loop.
• Long-context discounts: The model page doesn't spell out the over-272K price for Flex and Batch, so treat any estimate there as approximate.

Log cost, status and refusals from day one, because all three show up on the invoice before they show up in a user complaint.