Why Leaders Should Start OCI Generative AI Pilots On-Demand

Your pilot is approved, and someone suggests provisioning a dedicated cluster so everything is ready on day one. Three weeks later, finance asks why a five-day experiment was billed like a month. If you lead an OCI Generative AI pilot, start in OCI Generative AI on-demand mode and move to a dedicated AI cluster only once the load is steady. Oracle's own pricing examples make the case.

What OCI Generative AI on-demand actually charges

On-demand is pay as you go. According to Oracle's on-demand pricing page, chat is billed on prompt length plus response length in characters, embeddings on input length, and one character counts as one transaction. The page describes no minimum commitment and calls the mode a fit for experimenting and evaluating models.

Its worked example: a meta.llama-3.3-70b-instruct chat with a 220-character prompt and a 2,205-character response is 2,425 transactions. You pay 2,425 / 10,000 of the listed unit price. A pilot that stalls for a week pays only for the transactions it actually sent.

The 744 unit-hour minimum on dedicated AI clusters

Dedicated clusters are billed by the unit hour instead. Oracle's dedicated pricing page sets a minimum commitment of 744 unit-hours per hosting cluster. That is 31 days of one unit, 31 x 24.

The docs walk through a pilot-shaped case. Alice hosts cohere.command-r-08-2024 on three Small Cohere units and deletes the cluster after five days. That is 5 x 24 x 3 = 360 unit-hours, which is below the minimum, so she is charged for 744. The other 384 unit-hours, just over half the bill, bought nothing.

Pilots are exactly the workloads that get paused, re-scoped or cancelled in week one. A minimum charge hurts most when the plan is least settled.

Dedicated AI cluster quota starts at zero

There is a second catch. By default, the number of dedicated AI clusters per tenancy is 0, and you get more by filing a limit increase request, as the service limits page explains. The page says to find the limit name in the Dedicated AI cluster section of the model's card.

So moving to dedicated is a ticket with a lead time, not a toggle. On-demand lets the team build while that request is still pending.

The objection: dedicated gives predictable pricing and capacity

This is a fair objection. Dedicated AI clusters are dedicated to your models and not shared with other tenancies, and a unit-hour charge does not move with traffic. Once load is high and steady, that predictability is worth paying for.

But predictability pays off only when you know the load, and a pilot is how you find out. Buying a fixed amount before you have measured anything usually means buying the wrong size. There is also a model risk: on-demand-only models have no dedicated option at all. Starting on-demand lets you settle the model first, then check that a dedicated option exists.

Details to check before you switch

• The 744 unit-hour minimum applies to hosting. Fine-tuning has its own minimum of 1 unit-hour per job, and some models need at least 2 units.
• Imported models do not carry the 744 unit-hour hosting commitment.
• Some hosting examples multiply the unit price. Oracle's Command R+ example multiplies by 2 for hosting, while the Command R example does not. Read the note for your model.
• The docs use placeholders for prices. Take rates from Oracle's price list, not from a slide deck.

First step: write the switch-over trigger before the pilot starts

The failure to avoid is drifting into a dedicated cluster because nobody wrote down when to switch. Before kickoff, put the expected steady-state load and the trigger in the pilot plan.

pilot: <pilot-name>
serving_mode: on-demand
expected_steady_state:
  requests_per_day: <number>
  avg_chars_per_request: <prompt chars + response chars>
  peak_requests_per_minute: <number>
switch_to_dedicated_when:
  - load stays at or above <threshold> for <N> consecutive weeks
  - monthly on-demand cost exceeds the monthly cost of one hosting cluster (744 unit-hours x unit price)
  - a launch requirement (latency, capacity) fails in on-demand testing
before_switching:
  - confirm the model has a dedicated option on its pretrained-models page
  - file the dedicated AI cluster limit increase request (default is 0 per tenancy)

This gives the steering group a number to argue about instead of a feeling. The second trigger is a break-even you can compute.

MIN_HOSTING_UNIT_HOURS = 744  # per hosting cluster, from Oracle's dedicated pricing page

def billed_unit_hours(units: int, days: float) -> float:
    return max(MIN_HOSTING_UNIT_HOURS, units * days * 24)

def breakeven_chars_per_month(
    on_demand_price_per_10k: float,    # from the Oracle price list
    dedicated_unit_hour_price: float,  # from the Oracle price list
    units: int = 1,
    hosting_multiplier: float = 1,     # check the model's pricing note
) -> float:
    monthly_dedicated = (
        MIN_HOSTING_UNIT_HOURS * units * dedicated_unit_hour_price * hosting_multiplier
    )
    return monthly_dedicated / on_demand_price_per_10k * 10_000

print(billed_unit_hours(units=3, days=5))   # 744, not 360
print(billed_unit_hours(units=1, days=40))  # 960

billed_unit_hours reproduces Oracle's examples. breakeven_chars_per_month estimates the monthly character volume above which one always-on hosting cluster costs less than on-demand for the same model. Fill in both prices from the price list and compare the result with the volume you wrote in the plan.

A pilot that starts on-demand and ends with a written trigger turns the dedicated-cluster purchase into a decision backed by measured load, which is the version of this spend you can defend to finance when the AI workload reaches production.