IBM and Together AI's $240M Bet on Open-Model Inference

Enterprise procurement for AI capacity usually moves at the speed of a committee. This one moved fast enough to make headlines: IBM and Together AI just signed a multi-year, $240 million agreement to build a dedicated inference cluster for open-weight models, and they told everyone exactly when it lands.

What IBM and Together AI actually signed

The deal puts a large, dedicated cluster of NVIDIA HGX B300 systems on IBM Cloud, networked with NVIDIA Spectrum-X Ethernet. IBM owns and hosts the infrastructure. Together AI operates it to serve inference for open-weight models through its AI Native Cloud platform, the same one that already handles a reported 400 trillion tokens a month for models like DeepSeek, Nemotron, MiniMax, Kimi and GLM, alongside the Llama family it has served since Together's early days.

IBM says the cluster is built for 30x more "AI factory" output than prior generations of hardware, which is the kind of number vendors love to throw around. The concrete part is the timeline: the cluster is expected to go live in Q1 2027, and it's explicitly positioned as dedicated inference capacity, not training capacity that gets time-shared down to you when someone else finishes a run.

Why a dedicated cluster matters more than the chip name

The HGX B300 is the headline, but the word that should catch a project manager's eye is "dedicated." Most open-weight inference today runs on shared multi-tenant capacity, where your latency depends on whoever else is hammering the same pool. A dedicated cluster means predictable throughput and predictable cost, which is the entire game once you move an open-weight deployment from pilot to production.

It's also IBM signaling that it wants a seat at the inference table, not just the training table. Most of the AI infrastructure headlines this year have been about training superclusters. Inference is the boring, expensive part that actually determines your unit economics once real users show up, and that's where this deal is aimed.

What this means for your rollout timeline and budget

If you're a PM scoping an open-weight model rollout for late 2026 or 2027, this changes your vendor shortlist in one specific way: you now have a dated option. Q1 2027 is concrete enough to put in a Gantt chart next to your own launch window. If your production date lands after that, IBM Cloud plus Together AI becomes a real line item to get a quote for, not just a press release to skim.

It doesn't help you this quarter. If you need open-weight inference capacity now, this cluster isn't it, budget for existing providers and revisit this option when it's actually live. Treat the announcement as a capacity forecast, not a resource you can requisition today.

Here's roughly what a team already on Together AI's platform is calling today, and what would eventually route through the new dedicated capacity once it ships:

curl -X POST https://api.together.xyz/v1/chat/completions \
  -H "Authorization: Bearer <your-together-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    "messages": [
      {"role": "user", "content": "Summarize this incident report in three bullet points."}
    ],
    "max_tokens": 512
  }'

The API surface doesn't change when the new cluster lands. What changes is which hardware pool answers the request and how consistent your p99 latency looks under load, which is exactly the number you want in your capacity plan before you commit a launch date to stakeholders.

What to do Monday morning

Add "IBM Cloud / Together AI, Q1 2027" to your infrastructure options doc as a dated candidate, not a decision. Ask Together AI's sales team now whether early capacity reservations are available ahead of general availability, since dedicated clusters tend to sell out to whoever asks first. And if your rollout timeline is sooner than Q1 2027, don't wait on this news, lock in your current provider and revisit the comparison once the cluster actually ships.

For anyone planning where open-weight inference will run a year from now, this is the first deal with both a real number and a real date attached, and that combination is rare enough to plan around.