Why Leaders Should Govern Model Access With Azure AI Foundry

Ask five engineering leads how many teams are calling OpenAI, Anthropic, or Meta models directly with a personal API key, and you'll get five wrong answers. Not because they're lying: because nobody actually knows. That's the real cost of letting model access grow organically. By the time it shows up in a finance review or a security audit, it's not a line item, it's an incident.

Azure AI Foundry model access fixes the part of this problem that's actually fixable: not by banning experimentation, but by giving every team the same front door. Instead of a developer pasting a key from a personal OpenAI account into a prototype, they pull the model through Foundry's catalog, which now spans well over a thousand models from Microsoft, OpenAI, Meta, Mistral, DeepSeek, and others behind one control plane, one identity system, and one bill.

The shadow AI problem leadership actually has

Shadow IT used to mean an unsanctioned SaaS tool. Shadow AI means an unsanctioned model call, and it's worse because the data going out the door is often the prompt itself: customer records, source code, contract text. A pilot that started as "let's just try the API" can quietly become the way a whole team gets work done, with no record of what model saw what data.

Leadership doesn't need to see every prompt. They need to answer three questions on demand: which models are in use, who approved them, and what they cost. Scattered keys make all three unanswerable. A model catalog behind Microsoft Entra ID identity and Azure role-based access control (RBAC) makes them a query.

What routing through the catalog actually buys you

When every deployment goes through an AI Foundry project, you get things a spreadsheet of API keys never gives you:

• Access scoped to a role, not a person's memory. Assign the built-in Azure AI Developer role at the project level, and someone loses model access the moment they leave the team, automatically, through the same Entra ID group that manages everything else.
• Spend that shows up where finance already looks. Foundry resource usage flows into Azure Cost Management like any other Azure spend, so a model going from a $200/month pilot to a $20,000/month habit gets caught in a normal budget review instead of an end-of-quarter surprise.
• One place to enforce content filters and data boundaries. Content safety policies and network rules attach to the Foundry resource, not to each developer's individual habits.

Here's what scoping access to one project looks like in practice:

az role assignment create \
  --assignee "<user-or-group-object-id>" \
  --role "Azure AI Developer" \
  --scope "/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<foundry-resource-name>/projects/<project-name>"

That command grants deployment rights on exactly one project, not the subscription. Nobody needs owner rights on Azure to try a model; they need this one role, scoped tightly, and revocable in one line when the project ends.

The honest objection: doesn't this just slow teams down?

Yes, a little, and pretending otherwise would be dishonest. A developer used to pip install openai and a key in .env will find a project, a deployment, and a role assignment slower than that on day one.

But the comparison isn't "governed" versus "instant." It's "governed" versus "instant now, frozen later when security finds the key in a GitHub commit and pulls it without warning." The catalog's breadth also softens the friction: since most of the models a team would reach for are already in Foundry, the choice usually isn't between speed and governance, it's between one extra role assignment and a personal credit card.

Where to start Monday

Don't try to migrate every team at once. Pick the one with the most active experimentation, usually the one already costing the most in scattered API bills, and move it first:

1. Stand up one Azure AI Foundry project for that team.
2. Assign Azure AI Developer to the team, not to individuals, through an Entra ID group.
3. Turn off the personal keys once deployments are live, and watch the Cost Management view for two weeks before you decide the exercise was worth it.

Governance that leadership can't see doesn't count as governance. Once model access runs through Azure AI Foundry, every question about who's using what, and what it's costing, has an answer before someone has to go looking for one.