📰 AI NewsLeadershipIntermediate✨ AI-assisted

Microsoft Foundry Browser Automation Tool Reaches GA

WittyTech Daily··4 min read
#microsoft foundry#browser automation#playwright workspaces#ai agents

That vendor portal with no API and a login page nobody dares touch can now be automated on purpose. On October 19, Microsoft announced that the Browser Automation tool in Microsoft Foundry is generally available, so an agent can open a cloud browser, read a page and fill in a form. Foundry browser automation has left preview, but the governance questions have not gone anywhere.

What shipped: managed browsers for Foundry agents

The tool runs as an MCP tool (Model Context Protocol, the standard way agents call external tools) on Playwright Workspaces, a GA cloud browser service, so agents need no local browser install. Per the Microsoft Learn docs, each task gets a browser session ID, and the agent loops: inspect the page, act, check the result, continue.

Attach it to prompt agents or to hosted agents built with Microsoft Agent Framework or your own framework. The recommended route is a toolbox, a managed MCP endpoint that lets you reuse the tool across agents and centralize credentials, versioning and policy, as the GA announcement describes.

Live View and Take control keep a human in the loop

Live View shows an active browser session in real time. Take control lets a person step in, fix an unexpected state or finish a supervised step, then hand control back to the agent.

The agent posts a Live View URL at the start of a run, and the docs say to treat it as sensitive. For production, the docs favor run summaries, traces and logs over watching a screen.

Regions and private sites limit where Foundry browser automation fits

The tool is available in seven regions: australiaeast, eastasia, eastus, japaneast, switzerlandnorth, westeurope and westus3. For hosted agents, the Playwright workspace must sit in a region that supports hosted agents, and your model and project region must support the tool.

Private website browsing is the catch. Learn lists it as public preview, while the announcement says private apps behind an Azure virtual network are supported without labeling the status, so treat it as preview. Intranet-only apps are many enterprises' best use case, so plan around this gap.

Read the security warnings before you pick a use case

The docs warn the tool "controls web browsers on your behalf and can take unintended actions." The transparency note adds that malicious or confusing instructions on a page can make an agent run commands you didn't intend, putting at risk any account the browser can reach. The limitations section says to use only sites you trust and avoid pages that prompt for credentials, payments or other sensitive actions.

Now reread the docs' flagship example: pulling invoice details from a vendor portal into a legacy finance app. That is the tension. Start read-only, and follow the governance guidance: least-privilege access, isolated sessions, treating web content as untrusted input, and human approval for state-changing actions. Foundry traces can also hold prompts and tool-call data, so restrict who can read them.

Set up Foundry browser automation with least privilege

The docs split the roles. Foundry User covers day-to-day use, Foundry Project Manager creates the connection, and Contributor on the resource group is needed only while creating the Playwright workspace. For auth, the docs call project managed identity the recommended default for enterprise workloads; API keys are a prototyping option.

This Azure Developer CLI flow creates the connection and a toolbox.

azd ai connection create my-browser-conn \
  --kind PlaywrightWorkspace \
  --target "<browser-endpoint-from-workspace-overview>" \
  --auth-type api-key \
  --key "<playwright-workspaces-access-token>"
# my-toolbox.yaml
description: Browser Automation toolbox
tools:
  - type: browser_automation
    project_connection_id: my-browser-conn
azd ai toolbox create my-toolbox --from-file my-toolbox.yaml

--kind PlaywrightWorkspace needs exact PascalCase. The token is shown once, so store it only in the project connection. For managed identity, use the portal's toolbox flow.

The same toolbox from Python, after pip install "azure-ai-projects>=2.8.0":

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    BrowserAutomationToolboxTool,
    BrowserAutomationToolParameters,
    BrowserAutomationToolConnectionParameters,
)
from azure.identity import DefaultAzureCredential

PROJECT_ENDPOINT = "https://<account>.services.ai.azure.com/api/projects/<project>"

project = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=DefaultAzureCredential())
tool = BrowserAutomationToolboxTool(
    browser_automation=BrowserAutomationToolParameters(
        connection=BrowserAutomationToolConnectionParameters(
            project_connection_id="<browser-automation-connection-id>",
        )
    )
)
toolbox = project.toolboxes.create_version(
    name="browser-automation-toolbox",
    description="Toolbox with the Browser Automation tool",
    tools=[tool],
)
print(f"{PROJECT_ENDPOINT}/toolboxes/{toolbox.name}/versions/{toolbox.version}/mcp?api-version=v1")

The printed URL is the toolbox's MCP endpoint, which your agent connects to.

What to do Monday morning

  1. Pick one browser-only workflow with no API and a small blast radius, such as read-only data extraction.
  2. Provision the Playwright workspace in a supported region, connect with managed identity, then drop the Contributor role.
  3. Run with Live View on, then check Foundry traces to confirm what the tool did.
  4. Require human approval before any state-changing step, and keep credentials and payments out of scope.
  5. Name a cost owner: spend lands on the Playwright workspace resource, and the Foundry docs only link to its pricing page.

For anyone deploying agents in production, a browser agent is a user account with a very literal mind, so scope its access, require approval for state changes and treat every page it reads as untrusted input.

← More in Leadership