待翻譯：Browser automation agents with Pydantic AI and Playwright

AI 服務暫時不可用，以下為來源正文，待恢復後補全翻譯。

When we build agents, we often want to give them the ability to browse the web: open webpages, navigate from one page to the other, and read the content of a webpage. By combining Pydantic AI with the Playwright capability from Pydantic AI Harness, we can build agents that browse the web safely and programmatically. Using Pydantic AI with Microsoft Foundry models Pydantic AI is an open-source model-agnostic framework from Pydantic for building LLM-based applications and agents. It's type-safe and supports OpenTelemetry, making it a great choice for robust production applications. We can use Pydantic-AI with Microsoft Foundry models using either API keys or Entra token-based authentication. When possible, we always recommend the keyless route, so that's what we'll demonstrate here. We use the azure-identity package to authenticate with Entra, using either local or managed identity, and get back a token provider callback function for that credential: from azure.identity.aio import AzureDeveloperCliCredential, get_bearer_token_provider credential = AzureDeveloperCliCredential() token_provider = get_bearer_token_provider(credential, "https://cognitiveservices.azure.com/.default") Then we use the OpenAI package to configure the model connection: from openai import AsyncOpenAI client = AsyncOpenAI( base_url=os.environ["AZURE_OPENAI_ENDPOINT"] + "/openai/v1", api_key=token_provider, ) model = OpenAIChatModel( model_name=os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT"], provider=OpenAIProvider(openai_client=client), ) Let's explain the options used above: base_url: We point this at the OpenAI-compatible endpoint for our Foundry model. This endpoint works for Azure OpenAI models (like gpt-5.4, which this project deploys), and for cross-provider Foundry models that support the OpenAI v1 API, like Kimi-K2.7-Code. The base URL looks like "https://AZURE_OPENAI_SERVICE_NAME.openai.azure.com/openai/v1". api_key: We pass in the token provider callback function that generates OAuth2 tokens using our Entra credential. If we were using API keys, we'd simply pass in the key string here instead. model_name: We provide the name of the deployment, not the name of the model. Oftentimes, the deployment name is the same as the model, but not always - it depends on how you set it up in the Portal or infrastructure-as-code files. Notably, when using models on Foundry, you must always make an explicit deployment for the desired model, before you can use it. Integrating Playwright capability Playwright is a browser automation library. It was originally built for writing E2E tests to verify website correctness, and is still the best option for E2E tests today. Its browser automation capabilities also make it a powerful way to give an agent access to websites. When you yourself are developing a website, it's a great way to give the agent access to browse the website, do manual QA, and iterate on design improvements. We can also use Playwright to access other websites, as long as the website's terms permit programmatic access. To integrate Pydantic AI with Playwright, we bring in the PlaywrightBrowser capability from pydantic-ai-harness, a library of additional capabilities for Pydantic AI agents. from pydantic_ai_harness.playwright import PlaywrightBrowser browser = PlaywrightBrowser( allowed_domains=[website_hostname], block_private_addresses=True, headless=False, max_content_tokens=30000, action_timeout_ms=5_000, navigation_timeout_ms=30_000, screenshot_on_navigate=False, auto_install_chromium=False, ) Let's break down those parameters: allowed_domains: Restricts top-level navigation and data-moving requests (like fetch and XHR) to the specified hostnames. This prevents unexpected navigation and data transfer, keeping the agent’s scenario targeted. block_private_addresses: By default, this option is set to True to prevent navigation to localhost and private or reserved IP addresses, even when that address appears in allowed_domains. Set it to False only when the agent needs explicit access to a trusted locally deployed application. headless: By default, Playwright will run in headless mode, which means that the browser window is not visible. When developing, I often set this to False, since it can be helpful to actually watch Playwright control the browser. max_content_tokens: This option limits the amount of webpage text returned to the agent. This defaults to 4000 tokens, so I increased it to 30,000 tokens to allow for longer webpages. Keep in mind that the amount of content returned will increase the usage of the context window, affecting both performance and latency of subsequent LLM calls. action_timeout_ms and navigation_timeout_ms: Actions like clicking or typing and page navigations get separate deadlines, since they fail for different reasons. A click on a selector that does not exist should fail fast, so the action deadline defaults to 5 seconds, while a page load deserves more room; here I allow 30 seconds for navigation. A tool call can also pass its own timeout_ms when the agent knows a step is slow. screenshot_on_navigate: Controls whether Playwright takes a screenshot after every navigation and attaches to the agent session. This defaults to False, since screenshots can bloat the context window, but you may want to enable it for more design-heavy workflows or for human auditing purposes. auto_install_chromium: When set to True, the library itself will download the binary for the Chromium browser. This is off by default, so you must explicitly install chromium before running. Typically you would install chromium in your environments manually so that you can properly cache it across runs, like in CI/CD. Creating the Pydantic AI agent Now that we have the Foundry model connection and Playwright browser configured, we can construct a Pydantic AI agent that combines the model and capabilities together. We also include the FileSystem capability, restricted to an outputs folder, so that the agent can easily write out its Markdown reports. agent = Agent( model=model, capabilities=[browser, FileSystem(root_dir=OUTPUT_ROOT)], system_prompt="You are a careful manual QA agent testing a website...", ) Then we run the agent, asking it to do a manual QA pass on the specified website: result = await agent.run( f"Perform a manual QA pass on {url}." "Load this URL first, make a testing plan, and investigate the " "highest-value usability risks and functional bugs you can reproduce. " "Write the required report to outputs/qa-report.md.", ) The Pydantic AI agent sends the query to the Foundry model, along with the Playwright tool definitions, and the model decides which Playwright tool to call, looping until it's completed the task: Instrumenting OpenTelemetry for inspecting the browsing activity We can inspect the generated report to see that the agent successfully completed the task, but we usually want to dig deeper: What pages did it browse? What commands did it run on those pages? How many tokens were used during the process? Fortunately, we can instrument any Pydantic AI agent with OpenTelemetry, exporting the traces to any OpenTelemetry-compliant provider, like Pydantic Logfire or Azure App Insights. Let's step through the code to send traces to Logfire: trace_file = (OUTPUT_ROOT / "traces.jsonl").open("a", encoding="utf-8") configured_logfire = logfire.configure( send_to_logfire="if-token-present", token=os.getenv("LOGFIRE_TOKEN"), service_name="pydanticai-playwright-qa", console=logfire.ConsoleOptions(), additional_span_processors=[ SimpleSpanProcessor(ConsoleSpanExporter(out=trace_file))], ) That constructor sends the traces to Logfire based on the token saved in the environment. It includes logging of the traces to the console, plus an additional exporter to a local file. The console traces are helpful for us to watch while we are developing the agent, and the local traces file can be useful input for coding agents debugging an agent. By pointing an agent at the traces file, it can audit the Playwright browser calls and recommend improvements to the prompt and parameters. Next, we set up instrumentation specific to the packages we're using: configured_logfire.instrument_openai(client) configured_logfire.instrument_pydantic_ai(agent, include_content=True) That code calls instrument_openai for our calls through the openai package, and instrument_pydantic_ai for our calls through pydantic-ai package. Both of those packages export traces using the Generative AI semantic conventions, which exists to ensure that calls to LLMs, tools, and agents, are traced in a consistent way across observability platforms and agent frameworks. After running the agent, we can browse through the traces. Here's what a single run looks like: To also export traces to Azure Application Insights, we can add an additional span processor from the azure-monitor-opentelemetry-exporter package, pointing at our App Insights instance: connection_string = os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"] logfire.configure( # other arguments additional_span_processors=[ SimpleSpanProcessor(ConsoleSpanExporter(out=trace_file)), SimpleSpanProcessor( AzureMonitorTraceExporter.from_connection_string(connection_string) )]) Since both platforms support OpenTelemetry, the traces are the same across both. Accessing authenticated websites But wait, what if the target website requires user login? When Playwright starts a browser instance, it's completely isolated from your day-to-day browser instance, so it has no access to cookies. Typically, that is a very good thing, since we don't want agents to have arbitrary access to our logged in accounts. However, you may be building an agent that is dependent on access to a logged in website. In that case, we can explicitly pass a session state to the PlaywrightBrowser instance, and it will use the cookies and local storage from that state: browser = PlaywrightBrowser( storage_state=json.loads(Path("playwright/.auth/site.json").read_text()) ) To generate that state JSON file, we can run the Playwright codegen command to pop up the website. Once we login and close the browser, the browser state is saved to the target location. It's important to keep that storage file safe and secure - don't check into version control! uv run playwright codegen https://your-owned-site.example/ \ --save-storage=playwright/.auth/site.json Then pass the saved state to the agent through the command-line option: uv run python pydanticai_playwright.py https://your-owned-site.example/ \ --session-state playwright/.auth/site.json Next steps Download the full code for the Pydantic AI agent from this project: github.com/pamelafox/pydanticai-playwright-agent That repository also includes infrastructure-as-code (Bicep) for provisioning an Azure OpenAI model and configuring the full environment for you. Fork the code, customize it, and make your own browser-using agent!