AI 服務暫時不可用，以下為來源正文，待恢復後補全翻譯。
Multi-agent systems in production experience issues in ways that traditional monitoring misses. For example, the agent can’t invoke its foundation model (FM) and returns an empty response. This could be because of a missing AWS Identity and Access Management (IAM) permission on an agent’s execution role that doesn’t throw a 500 error. A supervisor agent with a poorly scoped prompt doesn’t increase error rates but instead starts routing 20 percent of requests to the unintended specialist while the infrastructure metrics stay green. Infrastructure monitoring and agent effectiveness monitoring require different approaches. Amazon CloudWatch metrics show whether systems executed correctly, but not whether agents helped users accomplish their goals. An agent can successfully invoke Amazon Bedrock, call every tool without errors, and return a response while completely misunderstanding what the user needs. Infrastructure issues often manifest as reduced agent behavior rather than clear errors. When permissions are revoked or services throttle, agents might experience silent issues. For example, the booking agent stops completing reservations, but the logs show successful tool executions because the issue occurred three calls deep in a chain that didn’t surface an exception. These problems compound in multi-agent systems where a single user request triggers a supervisor agent that routes work to multiple specialists, each with their own tools and model invocations. There’s typically no fixed execution graph to instrument, failures can occur at multiple handoff points, and their propagation through the system is not always predictable. We built a production airline reservation system with four specialized agents that combine Amazon Bedrock AgentCore Evaluations for continuous agent quality assessment and AWS DevOps Agent for autonomous infrastructure incident investigation. Amazon Bedrock AgentCore is a platform to build, connect, and optimize agents at scale, with any framework or model. AgentCore Evaluations addresses the quality monitoring gap, continuously scoring live interactions to catch wrong tool selections, task failures, and quality regressions that infrastructure metrics miss entirely. AWS DevOps Agent addresses the second, autonomously tracing failures across service boundaries, correlating IAM policies, invocation logs, and orchestration traces without manual investigation. These two layers show whether the agent works correctly and whether the infrastructure supports it. Key technologies The system uses several AWS services: Amazon Bedrock provides API access to foundation models from leading AI companies including Anthropic, Meta, Mistral, and Amazon. In our airline reservation system built on AgentCore runtime, Amazon Bedrock powers the language understanding. For model availability by AWS Region, refer to Supported models by AWS Region in Amazon Bedrock. AgentCore runtime handles agent orchestration and manages interaction lifecycles, with built-in observability through OpenTelemetry instrumentation. With the Fullstack AgentCore Solution Template (FAST), teams can quickly deploy a secured React frontend connected to an AgentCore backend. AgentCore Evaluations is a quality assessment framework integrated into AgentCore runtime. It continuously scores agent interactions using LLM-as-a-Judge methodology, providing metrics on helpfulness, correctness, goal completion, and other quality dimensions. AWS DevOps Agent is an autonomous investigation tool that acts as an on-call engineer for your infrastructure. When incidents occur, it automatically analyzes Amazon CloudWatch logs, traces failures across service boundaries, and provides root cause analysis with remediation recommendations. Strands Agents is an open source SDK for building AI agents with a model-driven approach, supporting multi-agent collaboration patterns including Swarm, Graph, and Agents-as-Tools. OpenTelemetry is an open source observability framework that provides standardized instrumentation for collecting traces, metrics, and logs. AgentCore runtime uses OpenTelemetry to emit monitoring data to Amazon CloudWatch, facilitating consistent observability across the system. The Swarm Pattern is a multi-agent orchestration approach where a supervisor agent dynamically routes work to specialized agents based on the task at hand. Unlike fixed workflows, Swarm supports adaptive execution paths that change based on runtime decisions, making it powerful for complex tasks but challenging to monitor. Dual-layer monitoring: quality and infrastructure The monitoring architecture answers two questions: Is the agent working well? Is the infrastructure healthy? Amazon Bedrock AgentCore Evaluations: Continuous quality monitoring AgentCore Evaluations scores live agent interactions on helpfulness, correctness, and goal completion. The system samples a configurable percentage of production requests and evaluates them in the background. Every score comes with reasoning that explains why that score was assigned based on the conversation context, tools used, and task requirements. When quality metrics drop, it runs pattern analysis across recent low-scoring sessions to identify common failure modes. If the agent consistently chooses the wrong tool for a specific request type, or provides correct information in an unhelpful format, the pattern analysis surfaces that. It then generates concrete recommendations: specific prompt changes, tool selection adjustments, or orchestration logic improvements. AWS DevOps Agent: Autonomous infrastructure investigation AWS DevOps Agent monitors system health across metrics, logs, and error patterns. When something breaks, the agent investigates on its own. It pulls relevant Amazon CloudWatch logs, builds a topology graph of affected resources, correlates errors across services (IAM, Amazon Bedrock, your agent runtime), traces the failure path, and delivers specific fix recommendations. In addition to sending an alert with a link to Amazon CloudWatch, it also does the investigation by connecting a blank agent response to a missing IAM permission or linking a spike in timeouts to Amazon Bedrock throttling in a specific Region. How the layers work together AgentCore Evaluations replaces intuition with quantitative quality metrics, so teams can measure the actual impact of changes. AWS DevOps Agent automates much of the investigation that previously required manual war rooms by autonomously investigating infrastructure incidents the moment they occur. Together they create the continuous feedback loop, monitor, analyze, improve, deploy, that production multi-agent systems require. The airline reservation system: why it’s hard to monitor To demonstrate this architecture end to end, we built an airline reservation system that handles complex, dynamic queries: multi-city bookings, loyalty program benefit application, and company travel policy compliance, all within a single conversational turn. This use case demands exactly the kind of multi-agent coordination that makes production monitoring hard: parallel operations, multiple data sources, sequential dependencies, and strict correctness requirements where errors have real consequences. Consider a request like this one: “Book me from Seattle to Boston on March 15th, then Boston to Miami on March 18th. Use my companion certificate for the second leg and make sure both flights comply with my company’s travel policy. I’m Gold status so apply any eligible upgrades.” Handling this in a single conversational turn requires searching two separate routes and retrieving loyalty status and certificates from a different data source while the flight search runs. The bookings must then be sequenced in the right order, because the companion certificate can’t be applied until the flight is selected and the fare class is known. Applying a certificate to an ineligible flight frustrates users. Booking a flight that violates corporate policy wastes money. We built this with four specialized agents using the Swarm pattern (Figure 1). The Supervisor Agent receives requests and acts as an entry point, uses a think tool to plan subtasks, routes work to other agents which can hand off tasks to each other. The Flight Agent searches routes and handles multi-city connections. The User Agent fetches loyalty status, certificates, and profile data. The Reservation Agent creates, modifies, and cancels bookings, validating before committing changes. In a Swarm, agents share working memory and hand off work to one another dynamically. Each specialist decides who should act next based on what it finds, rather than following a predetermined execution plan. The supervisor is only the entry point. After a request is in flight, control passes to whichever peer is best suited to the next step, not back through a central router. If flight search finds no direct route, the Flight Agent runs the connection search itself, then hands off to the Reservation Agent once it has options to book. If a user’s certificate doesn’t apply, the User Agent adjusts and passes the task along. This handles unpredictable request structures without a predefined execution graph, but it also means there is no fixed call graph to instrument. Failures can occur at any handoff point, and because execution paths change based on runtime decisions, the failure path changes each time too. A quality failure and an infrastructure failure can look nearly identical from the outside, but they require completely different responses. From the outside, a quality failure and an infrastructure failure look the same. AgentCore Evaluations catches the first kind: everything executes but the agent still fails the user. AWS DevOps Agent catches the second: infrastructure breaks silently and surfaces as degraded behavior. Figure 1: Swarm multi-agent pattern Dual monitoring architecture The following diagram shows how these components connect. Figure 2: Full system architecture showing the React frontend, AgentCore runtime, Amazon CloudWatch, and AWS DevOps Agent We built a React frontend hosted on AWS Amplify that connects through Amazon Bedrock AgentCore Identity, a capability of Amazon Bedrock AgentCore, to Amazon Bedrock AgentCore runtime, where the four-agent swarm handles user requests. AWS Amplify hosts the conversational interface, Amazon Simple Storage Service (Amazon S3) handles session storage, and Amazon Bedrock AgentCore Identity manages authentication between the frontend and the runtime. Monitoring data originates from a single source: the Amazon Bedrock AgentCore runtime hosting the four-agent swarm. Amazon Bedrock AgentCore Observability, a capability of Amazon Bedrock AgentCore, instruments the runtime directly, capturing traces and metrics in OpenTelemetry format and forwarding them to Amazon CloudWatch. Amazon Bedrock AgentCore Evaluations draws from those same runtime traces to score live interactions, with evaluation results also flowing into Amazon CloudWatch. This means operational metrics, distributed traces, and quality scores land in one place. The second monitoring layer connects to this same backend. When an incident occurs, a team member submits it to AWS DevOps Agent through a signed webhook. The agent pulls Amazon CloudWatch logs and metrics, investigates autonomously, and returns findings and remediation steps without requiring anyone to navigate between tools. Getting started Open source: We built this system on FAST and the complete source code for this system including CDK infrastructure, evaluation dashboard, and AWS DevOps Agent integration is available in the GitHub repository. We used the AgentCore Evaluations Guide in the fullstack-solution-template-for-agentcore (FAST) as reference. To use Amazon Bedrock AgentCore Evaluations, you need: AgentCore CLI (https://github.com/aws/agentcore-cli) AWS credentials with bedr [truncated for AI cost control]