待翻译：How OneAdvanced deployed over 50 AI agents on UK-sovereign AWS

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

This post is co-authored with OneAdvanced team Deploying AI agents on a United Kingdom (UK)-sovereign AWS architecture requires careful decisions about model hosting, data residency, and agent orchestration. OneAdvanced, a UK-based enterprise software provider serving over 10,000 customers, needed to deliver AI capabilities while making sure that no data would leave the UK. At the time, the specific models they wanted, Llama 4 Maverick and Llama Guard 4, weren’t yet available through managed services in the UK region. OneAdvanced took a different path: self-hosting open-weight large language models (LLMs) on AWS infrastructure they fully control. In this post, we describe how OneAdvanced built a UK-sovereign AI solution using Llama 4 Maverick and Llama Guard 4 on Amazon SageMaker AI. The solution pairs a Retrieval Augmented Generation (RAG) pipeline backed by Amazon Aurora PostgreSQL-Compatible Edition with the pgvector extension, over 50 specialized agents powered by Strands Agents SDK, and a tool layer running on Amazon Elastic Container Service (Amazon ECS). We walk through the architecture, key implementation decisions, and results. The challenge: Data sovereignty and model hosting OneAdvanced provides sector-focused software as a service (SaaS) solutions to organizations in healthcare, legal, and many other regulated industries. Their customers handle sensitive data daily, including patient records, legal case files, and compliance documentation. These customers expect that AI tooling meets strict data residency, security, and privacy standards. As Andrew Henderson, CTO of OneAdvanced, explains in the OneAdvanced AI launch video: “Data sovereignty, particularly in the UK, is a hard requirement for many of our customers, especially those in the public sector and highly regulated industries. They need to know exactly where their data is, who has access to it, and that it resides within the UK’s legal and regulatory framework to support total compliance and trust.” OneAdvanced initially prototyped with Amazon Bedrock and saw rapid results within a two-week sprint: chat completion, an Amazon Bedrock Agent for querying UK statute law, Snowflake data integration, and chart generation. However, to meet their sovereignty requirements, OneAdvanced needed to host models exclusively in their own UK-based AWS accounts. At the time of the engagement, the models they wanted, Llama 4 Maverick and Llama Guard 4, weren’t yet available through AWS managed services in the UK region. Self-hosting meant deploying, serving, and scaling these models while building a production-grade solution around them. This included content moderation, document retrieval, agent orchestration, and a no-code agent builder for non-technical users. Solution overview The following diagram shows the high-level architecture of the OneAdvanced AI solution. Figure 1: High-level architecture of the OneAdvanced UK-sovereign AI solution This architecture enabled OneAdvanced to achieve full UK data sovereignty and rapid agent deployment, supporting their ISO 42001 certification for AI governance while maintaining complete control over their model serving infrastructure. The solution consists of four components. vLLM serves Llama 4 Maverick (FP8) and Llama Guard 4 on Amazon SageMaker AI, running on p5.48xlarge instances in the London (eu-west-2) region. Over 50 Strands agents run on Amazon ECS, each with its own system prompt, tool configuration, and optional input form, with agent configuration stored in Amazon DynamoDB. Documents uploaded to Amazon Simple Storage Service (Amazon S3) are converted to markdown, chunked, and embedded into pgvector for retrieval. Llama Guard 4 checks user inputs for harmful content before the request reaches the main model. A typical request flows like this: the user sends a message, and Llama Guard checks it for harmful content (evaluated before the main inference model). The request routes to the appropriate Strands agent on Amazon ECS. The agent calls tools and retrieves relevant documents from pgvector and Amazon S3 as needed, or invokes specialist tools including web search. Prerequisites To self-host an AI model on AWS, you need the following resources and skills: An AWS account with access to p5.48xlarge instances in your target Region. For information about requesting GPU instance quota, refer to the Amazon SageMaker AI Developer Guide. Intermediate knowledge of deploying containers on Amazon ECS and managing storage with Amazon S3. For getting started, refer to the Amazon ECS Developer Guide. Access to your chosen AI model weights (Llama 4 Maverick and Llama Guard 4 require license acceptance on Hugging Face). Experience with Python-based machine learning (ML) frameworks and container orchestration. A PostgreSQL database with the pgvector extension enabled for vector similarity search. Model deployment on Amazon SageMaker AI OneAdvanced serves Llama 4 Maverick (meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8) and Llama Guard 4 (meta-llama/Llama-Guard-4-12B) using vLLM on Amazon SageMaker AI endpoints. They deploy on p5.48xlarge instances in the London region, using Hugging Face models with AWS Deep Learning Containers. Longer context windows drove the move to P5 instances. OneAdvanced targets 120K–128K token context lengths to support use cases like large document analysis and multi-turn conversations. During the advisory engagement with AWS, load testing with vLLM on P5 instances validated that the infrastructure could handle their throughput requirements. OneAdvanced started on p4d.24xlarge instances and moved to p5.48xlarge for production, also taking advantage of reserved instance discounts on GPU compute. Llama Guard 4 replaced an earlier deployment of Llama Guard 3 after OneAdvanced observed high false rejection rates with the previous version. The guard model runs serially before the main model, screening user inputs for harmful content before inference begins. Building over 50 agents with Strands Agents SDK A distinctive aspect of the OneAdvanced solution is its agent library: over 50 task-specific agents spanning healthcare, legal, HR, marketing, logistics, and more. Agents include a Care Incident Response assistant, a Clinical Safety Bulletin generator, a scheme of work generator for education, an operational scenario simulation, a performance review assistant, a Document Comparison tool, and an AWS Architect Agent, among many others. OneAdvanced went from their first agent to over 50 in only three weeks, with most agents built in less than a day. OneAdvanced evaluated several agentic frameworks, including LangChain, LangGraph, and others, before choosing Strands Agents SDK. The decision came down to a few factors: Strands takes a model-first approach with no rigid workflow definitions, it supports turn-taking and interview-style interactions, and it allowed OneAdvanced to move quickly from idea to deployed agent. As Nick Heap, Principal Software Engineer at OneAdvanced, explains: “Post-evaluation, Strands stood out as the clear frontrunner for the project. Its comprehensive suite of tools not only met our requirements but also offered a future-proof solution that closely aligned with our in-house vision.” Each agent is defined with a system prompt, a set of tools, and an optional structured input form. Agents are containerized and deployed on Amazon ECS, with runtime configuration stored in Amazon DynamoDB. Users browse the agent catalog and select the one that fits their task. OneAdvanced also built a no-code agent builder that non-developers can use to create and configure agents through a visual interface. Users define the agent’s persona, design input forms with drag-and-drop fields, write system prompts with references to form fields (using an @ syntax to inject form values), and select from a library of available tools. This design makes agent creation accessible to product managers, clinicians, and business analysts without writing code. Agents draw from a shared tool library that includes calculator, chart creation, file content reader, mermaid diagram generator, organization and personal knowledge search, spreadsheet query (including Snowflake integration), text file query, UK statute law search, and web search (opt-in per organization and per user query). For example, an agent can query a Snowflake database and then generate a chart from the results in a single interaction. OneAdvanced also uses sub-agents and advanced agentic patterns to maintain the context window across complex interactions. A notable design pattern is the interview-style agent. Some agents, like the Strategic Thinking assistant, ask structured questions before providing advice: “Question 1 of 3: What specific pain points does your AI solution aim to address?” This builds context through turn-taking rather than expecting users to provide everything upfront. Focused, guided interactions produce better outcomes than open-ended prompts. RAG pipeline The RAG system gives agents access to both personal and organization-level document collections. Users upload documents through the file management interface into either a personal space or a shared organization space on Amazon S3. Documents are converted to markdown and chunked into 2,048-token segments before embedding into pgvector for vector similarity search. For longer documents, recursive summarization handles content that exceeds the chunk size. For embeddings, OneAdvanced uses the intfloat/multilingual-e5-large-instruct model, chosen for its strong multilingual support and instruction-following capabilities. Rather than adopting an off-the-shelf RAG framework, OneAdvanced built their own retrieval system, internally called “Llamadex,” giving them full control over the pipeline. They initially chose pgvector for its simplicity and the speed it offered in getting to production. As the solution matures, OneAdvanced is reviewing this strategy to evaluate alternatives. When an agent needs to answer a question grounded in uploaded documents, it calls the appropriate retrieval tool, “Personal knowledge search” or “Organization knowledge search,” which queries pgvector and returns relevant chunks along with source references. Users can see exactly which documents were used to generate a response, supporting transparency and trust. Security and responsible AI Security runs through every layer of the OneAdvanced AI solution, a direct consequence of serving customers who handle sensitive data daily in sectors like health and legal. The infrastructure runs in the London AWS Region. No user data leaves the UK. No user queries or responses are retained or used for model training. As Nick Heap wrote: “An AI service that is only hosted in the UK and data is not used to train the AI model and OneAdvanced engineers do not have access to the data.” Uploaded documents in personal and organization spaces are fully isolated and not readable even by OneAdvanced staff. Llama Guard 4 screens user inputs for harmful content before they reach the main model. Privacy controls are customizable at organizational levels. On the infrastructure side, Amazon GuardDuty provides threat detection on the Amazon S3 document storage layer. OneAdvanced holds ISO 42001 certification for AI governance—a certification they report being among the first organizations in the UK and Europe to achieve. They’re also a signatory to the EU AI Pact, reflecting a proactive approach to responsible AI across their products and operations. Results Through the AWS advisory engagement, OneAdvanced went from prototype to production on a UK-sovereign AI solution. OneAdvanced deployed Llama 4 Maverick on P5 instances in the London region, giving them full control over model serving infrastructure with no data leaving the UK. They built and shipped over 50 Strands agents in only three weeks, covering use cases across [truncated for AI cost control]