AI 服務暫時不可用，以下為來源正文，待恢復後補全翻譯。
Executives need to make data-driven decisions during live business reviews, where accuracy and speed matter. A conversational agentic AI assistant can meet this need by answering data questions instantly. But the stakes are high: a wrong number or a slow response in front of leadership carries immediate professional consequences, and a capable large language model (LLM) alone can’t guarantee either. In production, that gap shows up as hallucinated metrics, API throttling, validation latency, and subjective language. Closing it requires production-ready quality assurance built into every step, from data retrieval to response delivery. This post details five techniques, implemented on Amazon Bedrock, that work together to deliver it: adaptive pipeline orchestration, cross-account multi-model failover, real-time streaming evaluation, composite evaluation framework, and data accuracy verification. Each addresses a distinct failure mode while operating as a coordinated system. This post is the second in our NarrateAI series. NarrateAI transforms business intelligence for over 4,000 AWS executive leaders through a two-layer architecture built on Amazon Bedrock AgentCore, a platform to build, connect, and optimize agents at scale, with any framework or model. The architecture comprises an Automated Narrative Generation Layer for batch processing and a Conversational AI Interface Layer for real-time interaction. Our previous post covered the business challenges, overall architecture, user experience, and enterprise deployment. This one goes deeper into the engineering, showing how the five techniques achieve approximately 99 percent numerical accuracy while streaming responses in real time. It’s written for engineers and architects building LLM applications who are familiar with LLM APIs and streaming responses. From business vision to technical implementation This post focuses exclusively on the advanced quality assurance mechanisms within the real-time layer. Consider an executive asking “Which regions aren’t meeting their targets and why?” The five techniques work together so that the response is numerically accurate and arrives in real time. It also maintains throughput under concurrent global usage and is professionally formatted for direct use in business reviews. Figure 1 shows how these five techniques form a layered dependency chain, where each layer feeds the one below it to produce a validated real-time response from raw queries. Adaptive Pipeline Orchestration routes queries by data volume so that most complete in a single fast pass while complex queries receive full parallel treatment. Cross-Account Multi-Model Failover expands available inference capacity across independent model-account quota spaces, reducing user-visible throttling. Real-Time Streaming Evaluation validates each paragraph the moment it’s produced, overlapping quality checks with generation. The Composite Evaluation Framework runs multiple independent evaluators in parallel against each paragraph. Data Accuracy Verification catches numerical hallucinations through a two-stage cascade. It starts with cheap exact matching and escalates to semantic verification only when needed. The following sections walk through each technique in the order it appears in the pipeline, from data retrieval through generation to quality assurance. Figure 1: How the five quality assurance techniques layer together to turn raw queries into validated real-time responses Adaptive pipeline orchestration Enterprise knowledge documents contain extensive information and queries vary enormously in how much of it they need. A focused question like “What’s my team’s quarterly attainment?” might draw on a handful of sections (discrete chunks of enterprise knowledge documents retrieved). A comprehensive request like “Give me a full regional performance analysis” requires synthesis across hundreds. Users expect the same response time either way. This heterogeneity requires a routing strategy that handles the common case quickly without sacrificing thoroughness for complex scenarios. No single processing strategy works for all queries, because the right approach depends on how much data a query actually requires. Single-pass concatenation is fast and cheap, but it breaks down when aggregated sections exceed the model’s context window (typically 200K tokens), forcing truncation and quality loss. Fixed multi-pass batch processing avoids truncation, but it pays for multiple LLM invocations on every query, including the roughly 90 percent that don’t need them. Our approach routes each query based on its total aggregated section volume ∣D∣ (where D is the set of retrieved document sections): single pass when the data fits, batch processing only when necessary. The result is a three-phase pipeline that classifies each query once and then processes it along the cheapest path that preserves quality. Three-phase adaptive processing The pipeline processes every query in up to three phases: Mode-Aware Consolidation packs retrieved sections and picks a route, Bifurcated Analysis executes that route as either a single fast-path call or parallel normal-path batches, and Conditional Consolidation merges the parallel results when needed. The following table summarizes how the two paths move through these phases before we walk through each phase in detail. Fast path (~90% of queries) Normal path (~10% of queries) Trigger Retrieved volume ∣D∣ ≤ θ Retrieved volume ∣D∣ > θ Phase 1: Mode-Aware Consolidation All sections concatenated into a single chunk Sections packed into N batches respecting document boundaries Phase 2: Bifurcated Analysis Single LLM call, streams directly to evaluation N parallel LLM calls, one per batch Phase 3: Conditional Consolidation Skipped entirely Merges N analyses, resolves conflicts, streams the final answer The first phase, Mode-Aware Consolidation, operates on retrieved document sections. It applies a greedy first-fit packing strategy to fill context windows up to the token limit while preserving section priority order and section boundaries. For smaller volumes (∣D∣≤θ, where θ is an empirically calibrated character-count threshold), all sections are concatenated into a single chunk, and this is called “fast path”. For large volumes (∣D∣>θ), sections are packed into optimal batches that respect document boundaries and this is called “normal path”. Threshold θ is set relative to the model’s context window limit and calibrated to capture 70–90 percent of queries on the fast path (in practice, approximately 90 percent take this path, as shown in the Results section). Because the execution mode is fixed in this first phase, each downstream phase can optimize for its specific path rather than handling both. The second phase, Bifurcated Analysis, executes the chosen path. The fast path makes a single LLM call with the complete context and streams results directly to the evaluation pipeline. The normal path distributes batches across N parallel LLM invocations (where N is the number of batches produced by Phase 1) that analyze chunks independently to achieve near-linear speedup. The third phase, Conditional Consolidation, operates exclusively on the normal path by synthesizing N independent analyses into a unified response. The consolidation LLM receives partial analyses with the original question and applies conflict resolution heuristics while streaming the final answer. The fast path bypasses this phase entirely because the final response only uses one analysis. Figure 2 shows the complete flow, with both paths converging on the streaming response. Figure 2: End-to-end flow of the three-phase pipeline, showing the fast path bypassing consolidation In production, this threshold-based routing delivers outsized efficiency. Approximately 90 percent of queries take the fast path (single LLM call, time-to-first-token (TTFT) within a few seconds, total latency typically under 25 seconds). The remaining 10 percent (complex multi-document queries) receive the full parallel batch treatment (approximately 50–75 seconds compared to under 25 seconds on the fast path) commensurate with their complexity. With N=4 typical batches for normal-path queries, the blended cost per query is approximately 1.4 LLM invocations. That is a 72 percent reduction in LLM invocations per query versus always using the multi-pass strategy, while maintaining full quality coverage for complex queries. Teams should calibrate their own threshold to match their workload’s query volume distribution and keep it below the model’s context window limit. Cross-account multi-model failover The adaptive pipeline consolidates information efficiently, but a fast pipeline is only useful if the model behind it stays available. During peak review periods, when thousands of users run their analyses simultaneously, a single throttled request erodes confidence in the tool and drives users back to manual spreadsheet analysis. The standard mitigation, exponential backoff with jitter, proved insufficient. Amazon Bedrock assigns independent quotas per model and per account. Treating each model-account pair as its own capacity space multiplies available throughput without provisioning new infrastructure. Rather than waiting for capacity to free up, we set out to discover capacity that was never contended in the first place. This section shows how treating every model-account pair as its own quota space multiplies capacity without provisioning new infrastructure. That capacity hides in plain sight. Amazon Bedrock quotas are independent across two axes: each model has its own limits, and each AWS account receives its own quota. A 3-model × 3-account configuration therefore provides nine independent quota spaces. At runtime, the system explores this grid by attempting the highest-ranking model first, with accounts randomized to help prevent hot-spotting. The implementation is a custom Strands model provider, a drop-in replacement for the standard BedrockModel that adds transparent capacity expansion while maintaining full compatibility with existing agent code. Figure 3 shows the grid and the cascade path a request follows through it. Figure 3: The failover cascade across a 3-model × 3-account configuration, where each cell is an independent quota space and requests degrade model quality only after exhausting the accounts at the current tier Three-mechanism coordination Three mechanisms coordinate to make this work. The first is model ranking, which establishes a quality-speed hierarchy. Queries attempt the highest-ranking model first and cascade only when necessary, so users receive the best available model automatically. The second is account-level distribution, which helps prevent hot-spotting through stochastic load balancing. Before each failover attempt, Python’s random.shuffle() randomizes the AWS role ARN list order by creating a copy and shuffling it in place. Load then distributes uniformly across accounts over time without requiring complex traffic-shaping algorithms or centralized coordination. Each request gets fresh randomization, naturally achieving approximately 1/N traffic distribution where N is the number of configured accounts. This maximizes aggregate Amazon Bedrock API quota utilization across the account pool. The third is detection and fast recovery, which avoids backoff entirely. The ThrottlingDetector catches ThrottlingException, ServiceQuotaExceededException, and TooManyRequestsException, immediately attempting the next model-account combination. AWS Security Token Service (STS) AssumeRole obtains fresh credentials in 100–200ms, which is negligible compared to the multi-second delays of exponential backoff with jitter. Over a six-month production deployment across over 4,000 users, the N×M quota exploration (where N is the number of models and M is the number of accounts) absorbed traffic surges and redu [truncated for AI cost control]