跳到主要內容
AI News HubLIVE
來源內容 · 翻譯待補全6 分鐘閱讀

待翻譯:Build long-horizon agents with Grok 4.7 on Amazon Bedrock

文章摘要

AI 服務暫時不可用,以下為來源摘要,待恢復後補全翻譯:Grok 4.7 is now available on Amazon Bedrock. In this post, we build a tool-calling agent on Grok 4.7 and evaluate it on three reasoning tasks at four reasoning effort levels, measuring correctness and token cost per completed task. We show how to tune the trade-off per request and monitor it in production.

來源AWS Machine Learning Blog作者: Suheel Farooq
待翻譯:Build long-horizon agents with Grok 4.7 on Amazon Bedrock
報告錯誤

更正渠道尚未開通,可先複製下方文章資訊留存。

查看更正說明
直接讀正文

AI 服務暫時不可用,以下為來源正文,待恢復後補全翻譯。

SpaceXAI’s Grok 4.7 is now available on Amazon Bedrock, which offers a choice of foundation models (FMs) from multiple providers. According to SpaceXAI, Grok 4.7 is trained to work longer on more difficult tasks and knows when to check its own work. That matters most for agents, which carry a task through many steps and must verify what they find along the way. An agent that works this way has to do more than look things up. It has to form a candidate answer, check it against what it has learned, and refine it until it fits. Grok 4.7 exposes a reasoning effort setting that controls how much deliberation happens before each response, and for agent workloads a practical question is which level to run. In this post, we build a tool-calling agent on Grok 4.7 and evaluate it on three tasks that require search rather than lookup. The tasks are diagnosing a two-service outage from its alerts, scheduling maintenance under interacting constraints, and choosing an optimal set of migrations under a budget. Each task has exactly one correct answer, verified by brute force. We ran all three at every reasoning effort level, six times each, for 72 agent runs. The agent answered 67 of the 72 runs correctly. At xhigh it answered all 18 correctly, and at low it answered 15 of 18 while using about an eighth of the output tokens. That gives a clear trade-off you can tune per request, and we show how to measure it on your own tasks and how to monitor it in production. We report token counts rather than currency throughout. Token prices change, and the counts are what your own costs are derived from, so a measurement expressed in tokens stays valid. What SpaceXAI reports about Grok 4.7 According to SpaceXAI, Grok 4.7 uses a larger base model than Grok 4.6 and was trained with a longer reinforcement learning run on a harder mix of tasks. This mix is weighted toward problems that take many hours to complete. SpaceXAI reports that the model is better at verifying its own work and at managing longer context. SpaceXAI’s published benchmark results follow that emphasis. The largest reported gains over Grok 4.6 appear in the categories that measure sustained work rather than single-turn answers. Benchmark Category Grok 4.6 Grok 4.7 Terminal-Bench 4.0 Multi-hour terminal work 20.3% 37.6% EEBench Electrical engineering 53.0% 64.0% Harvey Legal Agent Benchmark Legal work 15.8% 19.6% CursorBench 4.0 Longer-running coding 40.4% 46.3% HealthBench Professional Clinical reasoning 48.5% 56.7% AA Briefcase v1.1 Multi-hour office work 1,546 1,657 Source: SpaceXAI, Introducing Grok 4.7 (https://x.ai/news/grok-4-7). Scores are as published by SpaceXAI at the time of the model launch. Benchmarks show how a model performs across standard tasks. For your own workload, the useful next step is to measure on tasks shaped like yours, which is what the rest of this post walks through. SpaceXAI lists a 500K token context window for Grok 4.7, with text and image input and text output. On Amazon Bedrock, the model supports configurable reasoning effort, tool calling, and structured output. How you access Grok 4.7 on Amazon Bedrock Grok 4.7 runs on the bedrock-runtime endpoint and is accessed through a cross-Region inference profile. Two profiles are available: Inference profile Routing Use when us.xai.grok-4.7 Within the US geography You have US data residency requirements global.xai.grok-4.7 Commercial AWS Regions You have no residency constraints and want the widest capacity Cross-Region inference routes requests across multiple AWS Regions, which gives you higher throughput without managing capacity Region by Region. Requests name one of the inference profiles rather than a bare model ID. The model supports the OpenAI-compatible Chat Completions and Responses APIs, and the Amazon Bedrock Converse API. On Converse, reasoning is returned as a reasoningContent block alongside the text block. In inferenceConfig, set maxTokens only, and put model-specific parameters, such as reasoning effort, in additionalModelRequestFields. For the current list of supported Regions, service tiers, and token prices, see the Grok 4.7 model card in the Amazon Bedrock User Guide and the Amazon Bedrock pricing page (https://aws.amazon.com/bedrock/pricing/). Prerequisites An AWS account with access to Amazon Bedrock, and model access enabled for Grok 4.7. Python 3.9 or later, with openai, boto3, and aws-bedrock-token-generator installed. AWS credentials configured in your environment. Your AWS Identity and Access Management (IAM) identity needs bedrock:InvokeModel on both the inference profile and your account’s default project: arn:aws:bedrock:{region}::inference-profile/us.xai.grok-4.7 or the global. equivalent. arn:aws:bedrock:{region}:{account-id}:project/default. Include both resources in the policy statement. The default project permission is required in addition to the inference profile. Connect to the model For production use, we recommend generating a short-term bearer token from your existing AWS credentials, so authentication stays tied to your IAM identity and no long-lived secret is stored: from aws_bedrock_token_generator import provide_token from openai import OpenAI REGION = "us-west-2" MODEL_ID = "us.xai.grok-4.7" client = OpenAI( api_key=provide_token(region=REGION), base_url=f"https://bedrock-runtime.{REGION}.amazonaws.com/openai/v1", timeout=360, max_retries=0, # retries are handled explicitly, see "Build in retries" below ) Reasoning tokens count as output tokens and draw down max_completion_tokens before the answer text is produced, so size the limit to leave room for both. If the limit is reached during reasoning, the response returns without answer text, so check for empty content in agent code and raise the limit if you see it. Our runs used a limit of 32,768 tokens, which left headroom at every effort level. Grok 4.7 supports four effort levels: low, medium, high, and xhigh. Effort levels can change between model versions, so it is worth confirming them against the service rather than hardcoding a list: def supported_effort_levels(candidates=("none", "low", "medium", "high", "xhigh", "max")): supported = [] for level in candidates: try: client.responses.create(model=MODEL_ID, input="Say OK.", reasoning={"effort": level}, max_output_tokens=2048) supported.append(level) except Exception as exc: if "unsupported_value" not in str(exc) and "invalid_value" not in str(exc): raise return supported print(supported_effort_levels()) # ['low', 'medium', 'high', 'xhigh'] Design tasks that need reasoning Many agent evaluations test diligence: read a record, filter it, add it up. Those tasks are worth having, but to see what reasoning effort contributes, a task has to require search: the answer can’t be read off the data, it has to be proposed and then checked. We built a synthetic environment of 20 services, exposed through seven tools, and three tasks on top of it. Every task has exactly one correct answer, and each is built so that a quick heuristic is not enough to find it. Task What the agent has to do What makes it a reasoning task two_fault_diagnosis Two services failed silently at 03:00 and 12 others raised alerts. Identify the two that failed Failed services raise no alerts, so the answer is not among the alerting services. 190 possible pairs, and only one explains exactly the alerts observed maintenance_schedule Patch eight services over four nights, two per night, under ten constraints and a rule that two services from the same team never share a night 2,520 possible schedules, one valid. The team rule depends on data the agent has to look up migration_budget Choose candidates to migrate that maximize risk reduction within a budget, including transitive prerequisites 53 feasible sets, one optimal. Picking the best ratio first gives a set nine points short of the optimum Four practices keep a measurement like this trustworthy, and they apply to most agent evaluations. State every rule the grader relies on. If the prompt leaves an edge case open, a reasonable answer can be scored as wrong, and the evaluation ends up measuring the specification rather than the model. The diagnosis prompt, for example, says explicitly that a failed service raises no alert of its own. Prove each task has one answer. Each environment is small enough to enumerate, so we checked every candidate answer before running anything. For the schedule, that is a few lines: import itertools ALL_SLOTS = sorted(set(itertools.permutations([1, 1, 2, 2, 3, 3, 4, 4]))) def valid_schedules(services, constraints): return [s for slots in ALL_SLOTS if satisfies(s := dict(zip(services, slots)), constraints)] assert len(valid_schedules(SERVICES, CONSTRAINTS)) == 1 # exactly one answer assert len(valid_schedules(SERVICES, CONSTRAINTS_WITHOUT_TEAM_RULE)) > 1 # the rule matters Grade a required final line, not the whole response. Each prompt asks the agent to end with a line such as FINAL: auth, checkout. The grader reads only that line, so a response that discusses other candidates while explaining its reasoning is graded on its conclusion: def final_line(answer): lines = [l for l in (answer or "").splitlines() if "FINAL" in l.upper()] return lines[-1] if lines else None Track format separately from correctness. A missing final line is a different signal from an incorrect one. In our runs, all 72 responses included a gradeable final line. Constrain tool parameters with an enum where the valid values are known, so every tool call targets a real resource: TOOLS = [ {"type": "function", "function": { "name": "get_dependencies", "description": "Services that one service directly depends on.", "parameters": { "type": "object", "properties": {"service": {"type": "string", "enum": SERVICE_NAMES}}, "required": ["service"], }}}, # The remaining six tools follow the same shape. ] The system prompt is deliberately short, and asks the agent to check its answer before giving it: AGENT_SYSTEM = ( "You are an infrastructure analyst agent. Use the tools to gather facts and never guess " "a value you can look up. Before answering, check your answer against every rule in the " "task. End with the FINAL line exactly as the task specifies." ) The agent loop is a standard tool-calling loop: call the model with reasoning_effort set, execute each returned tool call, append the result keyed by tool_call_id, and repeat until the model stops requesting tools. Tracking prompt_tokens and completion_tokens across the loop gives the token counts reported in the following section. Results Six runs of each task at each effort level, 72 runs in total. The agent answered 67 correctly. Task low medium high xhigh two_fault_diagnosis 5 of 6 6 of 6 6 of 6 6 of 6 maintenance_schedule 5 of 6 5 of 6 5 of 6 6 of 6 migration_budget 5 of 6 6 of 6 6 of 6 6 of 6 All three tasks 15 of 18 17 of 18 17 of 18 18 of 18 Per run low medium high xhigh Mean output tokens 1,184 6,430 6,778 9,147 Output tokens relative to low 1.0x 5.4x 5.7x 7.7x Mean tool calls 17.2 18.7 17.8 18.1 Median elapsed time 18s 98s 89s 106s Figure 1: Share of runs answered correctly at each reasoning effort level (left), and mean output tokens per run with median seconds per run overlaid (right) Three observations. xhigh answered every run correctly. All 18 runs across the three tasks were correct at xhigh, and medium and high answered 17 of 18 each. With six runs per task and level, the differences between levels are indicative rather than statistically conclusive, and the direction was consistent across all three tasks. Effort changes how much the agent reasons, not how much it explores. Tool calls were essentially flat, between 17.2 and 18.7 per run at every level. The agent gathered the same facts regardless of effort, and what changed was how much reasoning it applied to them, with output tokens rising from 1,184 at low to 9,147 at xhig [truncated for AI cost control]

展開要點與分析

文章情報

工程師進階

要點

  • AI 服務暫時不可用,系統已先保留來源內容與降級元數據。
  • Grok 4.7 is now available on Amazon Bedrock. In this post, we build a tool-calling agent on Grok 4.7 and evaluate it on three reasoning tasks at four reasoning effort levels, meas…

要點與分析由自動化流程生成,可能有誤,請結合原始來源核實。