AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。
Fine-Tuning Agentic AI: A Practical Guide - MachineLearningMastery.com Fine-Tuning Agentic AI: A Practical Guide - MachineLearningMastery.com In this article, you will learn how to fine-tune an agentic AI system holistically, covering all four critical dials: training data, parameter-efficient fine-tuning, runtime hyperparameters, and preference alignment. Topics we will cover include: How to build and validate a well-formatted tool-calling fine-tuning dataset that prevents hallucinated function calls before training ever begins. How to configure and apply QLoRA for parameter-efficient fine-tuning, and how to tune inference-time hyperparameters such as temperature and retry policy with the same rigor as training hyperparameters. How to use Direct Preference Optimization (DPO) to teach judgment calls that supervised fine-tuning alone cannot express, and how to evaluate the result with a verdict-driven framework that catches catastrophic forgetting before it ships. Agentic AI fine-tuning, tool calling, LoRA, QLoRA, DPO, and agent hyperparameters all show up in the same search results because they are all part of the same underlying problem, and most guides only cover one piece of it. Fine-tune the base model well and ship it with the wrong runtime temperature, and it will still fail in production. Get the temperature right but train on a badly formatted tool-calling dataset, and it will still hallucinate function names. This article treats agentic AI fine-tuning as what it actually is: a system with four separate dials — training data, parameter-efficient fine-tuning, runtime hyperparameters, and preference alignment — and walks through tuning all four together rather than one in isolation. One example runs through the whole guide: a support-ticket triage agent being fine-tuned to reliably call three internal tools, lookup_order, issue_refund, and escalate_to_human, rather than answering from a general instinct about what sounds right. Prerequisites: Python 3.10+ pip install peft transformers datasets accelerate for the training-side examples (a real training run additionally needs bitsandbytes and a CUDA GPU, called out specifically where it matters below); no special hardware is needed for the dataset, hyperparameter, and evaluation examples, which run anywhere Why “Fine-Tuning an Agent” Means More Than Fine-Tuning a Model Before touching any of the four levers, it is worth being clear about when fine-tuning is even the right tool. Frontier base models are already excellent general instruction-followers, and what fine-tuning actually fixes in 2026 comes down to three things: exact output schema, narrow domain vocabulary, and consistent behavior that a prompt alone cannot reliably pin down. What it does not fix is missing knowledge; if your agent needs facts that did not exist at training time, that is a retrieval problem, not a fine-tuning problem, and no amount of training will make a model reliably know something it was never shown. Once fine-tuning is the right call, “fine-tuning the agent” splits into four genuinely separate problems, and skipping any one of them is a common way these projects underperform: The training data: does it teach the actual behavior you need, in the format the model will see at inference time? Parameter-efficient training: how you actually update the weights without needing a datacenter. Runtime hyperparameters: temperature, iteration limits, retry policy — all decided after training, at inference time, and just as capable of breaking a well-trained model as a bad training run. Preference alignment: teaching judgment calls that a single “correct” training label cannot express. The rest of this article covers all four, in order, against the same triage-agent example. Building the Tool-Calling Fine-Tuning Dataset Format matters more than volume for this specific kind of fine-tuning. A base model can already write fluent English about refund policy; what it does not reliably do is emit a syntactically exact tool call with the right argument names every time, and that is a formatting problem that a few hundred well-structured examples can fix far more reliably than a few thousand loosely formatted ones. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 # dataset.py import json TOOLS_SCHEMA = [ { "name": "lookup_order", "description": "Retrieves order details by order ID.", "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}}, "required": ["order_id"]}, }, { "name": "issue_refund", "description": "Issues a refund for an order. Only call this after confirming eligibility.", "parameters": { "type": "object", "properties": {"order_id": {"type": "string"}, "amount": {"type": "number"}}, "required": ["order_id", "amount"], }, }, { "name": "escalate_to_human", "description": "Hands the ticket to a human agent. Use for anything ambiguous, high-value, or policy-adjacent.", "parameters": {"type": "object", "properties": {"reason": {"type": "string"}}, "required": ["reason"]}, }, ] def make_example(user_message: str, tool_name: str, tool_args: dict) -> dict: return { "messages": [ {"role": "system", "content": "You are a support triage agent with access to tools."}, {"role": "user", "content": user_message}, { "role": "assistant", "content": None, "tool_calls": [{"type": "function", "function": {"name": tool_name, "arguments": json.dumps(tool_args)}}], }, ] } def validate_examples(examples: list[dict]) -> list[str]: """Schema validation, before training starts, not after a wasted run.""" valid_tool_names = {t["name"] for t in TOOLS_SCHEMA} tools_by_name = {t["name"]: t for t in TOOLS_SCHEMA} errors = [] for i, example in enumerate(examples): for message in example["messages"]: if message["role"] != "assistant" or "tool_calls" not in message: continue for call in message["tool_calls"]: name = call["function"]["name"] if name not in valid_tool_names: errors.append(f"Example {i}: unknown tool '{name}'") continue required = set(tools_by_name[name]["parameters"].get("required", [])) provided = set(json.loads(call["function"]["arguments"]).keys()) missing = required - provided if missing: errors.append(f"Example {i}: tool '{name}' missing required args {missing}") return errors Code explanation: every training row is stored in the same role/content chat format most current SFT trainers expect natively, which means the dataset plugs straight into a trainer without a custom collator to write and debug. validate_examples is the part worth taking seriously; it checks every tool call in the dataset against the real tool schema before a single training step runs, catching an unknown tool name or a missing required argument. Testing this against a deliberately broken pair of examples — one calling a tool that does not exist, one missing a required argument — the validator catches both correctly. That is a cheap, five-minute check that prevents training a model on a dataset that would teach it to hallucinate arguments, which is a far more expensive mistake to discover after a training run finishes. For scaling past a hand-written seed set, the current standard approach is synthetic generation with judge filtering rather than manual labeling at volume: write 150 to 200 seed examples by hand, expand them with a stronger teacher model, then score every generated row for instruction adherence and correctness and discard the bottom 10–20% before it ever reaches the trainer. Parameter-Efficient Fine-Tuning with QLoRA With a validated dataset in hand, QLoRA on a single high-memory GPU is the default starting point for most teams; it freezes the base model in 4-bit precision and trains a small set of low-rank adapter matrices on top, which is what lets a 70B-class model fit on hardware that a full fine-tune could not touch. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 from transformers import AutoModelForCausalLM from peft import LoraConfig, get_peft_model, TaskType model = AutoModelForCausalLM.from_pretrained( "your-base-model", load_in_4bit=True, device_map="auto", ) lora_config = LoraConfig( r=4, # rank of the adapter matrices, lower = fewer trainable params lora_alpha=32, # scaling factor applied to the adapter's output lora_dropout=0.05, # regularization on the adapter, helps on small datasets target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], task_type=TaskType.CAUSAL_LM, ) peft_model = get_peft_model(model, lora_config) peft_model.print_trainable_parameters() Code explanation: r, the rank, controls how expressive the adapter is, and it is the single hyperparameter worth understanding first, since it directly trades capacity against overfitting risk and adapter size. lora_alpha scales the adapter’s contribution relative to the frozen base weights, and the r=4, alpha=32, dropout=0.05 combination shown here is not arbitrary; it is the exact configuration used in a peer-reviewed tool-agent fine-tuning setup, tested specifically for tool-calling behavior on small instruct models. load_in_4bit=True is the part that requires a real CUDA GPU; quantized loading at this level does not run meaningfully on CPU, so this specific step needs real hardware. This mechanic was verified by direct wrapping. Since the load-in-4-bit step requires a GPU, a small model architecture was built locally and the identical LoraConfig logic was applied to it, confirming the adapter wrapping correctly freezes the base model and isolates the trainable parameters to a small fraction of the total — exactly the behavior QLoRA depends on. In that test, only 1.7% of total parameters ended up trainable, with the rest of the base model correctly frozen, confirming the config and wrapping code is structurally correct before it ever touches a real base model. Tuning the Agent’s Runtime Hyperparameters This is the step most fine-tuning guides skip entirely, and it is a mistake, because a perfectly trained model can still fail in production purely from bad inference-time settings. Temperature, the number of iterations an agent is allowed per task, and whether a failed tool call gets a retry are all decided after training, at inference time, and they measurably change real task success. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 # hyperparam_sweep.py import random random.seed(7) def simulate_agent_turn(temperature: float, allow_retry: bool) -> bool: """Returns True if the agent ends the turn with a valid tool call.""" base_error_rate = 0.08 error_rate = base_error_rate + (temperature * 0.15) made_error = random.random() base_error_rate return False def run_sweep(n_trials: int = 2000) -> dict: configs = [ {"temperature": 0.0, "allow_retry": False}, {"temperature": 0.7, "allow_retry": False}, {"temperature": 0.7, "allow_retry": True}, {"temperature": 1.0, "allow_retry": True}, ] results = {} for cfg in configs: successes = sum(simulate_agent_turn(cfg["temperature"], cfg["allow_retry"]) for _ in range(n_trials)) key = f"temp={cfg['temperature']}, retry={cfg['allow_retry']}" results[key] = successes / n_trials return results Code explanation: this models a fine-tuned agent whose baseline error rate rises with temperature — standard, well-documented behavior — but which also has a real chance to self-correct if a retry at a safer, deterministic setting is allowed after a failed call. Adding a single retry at temperature 0 after a failed call raised the success rate for the temperature-0.7 configuration to 98.7%, higher than either single-shot setting alone. The practical takeaway is that a retry policy is often a cheaper, faster lever than additional trai [truncated for AI cost control]