待翻译：Preparing data for supervised fine-tuning Part 1: Formatting and quality

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

Data preparation determines the ceiling of any supervised fine-tuning (SFT) project. You’ve evaluated your foundation model (FM), and out-of-the-box performance isn’t meeting your production requirements. Maybe the model doesn’t follow your output schema reliably, struggles with your domain’s classification taxonomy, or can’t maintain the tone your application demands. The question isn’t whether to customize, it’s how. This post assumes you have decided to fine-tune a foundation model and are evaluating how to prepare data for that work. Post-training customization provides three distinct levers. Each addresses a different gap between what the model can do today and what you need it to do. Continued pre-training (CPT) ingests large volumes of unstructured domain text to expand the model’s knowledge base. Use CPT when the model lacks familiarity with your domain’s terminology, concepts, or data patterns. Supervised fine-tuning (SFT) trains on curated input-output pairs to reshape the model’s behavior. SFT teaches the model how to respond: following instructions, adhering to schemas, adopting a specific tone, or producing structured outputs. It doesn’t inject new knowledge. It teaches the model to apply what it already knows in the way that you need, an idea sometimes called the Superficial Alignment Hypothesis. Reinforcement fine-tuning (RFT) optimizes behavior through reward signals rather than explicit demonstrations. RFT works when you can programmatically evaluate output quality but can’t easily demonstrate the reasoning path at scale. These techniques aren’t mutually exclusive. A production pattern is CPT, then SFT, then RFT: first expand knowledge, then shape behavior, then optimize through feedback. In practice, CPT is used less often and only required when the base model lacks critical domain vocabulary or knowledge your task requires. Foundation models like Amazon Nova are already pre-trained on a broad corpus, so SFT followed by RFT is usually sufficient. This post, the first of a two-part series, covers the foundations of SFT data preparation: quality checks, formatting requirements, and train/evaluation splits. We use code snippets from Amazon Bedrock documentation to illustrate key concepts while keeping the guidance applicable to any model you choose. The second post covers advanced strategies: readiness evaluation, data subset selection and filtering, data augmentation, and data mixing. Data quality checks Before you invest in formatting or training infrastructure, audit your raw data. Catching problems early saves substantial time and compute cost downstream. Accuracy and correctness Every response in your dataset should be a gold-standard answer you would be comfortable deploying to production. Incorrect examples can teach the model a persistent bad habit that’s difficult to unlearn. This risk is especially acute in SFT, where the model isn’t learning new facts so much as learning which patterns to imitate. A wrong demonstration gets imitated. The practical upshot is that quality beats quantity by a wide margin. LIMA showed that 1,000 carefully curated examples can match models trained on orders of magnitude more data. AlpaGasus showed that filtering an instruction set down to its cleanest 20 percent can train faster and score higher than the full set. If you’re working with human-annotated data, implement a multi-review process before examples enter your training set. Diversity of examples Dataset diversity is one of the strongest predictors of SFT success. Research on supervised fine-tuning scalability identifies two properties that govern how well fine-tuning generalizes. The first is semantic coverage, the breadth of task domains and prompt phrasings represented. The second is information depth, the richness of individual examples. A dataset that covers only a narrow slice of your production traffic will produce a model that performs well on that slice and poorly everywhere else. Audit your dataset for prompt variation first. Your examples should include the different ways users phrase the same intent, because a model trained only on “Summarize this document” won’t reliably handle “Give me the key takeaways.” Check domain and topic breadth next. If your task spans multiple domains, verify that each is represented proportionally to its production frequency. Then look at difficulty range: include straightforward cases alongside complex, multi-step problems, weighted toward the complexity profile of real traffic. Finally, explicitly include edge cases such as ambiguous inputs, incomplete information, and out-of-scope requests, paired with the responses you want the model to produce in those situations. A practical approach is to cluster your examples by embedding similarity and inspect the resulting clusters for gaps. Sparse or missing clusters indicate areas where the model will lack training signal. For example, a customer-support dataset might cluster into password resets, billing questions, and shipping updates. If refunds are a common request in production but no cluster covers them, that empty region is your gap. The model will have little signal for refund conversations, so collect or write refund examples before training. The second post in this series shows how to automate this idea with data subset selection methods. Consistency within similar tasks While diversity across your dataset is critical, examples that handle the same type of situation should be internally consistent. Inconsistency within the same task type sends contradictory signals about correct behavior. For example, if similar prompts produce both bullet-point and paragraph-form answers, the model won’t learn a reliable default for response structure. If you need concise, two-sentence answers, don’t include paragraph-length responses in your training data. Target behavior Two inconsistent examples (avoid) Two consistent examples (use) Return one lowercase label from a fixed set Positive This one reads as fairly negative. positive negative Summarize in two sentences, no preamble Login fails after reset iOS 17 Escalated to Tier 2 Sure! Here’s a quick summary: the customer reports that… Customer cannot log in after resetting their password on iOS 17. The ticket is escalated to Tier 2. Customer’s card was declined three times at checkout. Billing confirmed the issuer blocked the transaction. Decline out-of-scope requests with one fixed sentence I can’t help with that. I’m really sorry, but that falls outside what I’m able to assist with today. However, I’d be happy to… That request is outside the scope of this assistant. That request is outside the scope of this assistant. Deduplication Duplicate or near-duplicate examples cause the model to overfit on those patterns, inflating their importance relative to the rest of your dataset. Duplication is less critical in SFT than in pre-training, but it can creep in when you merge data from multiple annotators, combine datasets across projects, or use synthetic data generation. In those cases, apply both exact-match and semantic deduplication before training. Toxicity and safety screening Scan your dataset for harmful, biased, or inappropriate content. Even if your use case is narrow, the model can internalize patterns from problematic examples and surface them in unexpected contexts. Use automated classifiers, such as the open Llama Guard, to flag content for human review, and establish clear guidelines for what constitutes acceptable training data in your domain. Data formatting With quality checks complete, you can structure your data for training. Formatting isn’t just about syntax. How you structure examples shapes the model’s learned behavior, because SFT teaches the model to respond to a specific shape of input, not just its content. System prompt in the sample Include a system prompt in your training examples when you plan to use one during inference. The system prompt establishes context, persona, and constraints that shape model behavior. If training data lacks system prompts but inference includes them, you create a distribution mismatch that can degrade behavior. Conversational format (JSONL) Most modern SFT pipelines, including Amazon Nova recipes, use a conversational JSONL format where each line is a self-contained JSON object representing a conversation. The Amazon Nova 2.0 models use the Converse API format: { "schemaVersion": "bedrock-conversation-2024", "system": [{"text": "You are a helpful coding assistant."}], "messages": [ { "role": "user", "content": [{"text": "Write a Python function to check if a string is a palindrome."}] }, { "role": "assistant", "content": [ {"text": "def is_palindrome(s):\n cleaned = s.lower().replace(' ', '')\n return cleaned == cleaned[::-1]"} ] } ] } Key formatting rules to follow: One JSON object per line, with no pretty-printing across multiple lines. Validate every line parses as valid JSON before uploading. Maintain strict role alternation between user and assistant turns. Include system messages when your production setup uses them. Reasoning traces For models with reasoning capabilities (such as Amazon Nova 2.0 with reasoning_enabled: true), include intermediate thinking steps using the reasoningContent field in assistant turns. Training on reasoning traces is what transfers the behavior popularized as chain-of-thought from a prompting trick into the model itself. { "schemaVersion": "bedrock-conversation-2024", "system": [{"text": "You are a financial analyst. Provide data-driven answers with supporting calculations."}], "messages": [ { "role": "user", "content": [{"text": "Calculate YoY revenue growth. 2024: $4.2M, 2025: $5.1M"}] }, { "role": "assistant", "content": [ { "reasoningContent": { "reasoningText": { "text": "The user asks for year-over-year revenue growth. I need to calculate the percentage change: (new - old) / old x 100. That gives (5.1 - 4.2) / 4.2 x 100 = 21.43%." } } }, {"text": "YoY revenue growth is approximately 21.4 percent: ($5.1M - $4.2M) / $4.2M x 100."} ] } ] } The hardest part of using reasoning traces isn’t the format. It’s getting traces that actually teach the model something. Useful traces are faithful to the answer, meaning the reasoning actually leads to the final response. They’re proportional to difficulty: short problems get short traces and long problems get long ones, a trade-off that difficulty-aware trace compression makes explicit. They’re complete, with no thought leaps, because expert-written rationales often skip intermediate steps the model hasn’t yet learned. And quality matters more than quantity: s1 and LIMO both show that under 1,000 carefully curated reasoning demonstrations can elicit strong reasoning in a capable base model. The second post in this series covers how to source these traces at scale through distillation and self-generation. Use plain text for reasoning content and keep it directly relevant to the problem-solving process. When reasoning is enabled during training, it should also be enabled during inference for consistent behavior. Keep in mind that training on a non-reasoning dataset with reasoning_enabled: true can cause the model to lose its reasoning capabilities, because it learns to generate responses without applying reasoning. Tool calling and multimodal formats SFT supports training models for tool use (function calling) and multimodal understanding (documents, images, video). For tool calling, toolUse blocks appear in assistant turns and toolResult blocks in user turns, each referencing a unique toolUseId: { "schemaVersion": "bedrock-conversation-2024", "system": [{"text": "You are an expert in composing function calls."}], "toolConfig": { "tools": [ { "toolSpec": { "name": "getItemCost", "description": "Retrieve the cost of an item from the catalog", "inputSchema": { "json": { "type": "object", "properties": { "item_id": { "t [truncated for AI cost control]