翻訳待ち：Teaching an Open Model to Do Science

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

Teaching an Open Model to Do Science Teaching an Open Model to Do Science B +2 Bojan Jakimovski, Sara Kovachovska, Maziyar Panahi 12 min read • Jul 30, 2026 Case Studies Post-training an open model for tool use, biological reasoning, and auditable research workflows Guest post from Loka, developed in collaboration with Arcee AI, and AWS. Our earlier Trinity Mini work taught a compact model to classify drug–protein relations from biomedical abstracts. This project asked more of the same open model: investigate a scientific question with tools, reason across incomplete evidence, and return work another researcher can check. Loka, AWS, Arcee, and Prime Intellect built two reinforcement-learning environments around that goal. One teaches the model to seek evidence across biomedical tools. The other teaches it to infer Gene Ontology annotations from protein evidence and return strict JSON. We then ran 21 controlled post-training experiments and promoted the configuration that performed well on both tasks. Results at a glance Run 120 was the selected checkpoint. On the held-out Drug Tool evaluation, its score rose from 70.8% to 81.2%, increasing at every evaluation point. On held-out BioReason, it reached 0.863 on the reported combined score, which includes GO term F1, GO tree similarity, aspect coverage, and JSON validity. Before reinforcement learning, one GEPA prompt-search pass improved base-model validation by about 84% on BioReason and 7.7% on Drug Tool. These figures measure prompt-search validation before post-training and are separate from the final results. Selection also depended on tool traces, verifier outputs, citations, identifiers, structured responses, and behavior inside a working scientific application. Scientific work rarely begins as a clean classification problem. A researcher may start with a disease and a target rather than an answer set. The first query may fail. A protein may have several identifiers. One paper may support a hypothesis while another weakens it. A useful system has to decide what evidence to seek next, distinguish missing evidence from negative evidence, and leave a record of how it reached its conclusion. From prompting to learned scientific behavior A prompt can describe a procedure, but it does not make that procedure reliable across a multi-step run. Instructions to prefer primary evidence, resolve identifiers, use tools, and return valid JSON compete with partial results and formatting constraints. Models also tend to produce a helpful-looking answer when the evidence is weak. That failure mode is especially costly in science. Invalid JSON is easy to catch. A polished synthesis about the wrong protein can pass quietly into the next stage of a workflow. We therefore focused the research on two questions: What feedback would reward the scientific behavior we wanted? What held-out evaluations would show whether that behavior improved? Reward design followed from those questions. Drug Tool evaluated tool choice, arguments, retrieved facts, completion, efficiency, and concision alongside the final prose. BioReason evaluated biological content, output structure, and JSON validity. There is also an operational reason to specialize an open model. A compact model and a small LoRA adapter can run inside an organization’s cloud boundary, remain pinned to a known policy version, and be updated against that organization’s evidence and standards. The team controls the data path, reward, evaluation, serving cost, and incident response. This operational case complements the scientific case for better feedback. Two environments, two kinds of scientific work The training mixture used two published Prime RL environments in equal proportion. Together they cover investigation and conclusion. Drug Tool RL: learning to investigate The Loka Drug Tool SFT dataset came from the prompt bank used by Loka’s drug-discovery application. Arcee Trinity Large Thinking generated assistant and tool trajectories through OpenRouter’s OpenAI-compatible interface. Each example preserves the biomedical question, tool schemas, assistant actions, and serialized tool results. The dataset contains 800 training prompts and 200 held-out prompts across 17 workflow categories, with 5,049 structured tool calls in total. The categories include disease–pathway evidence, literature triage, contradiction checking, GEO biomarker screening, KEGG resolution, protein records, interaction networks, ortholog comparison, molecular generation, protein folding, and docking planning. The lokahq/drug-tool-rl@3 environment presents a biomedical question, a tool catalog, and a completion contract. The model must choose and parameterize tools, recover when retrieval fails, and finish with a concise synthesis. Seven retrieval tools cover PubMed, GEO, KEGG, UniProt, and STRING. When enabled, NVIDIA NIM-backed tools add protein folding, ligand docking, and molecular generation. Identical calls are cached within a rollout, and structural artifacts can move between stateful tools. Its reward combines grounded facts, successful tool use, appropriate tool choice and arguments, completion, efficiency, concision, and final-answer quality. Diagnostics record hallucinated identifiers, tool errors, rate limits, duplicate calls, and evidence overlap. A fluent answer after failed retrieval should score differently from a synthesis grounded in retrieved evidence. Calling many tools without reaching a useful conclusion should also lose credit. BioReason: learning to conclude The BioReason RL corpus contains 8,630 curated records across training and held-out splits. It builds on BioReason-Pro and combines protein metadata and sequence with InterPro, interaction, and subcellular-location evidence. Each record also includes GO-GPT candidate terms. The model must evaluate these noisy hypotheses rather than copy them as labels. In lokahq/bioreason-go-rl@1, the model must infer a functional summary and lists of Gene Ontology identifiers for Molecular Function, Biological Process, and Cellular Component. It must return exactly one JSON object. The verifier scores GO F1 by aspect, mean GO F1, tree similarity, aspect presence, and strict JSON validity. The final objective emphasized accuracy for aspects present in the evidence while retaining a format check. The evaluation protocol ran 200 held-out Drug Tool examples and 512 held-out BioReason examples every 20 training steps. These results measure behavior under specified environments and verifiers. They are not claims about clinical validity, wet-lab performance, or success across an entire drug program. Improve the task before training the policy The system prompt is part of an RL environment. It changes the actions a policy considers and the errors a verifier can observe. Before GRPO post-training, we ran one GEPA prompt-optimization pass on each environment. Trinity Mini produced the base-model rollouts. A Claude Sonnet 5 reflection model used those rollouts and verifier feedback to propose revised task instructions. The selected prompts improved base-model validation by about 84% on BioReason and 7.7% on Drug Tool. Figure 1. GEPA prompt search before RL. Starting from the same base-model validation index of 100, the selected prompts raised BioReason to 184 and Drug Tool to 107.7. The dashed line marks the shared starting point; each bar is the lift the reflection loop found before GRPO ever ran. We then retrained with the selected prompts and repeated the full held-out evaluation. The environments responded differently: one retained a post-training benefit, while the other largely absorbed the prompt gain during RL. That result shaped how we used GEPA. We ran prompt search once, before RL, to improve the task definition. We did not keep rewriting prompts around an already trained checkpoint. The rest of the research budget went to data, reward, and optimization changes. Figure 2. Dataset map. Drug Tool and BioReason turn distinct scientific records into verifiable learning signals. One path teaches evidence-seeking behavior; the other teaches precise biological annotation. Model and training setup Trinity Mini is a 26-billion-parameter mixture-of-experts model with 3 billion active parameters and a 128k native context window. It has 128 experts; eight routed experts and one shared expert are active for each token. Every experiment began from the same original arcee-ai/Trinity-Mini checkpoint. Training used GRPO with LoRA adapters. Prime Intellect’s prime-rl separated policy optimization from rollout inference and connected both to the two verifiable environments. Training and rollout workers ran separately, while the environments returned rewards and diagnostics. The active training context was 16k tokens. Rollouts allowed up to 7,168 completion tokens. Drug Tool training allowed 12 turns, while held-out Drug Tool evaluation allowed eight. Across the series, we kept the original model checkpoint, Muon optimizer, held-out protocol, and four-plus-four GPU topology fixed. Those controls made each run easier to interpret. A configuration change could be compared with a known baseline instead of a moving system. Figure 3. One AWS path from training to deployment. prime-rl builds GRPO batches across separate trainer and rollout-generation workers on Amazon EC2 P5. Loka's environments compute rewards, Amazon S3 preserves checkpoints and adapters, and the promoted LoRA adapter moves into the serving and agentic harness path. Click the figure to zoom into the full-resolution view. Twenty-one runs, one change at a time A promotion bar, a bounded search space, and persistent experiment records guided the 21 runs. The final recipe emerged from that process. GOAL.md defined the threshold across both environments. CONTEXT.md stored the fixed protocol, known failure modes, and important measurements. PLAN.md enforced a one-hypothesis-per-run rule. Each launch had a versioned configuration under rl/. runs.jsonl recorded settings, metrics, hypotheses, and decisions, while reports under analysis/ captured curves, component metrics, and representative rollouts. THREAD.md carried the current state and next hypothesis between runs. The loop was simple: Inspect held-out curves, component metrics, and representative rollouts. State one hypothesis about a failure mode or opportunity. Change a bounded part of the configuration or environment. Re-run both evaluations and separate model behavior from infrastructure failures. Keep, reject, rerun, or retune the change, then record the decision. Each run left enough evidence for another person to understand what changed and why. The final ledger entry for run 120 recorded: 1{ 2 "id": "120", 3 "research_question": "Which complete configuration is ready for workflow testing?", 4 "evidence": { 5 "drug_tool_avg1": 0.812, 6 "bioreason_avg1": 0.863, 7 "held_out_protocol": "fixed environment splits" 8 }, 9 "artifacts": [ 10 "configuration", 11 "evaluation curves", 12 "trace review" 13 ], 14 "decision": "promote" 15} This record mattered during handoffs and restarts. The next run began from written evidence instead of memory, and rejected directions remained visible. Why run 120 was promoted Run 120 met the promotion bar on both environments. Drug Tool increased from 70.8% to 81.2% during training. BioReason reached 0.863 on its combined held-out metric. Figure 6. Drug Tool held-out evaluation.Figure 7. BioReason held-out evaluation.Figure 8. Drug Tool reward during run 120.Figure 9. BioReason reward during run 120.Figure 10. Combined reward during run 120. This is also where run 120 shines. It reached a BioReason accuracy of about 86%, a combined score built from GO term F1, tree similarity, and JSON structure validity together, while Drug Tool accuracy climb The BioReason number combines several checks. It measures overlap with reference [truncated for AI cost control]