待翻译：Top LLM Observability and Evaluation Platforms in 2026: Langfuse, LangSmith, Braintrust, Arize, and More Compared

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

LLM applications fail in ways traditional software does not. The same prompt can produce different outputs. A retrieval step can return the wrong document while every HTTP status reads 200. An agent can loop through fourteen tool calls, burn thousands of tokens, and deliver a confidently wrong answer. Standard application performance monitoring (APM) alone does not capture this semantic behavior — prompt and output quality, retrieval relevance, or agent-level reasoning traces. This is the gap LLM observability and evaluation platforms fill. They record every span of an LLM pipeline — prompts, completions, retrievals, tool calls, token counts, latencies, and costs — and then score outputs for quality using automated evaluators. In 2026, this category has moved from optional tooling to core infrastructure for any team running AI in production. The market data reflects the shift. The Business Research Company sizes the LLM observability platform market at $2.69 billion in 2026, up from $1.97 billion in 2025, and projects $9.26 billion by 2030 at a 36.2% forecast CAGR. Gartner predicts that by 2028, LLM observability investments will account for 50% of GenAI deployments, up from 15% in early 2026. LangChain’s State of Agent Engineering survey of 1,300+ professionals found that 57% of respondents now run agents in production. Nearly 89% have implemented observability for their agents. Evaluation lags behind: 52.4% run offline evaluations, 37.3% run online evaluations, and 29.5% report no evaluation at all. Quality was cited by 32% as the top barrier to production deployment. This article compares the leading platforms across three axes: tracing depth, evaluation capability, and production monitoring. Figures were checked against primary sources (company documentation, press pages, and announcements) as of August 2026; where only secondary reporting exists, it is linked and identified as such. Rankings and “best for” judgments are editorial assessments, not measured benchmarks. How the Category is Structured in 2026 The market has split into four camps, and understanding the split matters more than any individual feature list. AI-native observability platforms: Langfuse, LangSmith, Braintrust, Arize, Opik — treat the LLM trace as the primary object. They capture nested spans across agents, retrievers, and tools, and attach evaluation scores to production traffic. Open-source and source-available evaluation libraries and platforms: Arize Phoenix, DeepEval (Confident AI), MLflow, RAGAS — focus on scoring outputs: faithfulness, hallucination, answer relevance, and task completion, often via LLM-as-a-judge. AI gateways: Helicone, Portkey, LiteLLM — sit as a proxy between the application and model providers. They add logging, caching, cost tracking, and routing with minimal code changes. APM extensions: Datadog LLM Observability, New Relic, Dynatrace — bolt LLM tracing onto existing infrastructure monitoring so AI signals correlate with CPU, memory, and network metrics. One standard now connects all four camps. The OpenTelemetry GenAI semantic conventions define vendor-neutral gen_ai.* span attributes for model calls, token usage, agent steps, and tool executions. OpenTelemetry, a CNCF project, maintains these conventions, which are adopted by platforms including Google Cloud, AWS, Azure, and Datadog. The conventions now live in a dedicated repository, with the GenAI registry under active development as of August 2026. Coding agents are converging on the standard too: GitHub Copilot’s agent telemetry exposes gen_ai.* span trees, Claude Code provides opt-in OpenTelemetry tracing, and Codex includes native OpenTelemetry export support. Instrumenting once against gen_ai.* improves backend portability and reduces vendor-specific instrumentation, even if implementations still differ. Buyers in 2026 should treat OTel compatibility as a hard requirement, not a nice-to-have. The Three Axes: Tracing, Evals, and Production Monitoring Because vendors use these terms loosely, precise definitions help before comparing platforms: Tracing is the record of everything an LLM application did. A trace contains nested spans: the user input, each retrieval call, each model invocation with its exact prompt and parameters, each tool execution, and the final output. Depth matters because agent traces are deeply nested with heavy payloads — a single conversation can generate megabytes of data across dozens of runs and tool calls. Non-determinism makes tracing non-negotiable: the same prompt produces different outputs, so an issue cannot be reproduced without capturing the exact input, model parameters, and temperature at call time. Evals answer the question tracing cannot: was the output any good? Offline evals score curated datasets before deployment, catching regressions when a prompt, model, or retrieval index changes. Online evals score live production traffic, typically via LLM-as-a-judge, sampling traces and grading them for faithfulness, relevance, toxicity, or task completion. The hardest failures are outputs that are technically valid but wrong for the domain — a hallucinated policy, a drifting tone, a retrieval miss that produces a confident but incorrect answer. Traditional latency, error-rate, and availability metrics do not detect these semantic quality failures. Production monitoring closes the loop: dashboards, cost attribution per model and user, latency percentiles, drift detection across prompts and use cases, and alerting when quality scores fall. The best platforms feed production traces back into eval datasets, so every real-world failure becomes a future regression test. A platform can be strong on one axis and weak on another. Gateways excel at monitoring but skip deep tracing. Eval libraries score outputs but do not watch production. The platforms below are ranked on how completely they cover all three. 1. Langfuse (ClickHouse) Langfuse describes itself as the most widely adopted LLM engineering platform, and its open-source adoption numbers back a strong claim. Tracing: Langfuse captures nested traces for LLM calls, retrieval, embedding, and agent actions through OpenTelemetry, LangChain, OpenAI SDK, and LiteLLM integrations. Its signature nested trace view collapses a multi-step RAG or agent run into a stepable tree with per-span latencies and token counts. An observations-centric data model shipped in March 2026, delivering 10x+ dashboard performance gains and laying the groundwork for Langfuse v4, which the company says runs up to 165x faster. Evals: The platform supports LLM-as-a-judge evaluators, human annotation queues, custom scores, and dataset-based regression testing that runs in CI via GitHub Actions. Evaluator templates cover hallucination, toxicity, and relevance. Production monitoring: Cost breakdowns by model, user, or session, plus session replays for conversational agents. Deployment: MIT-licensed core, self-hostable via Docker Compose in minutes, or managed on Langfuse Cloud with a free tier. Langfuse is widely regarded as the self-host leader in this category. Best for: teams that want a full-featured, open-source, framework-agnostic platform with strict data-residency control. 2. LangSmith (LangChain) LangSmith is LangChain’s commercial platform for observing, evaluating, and deploying agents. It is framework-agnostic with Python, TypeScript, Go, and Java SDKs plus OpenTelemetry support, but it is the default backend for LangChain 1.0 and LangGraph 1.0, where integration requires near-zero glue code. Tracing: Full conversation and agent-run traces expose every step, tool call, and intermediate state. Polly, a built-in AI assistant, summarizes large traces to pinpoint problems. LangSmith Engine clusters production failures into prioritized issues, locates root causes in traces and code, and proposes fixes for review. Evals: LLM-as-judge, code-based, and multi-turn evaluators run on datasets or live production traces. Judges can be calibrated against human preferences, and side-by-side comparisons gate regressions before deployment. Annotation queues let domain experts review agent outputs. Production monitoring: Online evals score live traffic, and automatic trace clustering detects usage patterns and failure modes. As of 2026, LangSmith provides a unified cost view across the full agent workflow — LLM calls plus custom costs for retrieval, tools, and external APIs. Deployment: Managed cloud on AWS or GCP, hybrid, and self-hosted configurations for teams with data-residency requirements. LangSmith Deployment adds a durable agent runtime with human-in-the-loop approvals, and enforces exactly-once semantics for individual run attempts. Best for: teams building on LangChain or LangGraph, and enterprises that want observability, evals, and managed agent deployment in one vendor. 3. Braintrust Braintrust is the eval-first platform in this list, behind one of 2026’s largest funding rounds in the AI evaluation and observability category. Tracing: Framework-agnostic SDKs across Python, TypeScript, and other languages capture full agent traces. Brainstore, a purpose-built database, handles queries over millions of complex traces efficiently. Evals: This is Braintrust’s core. Versioned datasets, automated and human scoring, model and prompt experiments, and CI regression testing let eval results block regressions before deployment. A playground tests prompt changes against real production data prior to release. Loop, an AI agent, analyzes traces to suggest better prompts, generate scorers, and build datasets automatically. Production monitoring: Real-time observability across prompts, responses, tool calls, latency, cost, and quality, with monitoring for hallucination, drift, and regression. Best for: product-focused AI teams that want evaluation as the center of the workflow, with CI/CD quality gates and production feedback loops in one system. 4. Arize AX and Arize Phoenix Arize AI runs a two-tier strategy: Arize AX for enterprises and Phoenix as its source-available, self-hostable layer. Tracing: Phoenix is OpenTelemetry-native and self-hostable under the Elastic License 2.0 — source-available, though not an OSI-approved open-source license, with strong integrations for LlamaIndex and the OpenAI Agents SDK. At the Series C announcement, Phoenix had over two million monthly downloads, making it one of the most widely adopted eval libraries. Evals: Arize’s ML-observability heritage shows here. Its eval primitives run deeper than most competitors, with pre-built templates, RAG-specific quality plots, and drift detection that catches outputs quietly degrading over time. Arize also introduced audio evaluation capabilities for voice applications and funds open research through its OpenEvals and AgentEvals initiatives. Production monitoring: Embedding clustering, drift detection, and monitoring that spans both traditional ML models and generative workloads, with deep Azure AI Foundry integrations. Best for: regulated or accuracy-critical workloads that need the deepest evaluation rigor, and organizations running classic ML and LLMs side by side. 5. MLflow MLflow, the Linux Foundation open-source project backed by Databricks, has evolved into a full agent observability platform. Tracing: Native tracing for agents with trace data fully owned by the user, and export in OTel GenAI semantic convention format so nothing is locked into a proprietary schema. Evals: Built-in LLM judges, multi-turn evaluation, judge alignment with human feedback, and integrations with RAGAS, DeepEval, Phoenix, TruLens, and Guardrails AI. MLflow also ships prompt optimization using GEPA and MIPRO algorithms that improve prompts automatically from eval results. Production monitoring: An AI Gateway centralizes LLM access with routing, rate limiting, fallbacks, and usage tracking across OpenAI, Anthropic, Bedrock, Azure, and Ge [truncated for AI cost control]