LLM Evaluation Frameworks Compared: How to Actually Measure What Your Model Does

LLM Evaluation Frameworks Compared: How to Actually Measure What Your Model Does - MachineLearningMastery.com

LLM Evaluation Frameworks Compared: How to Actually Measure What Your Model Does - MachineLearningMastery.com

In this article, you will learn how to evaluate LLM applications using the three dominant open-source frameworks — RAGAS, DeepEval, and Promptfoo — and why the LLM-as-a-judge mechanism they all rely on has measurable biases you need to actively design around.

Topics we will cover include:

How RAGAS, DeepEval, and Promptfoo differ in purpose and when to use each one, including which pairings experienced teams converge on.

How to implement a faithfulness check and a CI-gated quality evaluation with working code you can run immediately.

What position bias, self-preference bias, and verbosity bias are, how to detect them with an audit harness, and how to mitigate them in production.

There’s a lot to get through, so let’s get right into it.

Introduction

You ship an LLM feature after seeing a couple of outputs and decide it looks good. Three weeks later, a prompt tweak silently breaks something nobody was testing for, and nobody notices until a user complains. This is the default failure mode for LLM applications, and it’s different from a typical software bug. Traditional code fails with a stack trace. LLM outputs fail by being confidently, plausibly wrong, which is exactly the kind of failure a quick manual glance won’t catch.

Three open-source tools dominate the practical side of this problem in 2026: Promptfoo, DeepEval, and RAGAS. Each is built for a different shape of problem, not competing for the same job. Layered above them are production-monitoring platforms like LangSmith and Braintrust, which pick up where offline evaluation leaves off. None of these tools wins outright; most mature GenAI QA programs run two of them in parallel: a lightweight framework for blocking bad deploys plus a platform for ongoing monitoring and human review.

This article compares the frameworks that actually matter, walks through tested code for the two most common evaluation jobs, and covers the part most comparison pieces skip entirely: the fact that “LLM-as-a-judge“, the mechanism nearly every framework here relies on, has measurable, published biases you need to design around, not just trust.

What “Evaluating an LLM” Actually Means

Before comparing tools, it helps to separate three things people conflate when they say “LLM evaluation.” Picking the wrong category here is the single most common mistake teams make.

Model benchmarking compares raw model capabilities on standardized academic tasks, such as MMLU, GSM8K, and HumanEval. lm-evaluation-harness is the standard here, with no real substitute when the requirement is a standardized academic benchmark. If you’re choosing between GPT-5 and Claude for a new project, this is the category you want, but it tells you almost nothing about whether your specific application works.

Application evaluation asks a narrower, more useful question: does your RAG pipeline, chatbot, or agent produce correct, grounded, safe outputs on your data and your prompts? This is where RAGAS, DeepEval, and Promptfoo live, and it’s where this article spends most of its time.

Production monitoring tracks live traffic after deployment, catching regressions and drift that offline test sets never anticipated. This is LangSmith, Braintrust, and Arize Phoenix territory.

Most people asking “which eval framework should I use” actually need the second category, often paired with the third. The rest of this article focuses on that.

The Metrics Underneath the Frameworks

Before the framework comparison makes sense, it’s worth knowing what’s actually being scored, because every tool below implements some version of the same handful of metrics.

Faithfulness (or groundedness) checks whether an answer contains only claims supported by the retrieved context — the core mechanism for catching RAG hallucinations.

Context precision and recall check whether retrieval pulled the right documents, and only the right ones, before generation even happens.

Answer relevancy checks whether the response actually addresses the question asked, independent of whether it’s factually grounded.

G-Eval, introduced by Liu et al., uses chain-of-thought prompting combined with form-filling to guide an LLM judge through an explicit rubric, and has been shown to align with human preference more closely than naive “rate this 1-10” prompting. Beyond these, most frameworks add task-specific checks for toxicity, bias, and PII leakage.

The real differentiator between frameworks isn’t metric novelty; they mostly implement the same handful of ideas. It’s workflow fit: how the metric gets triggered, where the result goes, and whether it blocks a deploy or just generates a report.

RAGAS vs. DeepEval vs. Promptfoo, Head to Head

RAGAS is research-backed, with academic-grade methodology behind metrics like faithfulness, context precision, and context recall, but it’s scoped to retrieval and generation scoring, with no production monitoring or collaboration layer built in. Pick it when your architecture is retrieval-heavy and you want metrics with a published paper behind their definition, not just a vendor’s internal heuristic.

DeepEval is Python-native and pytest-based, with 14-plus metrics spanning hallucination, bias, toxicity, and RAG-specific checks — built explicitly to function as a CI/CD quality gate that can block a deploy. Pick it when evaluation needs to live inside your existing test suite rather than as a separate offline report someone has to remember to run.

Promptfoo is CLI-first and YAML-config-driven, strongest at multi-model prompt comparison and adversarial red-teaming, with 500-plus built-in attack vectors in its security-testing suite. Pick it for prompt engineering iteration across multiple models, or when red-teaming and security testing are the actual requirement.

The framing that matters most: DeepEval and RAGAS aren’t really competitors. DeepEval covers broad LLM application testing, RAGAS specializes specifically in RAG, and a meaningful share of production teams run both together — with RAGAS scoring the retrieval-specific dimensions and DeepEval handling everything else inside the same CI pipeline.

Category RAGAS DeepEval Promptfoo

Best for RAG-specific scoring CI/CD quality gates Multi-model comparison, red-teaming

Integration style Python library pytest-native YAML + CLI

Strongest metric set Faithfulness, context precision/recall 14+ metrics incl. bias, toxicity Security/attack vectors (500+)

Production monitoring No No No

Pairs well with DeepEval (broader coverage) RAGAS (RAG-specific depth) Either for prompt-side testing

Code Walkthrough: Catching Hallucination with a Faithfulness Check

Here’s the mechanism behind RAGAS’s faithfulness metric, demonstrated directly: decompose an answer into atomic claims, then check each claim against the retrieved context. A claim with no support in the context is a hallucination — exactly the failure mode that a quick manual read tends to miss, because the unsupported detail often sounds completely plausible.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

faithfulness_check.py

Prerequisites: none beyond Python's standard library (re)

Run: python faithfulness_check.py

#

Note: this demonstrates the faithfulness-checking MECHANISM that RAGAS's

real Faithfulness metric implements with an LLM judge. The keyword-overlap

check below is a simplified, fully offline-testable stand-in for that

LLM-based claim verification -- swap in RAGAS's actual metric for production use.

import re

def decompose_claims(answer: str) -> list[str]:

"""Split an answer into atomic, independently-checkable statements."""

sentences = re.split(r'(? bool:

"""

Check whether a claim has lexical support in the retrieved context.

RAGAS does this with an LLM judge; this overlap check demonstrates

the same supported/unsupported decision in a deterministic way.

"""

claim_words = set(re.findall(r'\b[a-zA-Z]{4,}\b', claim.lower()))

context_words = set(re.findall(r'\b[a-zA-Z]{4,}\b', context.lower()))

if not claim_words:

return True

overlap = len(claim_words & context_words) / len(claim_words)

return overlap >= 0.5

def compute_faithfulness(answer: str, context: str) -> dict:

"""

Faithfulness score = fraction of claims in the answer supported by context.

This mirrors RAGAS's actual metric definition: supported claims / total claims.

"""

claims = decompose_claims(answer)

supported = [c for c in claims if claim_supported_by_context(c, context)]

unsupported = [c for c in claims if c not in supported]

score = len(supported) / len(claims) if claims else 1.0

return {

"score": round(score, 3),

"total_claims": len(claims),

"unsupported_claims": unsupported,

}

if name == "main":

context = "Abuja became the capital of Nigeria in 1991, replacing Lagos as the seat of government."

Case 1: fully grounded answer -- every claim traces back to the context

grounded_answer = "The capital of Nigeria is Abuja. It became the capital in 1991."

result_1 = compute_faithfulness(grounded_answer, context)

print("Grounded answer:")

print(f" Faithfulness score: {result_1['score']}")

print(f" Unsupported claims: {result_1['unsupported_claims']}\n")

Case 2: the model adds a plausible-sounding detail the context never mentioned

hallucinated_answer = (

"The capital of Nigeria is Abuja. It became the capital in 1991. "

"The city has a population of over 3 million people."

)

result_2 = compute_faithfulness(hallucinated_answer, context)

print("Answer with a hallucinated detail:")

print(f" Faithfulness score: {result_2['score']}")

print(f" Unsupported claims: {result_2['unsupported_claims']}")

How to run (no dependencies required):

1

python faithfulness_check.py

Output:

1

2

3

4

5

6

7

Grounded answer:

Faithfulness score: 1.0

Unsupported claims: []

Answer with a hallucinated detail:

Faithfulness score: 0.667

Unsupported claims: ['The city has a population of over 3 million people.']

That population figure sounds entirely reasonable, which is exactly why a manual review would likely let it through. The faithfulness check catches it because it’s checking against the actual retrieved context, not against general plausibility. This is the mechanism running underneath RAGAS’s real Faithfulness metric, which uses an LLM to do the claim decomposition and support-checking instead of keyword overlap — more accurate, same underlying logic.

To run this with the actual RAGAS library against a live model:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Production pattern using the real RAGAS library

pip install ragas

from ragas import SingleTurnSample, EvaluationDataset

from ragas.metrics import Faithfulness

from ragas import evaluate

sample = SingleTurnSample(

user_input="What is the capital of Nigeria?",

response="The capital of Nigeria is Abuja. It became the capital in 1991. The city has a population of over 3 million people.",

retrieved_contexts=["Abuja became the capital of Nigeria in 1991, replacing Lagos as the seat of government."],

)

dataset = EvaluationDataset(samples=[sample])

results = evaluate(dataset, metrics=[Faithfulness()])

print(results)

Code Walkthrough: CI-Gated Evaluation with DeepEval

The pattern that makes DeepEval distinct from a standalone evaluation script is that it runs as a real pytest test, meaning a quality regression fails the build the same way a broken unit test would — instead of generating a report someone has to remember to read.

1

2

3

4

5

6

7

8

[truncated for AI cost control]