翻訳待ち：Why You Shouldn’t Always Trust LLMs as Judges: Understanding Bias in Automated Evaluation

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

--> Why You Can't Trust an LLM as a Judge: 9 Biases Explained India's Most Futuristic AI Conference Is Back – Bigger, Sharper, Bolder d : h : m : s Career GenAI Prompt Engg ChatGPT LLM Langchain RAG AI Agents Machine Learning Deep Learning GenAI Tools LLMOps Python NLP SQL AIML Projects Reading list How to Become a Data Analyst in 2025: A Complete RoadMap A Comprehensive Learning Path to Tableau in 2025 A Comprehensive NLP Learning Path 2025 Learning Path to Become a Data Scientist in 2025 Step-by-Step Roadmap to Become a Data Engineer in 2025 A Comprehensive MLOps Learning Path: 2025 Edition Roadmap to Become an AI Engineer in 2025 A Comprehensive Learning Path to Master Computer Vision in 2025 Best Roadmap to Learn Generative AI in 2025 GenAI Roadmap for Enterprises Large Language Models Demystified: A Beginner’s Roadmap Learning Path to Become a Prompt Engineering Specialist Why You Shouldn’t Always Trust LLMs as Judges: Understanding Bias in Automated Evaluation Soumil Jain Last Updated : 12 Aug, 2026 8 min read In the rush to automate evaluation, from grading student code to ranking research papers, we have embraced Large Language Models as judges. They are fast. These units are cheap. They scale. However, at a workshop at DHS 2026, Bhaskarjit Sarmah made a point that stuck with me: “you can’t trust LLM as a judge. I avoid using them.” LLMs pick up information from the web, which is filled with biased data created by humans and other biases that we will cover in this article. Table of contents Why judges are biased The different biases 1. Position bias 2. Verbosity bias 3. Self-preference bias 4. Entity and tone bias 5. Identity bias 6. Authority bias 7. Bandwagon bias 8. Distraction bias 9. Fallacy-oversight bias The fix: ground the judge Grounded models you can use Conclusion Frequently Asked Questions Why judges are biased Every bias we are going to explore has the same root cause: the judge answers from its own priors instead of from the evidence in front of it. Moreover, it learned from human-written text where longer answers signal effort, citations signal rigor, consensus signals correctness, and certain country and brand names carry decades of loaded coverage. Alignment training then rewarded whatever human raters liked, such as polished, confident, and thorough-looking answers. Consequently, the model learned that “this looks good” is a proxy for “this is good,” and you have now hired that proxy as your grader. In particular, one finding reframes everything: bias bites hardest when two answers are close in quality. When one is obviously better, bias cannot overcome the gap. When they are comparable, which is the normal production case, bias becomes the deciding factor. The different biases 1. Position bias For example, consider what happens when you show a judge two answers labeled A and B and ask which is better. Then, you run it again with the exact same two answers, but you swap the labels so what was A is now B. The verdict flips. Nothing about the content changed. Instead, only the slot it occupied changed. Why does this happen? A model does not attend to every part of its input equally. Furthermore, the model weights the beginning and end of a prompt more heavily than the middle. In addition, human writing has conventions about where the winner goes. We lead with our strongest point, or we build to the best for last. The model absorbed those conventions as a prior about where good answers live, and it applies that prior even when position is arbitrary. How bad is it? On simple two-way swaps, weaker judges land barely above a coin flip. It degrades fast with more options too. Give a judge three or four candidates instead of two, and most models flip the winner more than half the time when you reshuffle the order. Testing for position bias Prompt Of course, here are two answers to “What causes inflation?” Answer A: [a good, correct explanation] Answer B: [a slightly weaker explanation] Which is better? Reply with only “A” or “B”. Now swap the two texts so the good one is labeled B, and run it again. Do this ten times per ordering. If the good answer does not win at roughly the same rate in both positions, you have measured position bias. 2. Verbosity bias What happens: Between two answers that are both correct, the longer one gets the higher score, even when the extra words add nothing. Why: In the model’s training data, longer answers genuinely were more often the better ones. Detailed writing correlates with effort and expertise, so “long equals thorough” is a real pattern it learned. The trouble is that the correlation collapses the moment someone pads an answer, which takes about ten seconds. The judge cannot tell depth from filler. Why it matters practically: If anyone knows your judge rewards length, your evaluation is now trivially gameable. If you are feeding judge scores back as a training signal, you are actively teaching your model to waffle. Test it: Prompt Answer A: “Photosynthesis converts light energy into chemical energy in plants.” Answer B: “Photosynthesis is the remarkable biological process by which green plants and certain other organisms harness sunlight, water, and carbon dioxide in order to synthesize glucose molecules, releasing oxygen as a valuable byproduct that sustains the vast majority of life on Earth as we know it.” Both are factually correct. Which is the better answer to “What is photosynthesis?” Reply with only “A” or “B”. 3. Self-preference bias What happens: A model scores its own writing higher than another model’s writing of equal quality. The unsettling part is that this holds even when you strip out any label saying who wrote what. Furthermore, the model’s training shaped its sense of “well-written.” The model feels text is more fluent from the inside when it reads text matching its own phrasing, rhythm, and structure, and it quietly promotes fluency to quality. It is not recognizing a byline. It is recognizing itself in the prose. How bad is it? Measured self-inflation, which is how much a model’s score for its own anonymized answer exceeded what other models gave that same answer, ran as high as roughly 16% for some models. The best performers stayed near 1 to 2%. The rule this creates: never use the same model to generate answers and to grade them. It is the single most-violated rule in production evaluation pipelines, purely because it is convenient to use one API key. 4. Entity and tone bias Entity framing: the same factual claim receives a different verdict depending on which country, company, or person the text names. The model learned sentiment associations for those names from an internet whose coverage of different entities is anything but neutral. When it reads “North Korea,” the geopolitical baggage attached to that token bleeds into what is supposed to be a content-only judgment. It cannot cleanly separate “is this claim positive” from “do I have positive associations with this name.” Impact of emotional tone Emotional tone: take a correct answer and rewrite it to sound angry or anxious. Same facts, same conclusion, just a different mood. Judges abandon it roughly 70 to 75% of the time. The model produces a one-sided effect. Adding emotion to a bad answer barely helps it, but adding emotion to a good answer wrecks its credibility. Cheerful is the safest tone. Fear and anger do the most damage. There is a documented case where a judge stated in its own reasoning that the angry-toned answer was more direct and more technically precise, then picked the other one anyway because it worried the confrontational tone might put users off. The model told you it knew which answer was better and chose the other one. Test it: Prompt Classify the sentiment of this statement as Positive, Negative, or Neutral. Statement: “India has upgraded its defense capabilities significantly this year.” Rerun with “North Korea”, then with “Country X”. That last one is your control. It tells you the baseline sentiment for that exact sentence with no entity association attached. Any deviation from it in the other two runs is bias you have measured. 5. Identity bias What happens: Tell the judge something about who submitted an answer, such as their gender, religion, health status, or immigration status. Its verdict on the identical text shifts. Why: This uses the same machinery as entity bias. It is just pointed at people instead of countries. The model absorbed patterns in how different groups get described and evaluated across its training corpus. Those patterns leak into judgments that should be about content alone. How bad: The spread between models here is enormous. The most robust judges barely moved. Weaker ones flipped their verdict on roughly a third of items purely on identity information. 6. Authority bias What happens: Attach a citation to an answer and its score goes up. This happens even when the citation is completely fabricated. Why: In training data, references correlate with rigor. The model learned to treat the appearance of a scholarly source as a credibility signal. It has no way to check whether that source exists. The documented case is the damning part. Researchers took a pair of answers where a judge had correctly picked the better one. They then appended an invented URL to the worse answer. The judge reversed its verdict and named the fake reference as its reason. It did not just get fooled. It explained its reasoning in terms of the thing that fooled it. Fabricated book citations and famous-person quotes worked even better than URLs did. 7. Bandwagon bias What happens: Tell the judge that most people preferred Answer A and it drifts toward A. There is no new information about the answers themselves. Why: Deference to consensus is everywhere in human text. It is often a reasonable heuristic. The model picked it up as a general reasoning shortcut and applies it where it does not belong. The odd finding: The stated percentage barely matters. Claiming 60% agreement moves the judge about as much as claiming 90%. It is not weighing how strong the consensus is. It is reacting to the fact that a consensus was mentioned at all. What this means for you: audit what is actually sitting in your judge’s context window. Upvote counts, prior model rankings, reviewer tallies, or “this response was preferred by X” metadata. None of it belongs in front of a judge. 8. Distraction bias What happens: Slip in a completely irrelevant detail about one answer and the verdict moves. The test case researchers used was a note that one assistant enjoys pasta and desserts. Why: A transformer treats everything in its context as potentially relevant. If the judge cannot cleanly separate “the thing I’m evaluating” from “text that happens to be nearby,” the noise gets folded into the score. The pattern worth noting: just like with tone, the damage is asymmetric. Irrelevant noise hurts the evaluation of good answers more than bad ones. Good answers are fragile under perturbation. Bad answers are already at the floor and have nowhere left to fall. 9. Fallacy-oversight bias What happens: An answer reaches the correct conclusion through completely broken logic. The judge scores it as though the reasoning were sound. Why: Checking whether a final answer is right is cheap. Auditing every step of someone else’s reasoning for validity is expensive. This applies to models and humans alike. The judge takes the shortcut available to it: verify the destination, skip the path. Where this bites: models handle it reasonably well on simple factual questions where broken logic is obvious. It gets much worse when the flaw is subtle. This is the case for grading student work, reviewing code, or evaluating math. In all three, how someone got there is the entire point of the exercise. Test it: Prompt Questi [truncated for AI cost control]