LLM as judge
Using a language model to grade another model's output against a written rubric, for qualities that exact comparisons cannot check.
Code can check an answer with one right form, such as a JSON shape. Many agent answers can be worded a hundred ways. An LLM judge reads the input, the output and a rubric ("Does the reply answer the question and cite the order number?") and returns a score or a verdict. It is one kind of graderGrader: The logic that scores one result in an eval: an exact comparison, a schema check, a script, a model acting as judge, or a person. in an evalEval: A test for an AI system: give it an input, run it, and apply grading logic to the result to decide whether it succeeded..
In absolute (single-answer) grading, the judge scores one output on a scale. In pairwise grading, it sees two outputs and says which is better, or that they tie. Pairwise suits regression tests: is the new version at least as good as the old?
Zheng et al. (2023) named several biases of judges:
- Position bias: "a propensity to favor certain positions over others". In the paper's tests most judges favoured the first answer.
- Verbosity bias: a judge "favors longer, verbose responses, even if they are not as clear, high-quality, or accurate as shorter alternatives".
- Self-enhancement bias: judges "may favor the answers generated by themselves". The authors note their data could not settle whether this effect is real.
They also found judges grading maths and reasoning poorly, misled by the answers under review even on problems they could solve alone. Among their fixes: swap the order of the two answers and count a win only when both orders agree, and give the judge a reference answer.
Example
A pairwise judge compares a new reply with the golden reply twice, swapping the order. It prefers the new reply both times: a win. It prefers whichever came first both times: that is position bias, and the pair counts as a tie.
Common confusion
A judge's score is easy to mistake for ground truth, but it is a measurement with its own errors. Calibrate it first: have people grade a sample, run the judge on the same sample and check that they agree. Judges can also be steered by text inside the output they grade, so keep code checks for anything that can be checked exactly. LLM-as-judge vs deterministic checks covers which to use where.
In L2M
Golden sets have an llm_judge scorer with a rubric, absolute or pairwise mode and 1 to 5 votes. Pairwise mode asks both orders, in random sequence, and notes when the judge picked the same position both times. Default pass marks are 0.7 (absolute) and 0.5 (pairwise: at least as good as the golden run). Judges are off until EVAL_JUDGE_ENABLED=true, the outputs are framed as data rather than instructions, and every judge call counts toward the job's cost and budget.
Further reading
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (Zheng et al., NeurIPS 2023 Datasets and Benchmarks, arXiv)