LLM-as-a-Judge: Why Review Is Not Proof

If we give an LLM clear instructions, why add a review step?

From recipe to review

Think of a restaurant kitchen. A recipe tells the chef what to make, but someone still needs to taste the dish.

“Too salty.” The feedback goes back to the same chef, who adjusts the dish before it is tasted again.

LLM-as-a-Judge brings that review step to AI-generated answers. A model evaluates the response against clear criteria.

1. Draft
2. Review
3. Revise
4. Review again
5. Verify

The feedback returns to the original writer. The Judge can be a different model, or the same model in a separate call.

Download the GIF · 44-second animation

Agreement is not proof

Now imagine a support bot writes:

“Your refund of $150 has been approved.”

The Judge might flag the lack of evidence for that promise. It might also miss it.

Agreement is not proof.

Code should compare structured fields, such as the amount and approval status, with the actual refund record (for example, the refund data stored in the company’s database).

If the record says $120 and the refund isn’t approved, that response should be blocked, even if the Judge approved it.

What the deterministic check can look like

The model can return a structured draft alongside its proposed wording. The application then compares the claims with the trusted record. The Judge’s score is deliberately not part of this decision.

// Structured data proposed by the LLM
type RefundDraft = {
  amountCents: number;
  status: "approved" | "pending" | "rejected";
};

type RefundRecord = RefundDraft & { id: string };

function verifyRefundDraft(
  draft: RefundDraft,
  record: RefundRecord
): string[] {
  const errors: string[] = [];

  if (draft.amountCents !== record.amountCents) {
    errors.push("Refund amount does not match the system record");
  }

  if (draft.status !== record.status) {
    errors.push("Refund status does not match the system record");
  }

  return errors;
}

const llmDraft: RefundDraft = {
  amountCents: 15_000,
  status: "approved"
};

// Loaded from the source of truth, not from the LLM
const refundRecord: RefundRecord = {
  id: "refund_42",
  amountCents: 12_000,
  status: "pending"
};

function decideRefundResponse(
  draft: RefundDraft,
  record: RefundRecord
) {
  const errors = verifyRefundDraft(draft, record);

  if (errors.length > 0) {
    return { action: "block" as const, errors };
  }

  return { action: "send" as const };
}

const decision = decideRefundResponse(llmDraft, refundRecord);
// { action: "block", errors: [amount mismatch, status mismatch] }

In this example, the response is blocked for two concrete reasons: the amount is wrong, and the refund is still pending. A high Judge score cannot override either rule.

For an even stronger boundary, build the final refund sentence from the verified record instead of letting the model insert the amount or approval status into free text.

Where I’d use a Judge

A Judge can help assess clarity, relevance, completeness, and tone. But it adds cost and latency, and can favor longer answers even when they aren’t better.

I’d use the Judge to improve the response, not to authorize the refund. For that, I’d check the actual system records and enforce the business rules.

High-stakes or unclear cases still need human review.

Keep reading: Explore the cost and latency tradeoffs in cost-aware model routing, or see how agents act on new evidence in ReAct in Agentic AI.