AI Engineering Series

No evals, no deploy.

Every serious LLM team learns the same lesson: you cannot improve what you do not measure, and you cannot measure what you have not defined. Evaluation is not a phase at the end — it is the operating system of the whole project.

↓ The five stages ↓ Skip to anti-patterns

This is the procedure, step by step: from the first embarrassing output to the production feedback loop. The companion piece on what to measure is already on this site.

01—The pipeline

Five stages, one loop#

Evaluation isn’t one activity — it’s five, layered from your laptop to production, with the last stage feeding the second forever.

01Dev checksMillisecond assertions while you iterate — format, length, regex, schema validity.

02Golden dataset30–50 curated cases with expected behavior. Versioned like code, fed by production.

03CI regressionEvery pull request runs the suite. Quality is a build artifact, not a vibe.

04Deploy gateShip when per-category failure rates are acceptable — not when one score looks pretty.

05Production monitorSample 5–10% of live traffic; every real failure becomes tomorrow’s regression case.

90% target agreement between your LLM judge and human reviewers before trusting it at scale

30–50 curated cases are enough for a first golden dataset — case quality beats quantity

5–10% of production traffic to sample for ongoing review — small, steady, human-checked

5 stages in the loop — and the fifth feeds the second, forever

The compound interest of testing

With eval gateWithout evals

How to read this: prompts change, models change, data drifts. Without a gate, each change quietly taxes quality; with one, every change is a measured experiment.

02—The procedure

Step by step#

Six working steps, in the order that saves the most pain. Each one has a deliverable you can point to.

Open your logs and collect 10–20 real outputs you would be embarrassed to ship. Cluster them: wrong facts, wrong format, wrong tone, unsafe content, broken tool calls. Each cluster becomes a scored category — this taxonomy is the foundation everything else stands on.

The usual failure taxonomy

Factual errors: invented figures, wrong citations, hallucinated APIs.
Instruction violations: ignored constraints, wrong language, missing steps.
Format breaks: invalid JSON, schema drift, truncated output.
Tone misses: off-brand, condescending, or needlessly verbose.
Safety issues: leaks, refusals that should happen (or shouldn’t).

Most teams skip this step.They adopt an off-the-shelf benchmark that measures somebody else’s problem — and then wonder why the score goes up while users keep complaining.

The golden dataset is the test suite for behavior that can’t be asserted with assert. Each case pairs an input with expected behavior — not necessarily one right answer, but a checkable contract: must include, must not, format rules.

golden.jsonl

# one case per line, reviewed like code
{"input": "Summarize ticket #4821",
 "expect": {"must_include": ["refund amount"],
            "must_not": ["invented policy"],
            "format": "3 bullets max"}}

Rules that keep it honest

Version it in git: a score change means nothing if the test changed too.
Feed it from production: every live failure becomes a case.

Traps to avoid

Happy-path only: add ambiguous, hostile, and adversarial inputs.
Set-and-forget: a stale dataset tests a product you no longer ship.

Quality of cases beats quantity.Fifty cases that mirror real traffic outperform five thousand scraped from a tutorial.

Scorers come in three tiers, and the order matters. Deterministic checks (schema validity, regex, length, tool-call arguments) catch everything they can catch — for free, in milliseconds. LLM-as-judge grades the subjective remainder: faithfulness, helpfulness, tone. Humans calibrate both and own the edge cases.

Deterministic checks

JSON schema validity, required fields present.
Regex and format rules (dates, IDs, citations).
Tool-call arguments match expectations.
Length, language, forbidden phrases.

LLM-as-judge

Faithfulness to the retrieved sources.
Instruction following and completeness.
Tone, helpfulness, brand voice.
Pairwise A/B: which of two outputs is better?

If a regex can catch it, don’t pay a model to.Every judgment delegated to an LLM that code could have made is money spent adding noise where there could have been certainty.

Before trusting an LLM judge at scale, hand-label at least 50 examples and measure agreement — around 90% alignment with your human reviewers is the working threshold. Write rubrics with examples, not adjectives: “good” grades nothing; “correct refund amount, no invented policy, ≤3 bullets” grades everything.

Judge hygiene

Version the judge prompt alongside the rubric — a judge change is a metric change.
Prefer pairwise comparisons over absolute scores; they are more reliable per dollar.
Re-grade the judge every time its prompt or model changes.
Keep humans on the tail: the weird 5% is where judges drift first.

An uncalibrated judge doesn’t measure quality.It measures its own quirks — and optimizes your product toward them.

Run the suite on every pull request, and gate per category, not on one blended score. A single composite number hides exactly the failures you care about: a faithfulness regression can surf into production on a tone improvement.

eval-gate.yml

# ship policy, per scored category
safety:       block on any failure
faithfulness: allow 0% regression
format:        deterministic — must pass 100%
tone:          allow 2% regression
# report deltas, not just scores

The gate is a product decision, not a metric.“What failure rate do we tolerate, per category, before a human gets paged?” — answer in writing before the first incident answers for you.

Sample 5–10% of live traffic for ongoing review — small, steady, human-checked. Read traces, not just metrics: the reasoning path tells you why a score moved. And close the loop: every production failure becomes a golden-dataset case, so the suite gets sharper every time reality bites.

The production loop

Sample: 5–10% of traffic, stratified across user segments.
Review traces: prompts, retrievals, tool calls — not just final scores.
Recalibrate on schedule: models drift, data drifts, judges drift.
Feed failures back: the dataset is a living asset; the day it stops growing, your evals start lying.

Eval is a loop, not a launch checklist.Define → test → gate → catch → add to the test. Teams that run the loop ship confidence; teams that skip it ship hope.

03—The metrics

Measure each layer separately#

An LLM application is a stack, and each layer fails in its own way. A single blended score will never tell you which pipe broke.

Layer	Question it answers	Typical metrics
Retrieval (RAG)	Did we fetch the right context?	Context recall & precision, hit rate
Generation	Is the answer faithful and useful?	Faithfulness, answer relevance, citation accuracy
Agent behavior	Did it act correctly?	Tool-call correctness, task completion, step economy
Safety	Can it cause harm?	Refusal accuracy, jailbreak resistance, PII leakage
System	Is it fast and affordable?	Latency p95, cost per request, cache hit rate

A RAG system can ace generation and fail retrieval — or the reverse.Score the layers separately, or you’ll fix the wrong pipe with total confidence.

04—The anti-patterns

How eval programs die#

Most eval programs don’t fail loudly. They quietly become theater — scores that move, dashboards that glow, and quality that drifts.

Offline-only confidence.

Passing your suite means nothing if production inputs look different. Sample live traffic, or you are testing a fiction.

One metric to rule them all.

A composite score is a great way to ship a faithfulness regression hidden by a tone improvement. Per-category gates, or per-category blindness.

Unversioned criteria.

Changed the rubric last month? Then last month’s scores are archaeology. Version prompts, rubrics, datasets, and judges together.

Happy-path datasets.

If your golden set has no adversarial, ambiguous, or hostile inputs, you tested the demo — not the product.

The loop in one line: define failure → test for it → gate on it → catch it in production → add it to the test.

Step by Step LLM Evaluation

Truth, Hypothesis, Law, and Scientific Theory

What is Data Cleaning?

No evals, no deploy.

Five stages, one loop#