LLM Evaluation

%alireza rashidi data science%
LLM thinking
What is the difference between truth, hypothesis, law and theory?
Intelligent Value Extraction
0
%alireza rashidi data science%

Grungy wooden reality check sign on a sign post against cloudy sky

The LLM Evaluation Framework — Measuring the Unmeasurable
The LLM Evaluation Framework
AI Engineering Series

Measure the unmeasurable.

Building a prototype is easy; taking it to production is hard. The difficulty is evaluation. Unlike traditional software, where tests pass or fail, generative outputs are probabilistic and subjective — so reliability needs a multi-layered strategy, from raw mathematical confidence to human-aligned operations.

↓ See the five layers↓ Skip to the tools
Ali Reza Rashidi7 min readUpdated 6 August 2026

A working demo is a prompt away. A trustworthy system is five layers of measurement deep. What you don’t measure, the model decides for you.

In this piece
  1. Overview — why evaluation is the job
  2. The layers — four kinds of measurement
  3. The stack — DeepEval, Ragas, LangSmith, Phoenix
  4. Choosing — how to assemble it
  5. Sources — the research behind it
01—Overview

Why evaluation is the job#

Traditional software fails loudly. LLMs fail fluently — with perfect grammar and invented facts. That is why evaluation is not a phase at the end; it is the architecture.

ConfidenceHow sure was the model? Token-level signals, zero extra cost.
RAG TriadRelevance, faithfulness, answer fit — scored separately.
SafetyRed-team yourself before the internet does it for you.
OpsLatency and cost decide whether quality ever ships.
02—The layers

The four measurement layers#

Move from the cheapest signal to the most expensive. Each layer catches what the previous one cannot.

Logprobs are the model’s internal scores, generated with each token at zero extra cost. A sudden confidence drop almost always signals trouble — you can even spot uncertainty on a specific word, like a date.

Uncertainty visualization
0 25 50 75 100 Factual query 92% Creative task 64% Hallucination 23% Mean token-level confidence by output type — illustrative.

How to read this: confident output is not necessarily correct output — but unconfident output is almost always worth a second look.

The pros

  • Zero latency cost: logprobs ride along with the token; no extra API calls.
  • Early warning: a confidence dip is the cheapest hallucination alarm there is.
  • Granular: word-level uncertainty, not just answer-level.

The cons

  • Sycophancy: RLHF-tuned models are often confidently wrong to please you.
  • Closed models: several APIs still don’t expose logprobs at all.
  • Calibration drift: confidence behavior changes between model versions.

Every RAG answer decomposes into three independent failures. Score them separately and you know which part of the pipeline to fix.[1]

1Contextual relevanceIs the retrieved data actually useful for the query? · Essential
2Faithfulness / groundednessIs the answer derived only from the context? · Critical
3Answer relevanceDoes the final output address the user’s prompt? · Essential
⚖Model-as-a-judgeA frontier model grades a smaller one — cheap, scalable, and good enough to trend.[2]
Scenario — the medical bot

User: “Does this drug cause headaches?” Context retrieved: “Patient X reported nausea.” (No mention of headaches.) Bot answer: “No, this drug does not cause headaches.”

Evaluation: Faithfulness — FAIL (the context didn’t say “no”; the bot made it up). Context relevance — FAIL (nausea data is unrelated to headaches). Two different bugs, two different fixes — one vague “bad answer” would have hidden both.

Modern evaluation prioritizes safety: attacking your own system with adversarial prompts to see if it leaks data, ignores its instructions, or produces toxicity.

Jailbreak resistance
0% 25% 50% 75% 100% Base model 85% With guardrails 12% Share of adversarial prompts that succeed — lower is better. Illustrative.
Jailbreak failure rate

How to read this: guardrails do not make a model safe — they make it measurably safer. The remaining 12% is why you also log and review.

Automated safety

  • Scalable: 10,000 adversarial prompts in minutes.
  • Standardized: established benchmarks like RealToxicityPrompts.[3]

Human review

  • Nuance: humans catch cultural context scripts miss.
  • Creative attacks: humans invent the jailbreaks automation never imagined.

Latency, cost, and throughput are evaluation metrics too — they decide whether the beautiful answer ever reaches a user, and whether the unit economics survive contact with traffic.

Price vs. performance
0 25 50 75 100 0 25 50 75 100 Speed (relative) Quality (relative) Llama 3 8B $ · fast GPT-4o mini $$ · fast Claude Sonnet $$$ · quality GPT-4o-class $$$ · quality Mistral Large $$ · balanced Frontier-class models, illustrative tiers — re-benchmark on your own workload before buying.

How to read this: the frontier models sit top-left; small open models sit bottom-right. Most products should start cheap and fast, then climb only where quality pays.

Evaluate the experience, not just the answer.P95 latency and cost per 1,000 requests are product metrics.
03—The stack

The implementation stack#

Don’t build evaluation from scratch. The ecosystem has matured — specialized frameworks now cover unit testing, RAG scoring, observability, and production monitoring.

DeepEval integrates directly into your CI/CD pipeline (like GitHub Actions) and fails a build if model accuracy regresses. If you know Pytest, you already know the workflow.

test_summary.py
assert_test(summary, max_length=50, tone="formal")
# Fails if summary > 50 words or slang is used.

Pros

  • Developer friendly: works just like Pytest.
  • CI/CD native: blocks bad models before deployment.

Cons

  • Code heavy: requires writing Python test cases.
  • Synthetic data: often relies on AI generating its own test data.

Ragas is mathematically specialized for Retrieval-Augmented Generation: faithfulness, context relevance, and answer relevance as first-class metrics.

ragas report
Score: 0.45 (Faithfulness)
# Flag: the bot added information not present in the PDF source.

Pros

  • Standardized: the industry standard for RAG metrics.
  • Model agnostic: works with LangChain, LlamaIndex, or raw API calls.

Cons

  • Slow: LLM-as-a-judge adds latency to test runs.
  • Costly: evaluation itself burns frontier-model calls.

LangSmith visualizes the entire execution trace, letting you replay specific user sessions and inspect each step — retrieval, prompts, tool calls, outputs.

trace
Trace ID: #8821a → Step 3 (Retriever) → Failed
# Shows exactly which document chunk caused the confusion.

Pros

  • Visual tracing: best-in-class UI for debugging complex chains.
  • Playground: one click to edit a prompt and re-run a failed trace.

Cons

  • Vendor lock-in: heavily optimized for the LangChain ecosystem.
  • Data privacy: logs go to the cloud (enterprise setup for on-prem).

Phoenix monitors production traffic to detect drift: when users start asking questions the model wasn’t trained for, or sentiment shifts under your feet.

alert
Alert: Negative sentiment spiked 20%
# Cluster of angry users discussing the new pricing model.

Pros

  • Cluster visualization: see groups of similar user queries in embedding space.
  • Open source: a robust local version you can run in notebooks.

Cons

  • Complexity: a steeper curve for non-data scientists.
  • Embedding focused: less intuitive for simple text analysis.
04—Choosing

How I would assemble it#

Start with the free signals, add structure where the failures are, and never let a model grade its own homework without a human spot-checking the grader.

1
Wire up logprob confidence on day one.

It costs nothing and catches the first hallucinations.

2
If you do RAG, score the triad with Ragas.

Separate retrieval bugs from generation bugs or you’ll fix the wrong one.

3
Trace everything with an observability tool.

When — not if — the app fails, you want the replay, not the vibes.

4
Monitor drift in production.

Users change faster than models do.

A realistic stack: DeepEval in CI to catch regressions, Ragas for retrieval quality, LangSmith or Phoenix for traces and drift — plus a weekly human review of the cases the judges flag. Evaluation is a loop, not a gate.

05—Grounding

Sources#

The charts are illustrative; the measurement layers are not. These are the papers that define them.

  1. The RAG triad, formalized. Es, James, Espinosa-Anke & Schockaert, “RAGAS: Automated Evaluation of Retrieval Augmented Generation” (EACL 2024, System Demonstrations) — reference-free scoring of faithfulness, context relevance, and answer relevance. arxiv.org/abs/2309.15217
  2. Model-as-a-judge, with its caveats. Zheng et al., “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena” (NeurIPS 2023) — strong models match human preferences at over 80% agreement, but position and verbosity biases are real, which is why the grader needs spot-checking too. arxiv.org/abs/2306.05685
  3. The safety benchmark mentioned above. Gehman et al., “RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models” (Findings of EMNLP 2020) — 100k naturally occurring prompts for measuring toxic completions. arxiv.org/abs/2009.11462
The LLM Evaluation Framework
For teams who would rather measure their model than admire it.
Part of the AI Engineering Series · Updated 6 August 2026. Chart values are illustrative tiers, not vendor benchmarks.
Ali Reza Rashidi
Ali Reza Rashidi
Ali Reza Rashidi, a Senior Data Scientist-Gen Al | Al Architect | MLOps with over ten years of experience, He is the author of three books that delve into the world of data and management.

Related posts

%alireza rashidi data science%

LangGraph vs LangChain and more

Read more
What is the difference between truth, hypothesis, law and theory?

What is the difference between truth, hypothesis, law and theory?

Intelligent Value Extraction

Read more
%alireza rashidi data science%

LLM thinking

Read more

Leave a Reply

Your email address will not be published. Required fields are marked *

error: Content is protected!