Question 1

What is evaluation-driven development for AI applications?

Accepted Answer

Evaluation-driven development (EDD) is building evals first and letting them drive iteration — you define success criteria and a test set before/alongside building, then measure every prompt, model, or pipeline change against them. It's TDD for AI: replace "does this feel better?" with "did the eval score go up?"

Define success metrics + eval set → build a version → measure against the eval → improve based on results → repeat. Every change is gated by numbers, not vibes.

WHY IT MATTERS
"LLM output is non-deterministic and subjective, so without evals you're developing on vibes — a prompt tweak that 'looks better' may have silently regressed 20% of cases. EDD makes progress measurable and prevents regressions. The discipline is: no prompt change ships without an eval score. 'A prompt without an eval is a guess.'"

Question 2

What are the key differences between evaluating traditional ML vs LLM applications?

Accepted Answer

Traditional ML has clear ground truth and objective single-number metrics (accuracy, F1, AUC, RMSE) on a held-out set. LLM evaluation is fundamentally harder: outputs are open-ended (often no single right answer), quality is subjective and multi-dimensional, generation is non-deterministic, and you need LLM-judge or human evaluation rather than a formula.

Traditional ML vs LLM applications
- Ground truth: Traditional = Clear labels | LLM = Often none / many valid answers
- Metrics: Traditional = Objective (accuracy/F1/AUC) | LLM = Subjective, multi-dimensional
- Determinism: Traditional = Deterministic | LLM = Non-deterministic (sampling)
- Method: Traditional = Compute a metric | LLM = LLM-judge / human / task success
- Scope: Traditional = One narrow task | LLM = Broad capabilities; contamination risk

FRAMING
"The core shift: traditional ML asks 'is the prediction correct?' with a clear answer key; LLM eval asks 'is this open-ended output good?' across many dimensions with no single key. That's why LLM eval leans on LLM-judges, human eval, rubrics, and multi-trial reliability instead of a one-line metric."

Question 3

How do you evaluate LLM outputs, and what metrics do you use?

Accepted Answer

Match the method to the task and to whether you have a reference answer. Broadly three families: reference-based metrics (BLEU/ROUGE/BERTScore, exact match — when you have gold outputs), LLM-as-judge / rubric scoring (for open-ended generation), and task-specific metrics (accuracy/F1 for classification, pass@k for code, RAGAS for RAG), plus human eval for the highest bar.

Dimensions to score
Correctness/accuracy, relevance, coherence/fluency, faithfulness/groundedness (RAG), safety, format adherence, helpfulness. Rarely one number — you track several.

SELECTION RULE
"There's no universal LLM metric — pick by task. Closed tasks (classification, extraction) → objective metrics. Reference generation (translation/summarization) → BLEU/ROUGE/BERTScore. Open-ended (chat, writing) → LLM-judge + human. RAG → RAGAS. The mistake is forcing one metric everywhere; you evaluate multiple dimensions with the right tool for each."

Reference in assignment: outcomeschool.com/blog/llm-evaluation.

Question 4

What is the difference between offline and online evaluation for AI systems?

Accepted Answer

Offline evaluation runs on a fixed dataset before deployment — for development, CI regression gates, and comparing versions in a controlled setting. Online evaluation measures the system on real production traffic — A/B tests, user feedback, and live quality/cost metrics. You need both: offline to ship safely, online to know it actually works for users.

Offline vs Online
- Data: Offline = Fixed eval set | Online = Live production traffic
- When: Offline = Pre-deploy, CI | Online = In production
- Signals: Offline = Metrics vs gold | Online = A/B, feedback, engagement
- Strength: Offline = Controlled, repeatable | Online = Real-world validity

KEY POINT
"Offline tells you which version should win; online tells you which actually improves user outcomes — and they don't always agree (offline eval sets can't capture the full input distribution). Ship gated by offline regression, then confirm with online A/B + monitoring. Relying on only one is a common failure."

Question 5

What is the role of golden datasets in AI evaluation?

Accepted Answer

A golden dataset is a curated, human-verified set of (input, expected-output or acceptance-criteria) that serves as your trusted ground-truth benchmark. It's what you measure every version against — enabling regression testing, version comparison, progress tracking, and CI gates.

Building a good one
- Representative of real production inputs + edge cases and known failure modes.
- High quality & human-verified; quality > size.
- Held out — not used during prompt development (or you overfit to it).
- Versioned and grown over time (mine production failures into it).

WHY FOUNDATIONAL
"The golden set is the anchor of your whole eval system — its quality caps the trust you can place in every number. Keep it clean, representative, and separate from development so it stays an honest test. A regression suite is only as good as its golden data." (Cross-ref LLMOps CI/CD.)

Question 6

How do you set up an evaluation framework from scratch for a new LLM application?

Accepted Answer

Work top-down from "what does good look like?" to an automated, CI-integrated harness. The sequence: define success criteria → build a golden dataset → choose metrics per dimension → build the eval harness → set baselines → gate CI → add online eval → iterate.

1. Define task + success criteria (per dimension) → 2. Golden dataset (real + edge cases, labeled) → 3. Pick methods (auto metrics / LLM-judge / human) → 4. Eval harness (run → score → aggregate) → 5. Baselines (current system, prompting) → 6. CI regression gate → 7. Online eval (prod sampling + feedback) → 8. Iterate: mine failures back in.

WHERE PEOPLE GO WRONG
"Start small — 20–50 well-chosen cases beat a huge messy set — and expand from real failures. The two mistakes are (1) building elaborate metrics before you have representative data, and (2) never closing the loop from production failures back into the eval set. Evaluation is a living system, not a one-time script."

Reference in assignment: outcomeschool.com/blog/llm-evaluation.

Question 7

Explain BLEU, ROUGE, and BERTScore. When would you use each?

Accepted Answer

All three compare a candidate to a reference, but differently. BLEU is precision-based n-gram overlap (with a brevity penalty) — built for machine translation. ROUGE is recall-based n-gram overlap — built for summarization. BERTScore uses contextual embeddings to measure semantic similarity, catching paraphrases that exact n-gram overlap misses.

Metric → Measures → Best for
- BLEU → N-gram precision + brevity penalty → Translation
- ROUGE → N-gram recall (ROUGE-N, ROUGE-L) → Summarization
- BERTScore → Semantic similarity via embeddings → When paraphrase/meaning matters

THE LIMITATION TO NAME
"BLEU/ROUGE reward surface word overlap, so they penalize valid paraphrases and can't judge factuality or coherence — weak for open-ended/creative generation. BERTScore fixes the paraphrase problem via embeddings but still needs a reference. For modern open-ended tasks, LLM-as-judge usually correlates better with humans than any of these."

Reference in assignment: outcomeschool.com/blog/llm-evaluation.

Question 8

What is LLM-as-a-judge evaluation, and what are its limitations?

Accepted Answer

LLM-as-a-judge uses a strong LLM to score or compare outputs against a rubric — either pointwise (rate this output 1–5) or pairwise (is A or B better?). It's scalable, cheap, flexible, and correlates reasonably with human judgment, which is why it's the workhorse of modern LLM eval. But it has documented systematic biases you must control for.

Key limitations / biases
- Position bias: favors a specific position (often the first) in pairwise — flipping order can flip the verdict.
- Verbosity bias: prefers longer responses regardless of quality.
- Self-preference (self-enhancement) bias: favors outputs from its own model/family (linked to self-recognition; can swing dramatically).
- Others: reference-answer bias, sycophancy, formatting sensitivity, limited domain understanding.

Mitigations
Swap positions and average both orders; use rubric-based scoring constrained to specific criteria (not open-ended "which is better"); calibrate against human labels; ensemble multiple judge models and report inter-judge agreement; use a different/stronger judge than the model under test.

HONEST CAVEAT
"LLM-judges are powerful but biased — even large reasoning models stay susceptible. Ensembling

Question 9

What is G-Eval, and how does it use LLMs for evaluation?

Accepted Answer

G-Eval (Liu et al., 2023) is an LLM-as-judge framework that improves reliability with two ideas: chain-of-thought evaluation steps and a probability-weighted (form-filling) scoring scheme. You give it the task and criteria; it auto-generates detailed evaluation steps via CoT, then scores by "filling a form," and G-Eval weights each possible score by its token probability for a finer-grained, more continuous result.

1. Give criteria (e.g. coherence, relevance) → 2. LLM generates step-by-step eval instructions (CoT) → 3. LLM scores the output following those steps → 4. weight scores by output-token probabilities → continuous score.

WHY IT'S BETTER THAN RAW SCORING
"G-Eval's CoT steps make the judgment more structured and human-aligned, and probability-weighting avoids the coarse, clustered integer scores you get from naive prompting (everything becoming a '4'). It correlates better with humans on tasks like summarization — but it's still an LLM-judge, so the position/verbosity/self-preference biases still apply."

Reference in assignment: outcomeschool.com/blog/llm-as-a-judge.

Question 10

How do you conduct human evaluation for AI systems?

Accepted Answer

Human eval is the gold standard for subjective quality. Design it for reliability: prefer pairwise comparison (A vs B — easier and more consistent than absolute scoring), use clear rubrics, train and calibrate annotators, use multiple annotators per item, and measure inter-annotator agreement (Cohen's/Fleiss' kappa).

Best practices
- Pairwise or ranking over Likert where possible (humans compare better than they score absolutely).
- Blind & randomized (hide which system, shuffle order) to cut bias.
- Clear guidelines + calibration rounds; track kappa as a quality metric.
- Aggregate multiple labels; adjudicate disagreements.

TRADE-OFF & USE
"Human eval is accurate but slow and expensive, so use it strategically: to validate high-stakes changes, to calibrate your cheaper LLM-judge, and for final sign-off — not on every commit. Low inter-annotator agreement usually means an ambiguous task or unclear rubric, not bad annotators." (Cross-ref Fine-Tuning annotator-agreement.)

Question 11

What are benchmark suites (MMLU, HumanEval, GSM8K), and how do you interpret them?

Accepted Answer

They're standardized public tests of specific model capabilities. MMLU — 57 subjects of multiple-choice questions measuring broad knowledge/reasoning. HumanEval — 164 Python problems scored by pass@k (does the generated code pass unit tests?). GSM8K — grade-school math word problems testing multi-step arithmetic reasoning.

Benchmark → Tests → Metric
- MMLU → Broad knowledge (57 subjects) → Multiple-choice accuracy
- HumanEval → Code generation → pass@k (unit tests)
- GSM8K → Math word-problem reasoning → Answer accuracy

INTERPRET WITH CAUTION
"Benchmarks measure narrow slices and suffer contamination (test data leaking into training inflates scores), saturation (top models near the ceiling, so differences aren't meaningful), and a big gap from your task. Use them as a coarse capability filter, not proof of fit — a model topping MMLU can still fail your specific use case. Always evaluate on your own data."

Reference in assignment: outcomeschool.com/blog/llm-evaluation.

Question 12

How do you evaluate a RAG system end-to-end?

Accepted Answer

Evaluate retrieval and generation separately, then end-to-end — because a wrong answer is either a retrieval miss or a generation failure, and you fix different things. Retrieval: recall@k, precision@k, MRR, NDCG, context relevance. Generation: faithfulness/groundedness (is the answer supported by the retrieved context?), answer relevance, and correctness vs gold.

Frameworks & the RAG triad
RAGAS (faithfulness, answer relevance, context precision/recall), TruLens's RAG triad (context relevance → groundedness → answer relevance), ARES. Plus overall answer quality via LLM-judge/human, and latency/cost.

THE KEY DISCIPLINE
"Never score RAG with one end-to-end number — decompose it. If retrieval recall is low, fix chunking/hybrid/reranking; if faithfulness is low, fix the generation prompt/grounding. Separating retrieval vs generation failure is what makes RAG debugging tractable." (Cross-ref RAG assignment — Prakhar ships this in CareerLift/ThinkStudio.)

Question 13

How do you evaluate the quality of AI agents?

Accepted Answer

Agents are multi-step, so a single score won't do — evaluate several dimensions: outcome (did it achieve the goal?), trajectory (was the path good — tool sequence, reasoning), tool-call accuracy (right tool, right args, right order), efficiency (steps, loops, cost, latency), and safety.

Dimension → Measures → How
- Outcome → Goal achieved → Gold state / task success / tests
- Trajectory → Path & reasoning quality → Gold-path or LLM-judge
- Tool-calls → Right tool & args → Span/BFCL-style scoring
- Efficiency → Steps, cost, latency → Instrumentation

Benchmarks: τ-bench, SWE-bench, BFCL, AgentBench. Use trace/span-level evaluation to localize failures and multi-trial (pass^k) for reliability.

INSIGHT THAT IMPRESSES
"A correct outcome via a wrong trajectory is a false positive — the agent got lucky and will fail under different conditions. So score the path, not just the result, and report multi-trial reliability, not a single lucky run. LLM-judges alone can't assess whether the tool sequence or domain policy was right — combine with objective checks." (Cross-ref Agents assignment.)

Reference in assignment: outcomeschool.com/blog/ai-agent-evaluation.

Question 14

What is AI Agent Observability?

Accepted Answer

Agent observability is the ability to see what an agent did — capturing a full span-level trace of each run (every reasoning step, tool call, input/output, tokens, latency, cost) plus quality signals — so you can debug failures, monitor quality/cost, and improve. It's the data layer that evaluation scores and that incident debugging depends on.

What's agent-specific
- Trajectory visibility: the full multi-step chain, not just the final output.
- Tool-call tracing: which tools were called, with what args, and their success/failure.
- Loop/anomaly detection; per-step latency/cost attribution.

WHY IT'S ESSENTIAL
"In a multi-step agent, a bad final answer could come from any step — you can only localize it with span-level traces (e.g. 'the search tool returned junk in step 3'). Observability is a prerequisite for both debugging and evaluation: you can't score or fix a trajectory you can't see. Tools: LangSmith, Langfuse, Arize Phoenix." (Cross-ref LLMOps observability.)

Reference in assignment: outcomeschool.com/blog/ai-agent-observability.

Question 15

How do you evaluate multi-turn conversation quality?

Accepted Answer

Evaluate the whole dialogue, not just isolated responses. Key dimensions: coherence across turns, context retention (does it remember earlier turns?), goal completion (did the conversation achieve the user's objective?), consistency (no contradictions), and appropriate turn-taking.

Methods
- LLM-judge over the full conversation against a rubric.
- Task-success: did the multi-turn task complete?
- Simulated users: an LLM plays the user to generate many multi-turn conversations at scale.
- Human eval for nuanced quality.

THE CORE CHALLENGE
"Errors compound across turns — a great turn 1 with a broken turn 5 is a failed conversation. Single-turn metrics miss this, so evaluate the trajectory and the final goal completion. Simulated-user testing is the scalable way to generate realistic multi-turn cases without armies of humans." (Cross-ref agent trajectory eval.)

Question 16

How do you implement continuous evaluation for production AI systems?

Accepted Answer

Continuous evaluation runs on live traffic, not just pre-deploy: sample production requests → auto-eval (LLM-judge + heuristics) and collect user feedback → track quality/cost/drift over time → alert on regressions → mine failures back into the eval set. It catches what offline eval can't: drift, silent regressions, and emerging failure modes.

Prod traffic → sample → auto-eval + user feedback (thumbs, implicit signals) → dashboards (quality/cost/drift trends) → alerts on regression → failures → back into golden set → offline regression suite. Loop closes.

WHY IT'S NON-NEGOTIABLE
"Quality degrades silently in production — data drift, a provider's model update, or a new user behavior can tank quality with no code change. Offline evals can't see this; only continuous online evaluation + drift monitoring catches it. The flywheel — prod failures becoming eval cases — is what makes the system improve over time." (Cross-ref LLMOps monitoring.)

Question 17

How do you detect and measure hallucinations in LLM outputs?

Accepted Answer

Combine several signals, since no single method is reliable: groundedness/faithfulness checking (verify each claim is supported by the source/context, via NLI entailment or an LLM-judge), fact-checking against a knowledge source, self-consistency (sample multiple outputs — disagreement flags fabrication, à la SelfCheckGPT), and uncertainty signals (low token probability / high entropy correlates with made-up content).

Metrics
Hallucination rate, faithfulness score, factual precision — measured on a labeled set or via automated groundedness checks.

PRACTICAL STANCE
"For RAG, hallucination is best measured as faithfulness — is every claim supported by the retrieved context? For open-domain, use self-consistency and fact-checking against a source. Detection is probabilistic, so combine methods and set a threshold; there's no perfect hallucination detector, which is why grounding (RAG) + citations is the real defense." (Cross-ref RAG/vectors.)

Question 18

How do you measure factual consistency in LLM outputs?

Accepted Answer

Factual consistency asks: does the output stay faithful to a given source (no contradictions or unsupported claims)? The two main automated approaches are NLI-based (does the source entail each output claim? entailment = consistent, contradiction/neutral = inconsistent) and QA-based (generate questions from the output, answer them from the source, and compare — QAGS/QuestEval).

Methods & tools
- NLI entailment per claim (SummaC, FactCC).
- QA-generation consistency (QAGS, QuestEval).
- LLM-judge entailment ("is this supported by the source?").
- Report a consistency score; used heavily for summarization + RAG.

DISTINCTION TO DRAW
"Factual consistency is about faithfulness to a given source (intrinsic), which is different from world-factuality (extrinsic). NLI and QA-based methods measure it automatically at claim level. This is exactly the 'faithfulness' pillar of RAG evaluation — a summary can be fluent and relevant yet contradict its source, and only a consistency check catches that."

Question 19

How do you evaluate bias in AI model outputs?

Accepted Answer

Measure disparate treatment or outcomes across demographic groups using multiple lenses: counterfactual testing (swap a sensitive attribute — name, gender, race — in otherwise-identical inputs and measure output differences), bias benchmarks (BBQ, StereoSet, BOLD, WinoBias), disaggregated evaluation (performance per group, check gaps), and fairness metrics on decisions (demographic parity, equal opportunity, equalized odds).

Approaches
- Counterfactual/perturbation: identical input, different demographic → should get similar output.
- Disaggregated metrics: break accuracy/quality down by group; measure the gap.
- Stereotype benchmarks; sentiment/representation analysis; red-teaming for biased outputs.

KEY NUANCE
"Report metrics per group and the gaps between them, not just an aggregate that hides disparity. And note the hard part: fairness definitions mathematically conflict — you generally can't satisfy demographic parity, equal opportunity, and calibration at once (impossibility results), so which metric you target is a context/policy choice." (Sets up Q25.)

Question 20

How do you evaluate the robustness of an LLM application across input variations?

Accepted Answer

Test that quality stays stable under realistic input perturbations, not just on clean examples. Apply perturbations (paraphrases, typos, casing, formatting, added noise) and measure output consistency and quality drop; use invariance tests (meaning-preserving changes shouldn't change the output) and directional tests (some changes should change it appropriately) — the CheckList methodology — plus distribution-shift / OOD and prompt-sensitivity checks.

What to measure
Performance variance across variations, worst-case degradation, and consistency (does a trivial rewording flip the answer?).

WHY IT MATTERS
"Real users don't type clean inputs — they misspell, rephrase, and format oddly. A model that scores 90% on your clean set but swings wildly on paraphrases isn't production-ready. Robustness testing measures the variance, not just the mean; brittleness to trivial rewording is a common, silent failure."

Evaluation & Testing

Evaluation & Testing