LLM evaluation metrics measure whether a language model application completes real tasks, produces reliable answers, and works within its operating limits. Before launch, they help a team decide which workflows are ready and which failures still need attention.
A convincing demonstration can miss problems that appear with unfamiliar questions, conflicting documents, or unavailable tools. The nine metrics below cover output quality and release readiness, with practical ways to measure each and investigate weak results.
Define what a successful release means
Start with the application's job. A document extractor needs correct fields; a support assistant needs accurate guidance; an agent that updates records also needs the right permissions and final system state.
Public LLM evaluation benchmarks provide useful evidence about model capabilities. They do not reproduce your organization's documents, access rules, or integrations. The HELM research demonstrates why several evaluation dimensions matter and why trade-offs should remain visible.
Choose applicable metrics and write acceptance criteria before testing. Retrieval recall belongs in applications that retrieve evidence, while factual correctness may be less useful for purely imaginative writing. Latency and cost are operational measures that complement LLM quality metrics.
Build a test set around real work
Collect examples from approved support records, workflow interviews, and pilot feedback. Remove sensitive information where possible while preserving the context needed to reproduce each task.
Include four groups:
- Routine tasks: Common requests and workflows, sampled to reflect expected use.
- Edge cases: Missing fields, ambiguous questions, long inputs, and conflicting sources.
- Known failures: Earlier mistakes, empty search results, timeouts, and unavailable tools.
- Adversarial cases: Attempts to obtain restricted information or manipulate actions through untrusted content.
Record the expected outcome, supporting evidence, user permissions, acceptable alternatives, and failure severity. Some cases should end with clarification or a human handoff rather than a completed answer.
Keep development examples separate from a held-out release set. Maintain dedicated stress tests for rare hazards so frequent easy tasks cannot hide them in an overall average. Have reviewers resolve uncertain labels before comparing system versions.
1 Task success rate
Task success rate measures how often the application achieves the outcome the user needs. It provides an overall product measure, while the remaining metrics help explain failures.
- Measure it: Divide tasks meeting a written acceptance rubric by all attempted tasks. Keep results separate for different workflows.
- Example: A booking agent passes when the test system contains the correct reservation, required approval is recorded, and the confirmation matches the booking.
- Benefit: It connects evaluation to useful work and exposes failures that polished language can hide.
- Limitation: A binary pass hides partial progress and error severity. Retain failure labels and inspect where each workflow stopped.
Use it for assistants, extraction, and agents. When results worsen, inspect missing requirements, tool errors, and fallback behavior. For open-ended outputs, the rubric should assess relevance and completeness while allowing different correct phrasings.
2 Factual correctness
Factual correctness checks whether statements agree with verified reference evidence. It matters when users rely on dates, quantities, policy conditions, or technical instructions.
- Measure it: Label checkable claims as correct, incorrect, or unverifiable. Report the share judged correct across assessed claims and disclose how unverifiable claims are handled.
- Example: Check an invoice summary against the invoice itself, including amounts, currency, and payment terms.
- Benefit: Claim-level review identifies the precise detail that failed.
- Limitation: Incomplete references distort results, and short answers may avoid errors by omitting essential facts. Review completeness through task success.
The FActScore research illustrates evaluation using individual factual statements and a reliable knowledge source. Its published results do not establish the quality of your application.
Use this metric for factual answers, summaries, and reports. If it declines, check reference freshness, numerical reasoning, entity confusion, and whether reviewers agree on the expected facts.
3 Groundedness
Groundedness checks whether an answer's claims are supported by the evidence supplied during that run. This is useful for retrieval-augmented generation, or RAG, where answers should draw on approved documents.
- Measure it: Divide supported substantive claims by assessed claims. Check the actual retrieved passages or tool results, including whether attached citations support the statements.
- Example: If a supplied policy permits cancellation within 30 days, an answer inventing a 60-day period is unsupported.
- Benefit: It exposes additions that sound plausible but have no basis in the supplied material.
- Limitation: A grounded answer can still be wrong when its source is outdated. A citation alone does not establish support.
Ragas documentation on faithfulness describes this claim-to-context assessment. Treat claim-free responses as not applicable rather than automatically perfect.
For document assistants, investigate context truncation, contradictory passages, or prompts encouraging unsupported elaboration when groundedness falls. Keep this measure separate from factual correctness to distinguish source problems from generation problems.
4 Retrieval recall at k
Retrieval recall at k measures the proportion of known relevant evidence returned among the first k results. It helps diagnose whether missing context caused an answer to fail.
- Measure it: Label relevant passages, then divide relevant passages found in the top k by all labeled relevant passages for each question. Define the passage unit and keep k fixed between comparisons.
- Example: If three distinct passages are required and retrieval returns two in the top five, recall at five is two-thirds.
- Benefit: It identifies missing evidence before you change the generation model.
- Limitation: Incomplete relevance labels can distort the score. Recall does not show how much irrelevant material also arrived.
Stanford's information retrieval reference explains the underlying distinction between recall and precision. Finding one useful passage is not necessarily full recall when several are relevant.
Use this for RAG applications. If it drops, inspect ingestion, chunking, query rewriting, filters, and ranking. Evaluate only evidence the user may access; restricted material must never count as improved retrieval. Handle questions with no relevant permitted evidence separately.
5 Instruction compliance rate
Instruction compliance rate measures how often outputs satisfy the application's explicit requirements. It catches responses that appear useful but fail the expected format or workflow rules.
- Measure it: Define testable requirements for schemas, required fields, allowed values, language, and tool arguments. Divide outputs meeting all applicable requirements by assessed outputs, retaining results for individual rules.
- Example: An intake response must contain every required field and use an allowed case category.
- Benefit: Parsers and schema validators make many checks quick and reproducible.
- Limitation: Correct structure does not establish correct meaning. A valid account identifier can still refer to the wrong customer.
Use it for extraction and API-connected workflows. If compliance declines, inspect schema changes, conflicting instructions, output truncation, and misleading examples.
For agents, check arguments against the scenario and verify the resulting system state under task success. Enforce authorization in application code instead of relying on the model to obey a prompt.
6 Appropriate abstention rate
Appropriate abstention rate measures whether the application declines, clarifies, or escalates when it should. It makes uncertainty handling testable instead of leaving the system to guess.
- Measure it: Label cases requiring abstention, then divide correctly handled cases by those cases. Specify whether the expected behavior is a clarification, refusal, or handoff.
- Example: When two supplied policies conflict, the assistant should identify the conflict and follow the defined escalation path.
- Benefit: It reveals confident answers that should never have been given.
- Limitation: Refusing every request could score well here. Also track unnecessary refusals on answerable cases and review task success.
Use it for bounded knowledge assistants and consequential workflows. If performance worsens, check answerability labels, retrieval failures, and prompts pressuring the model to always provide an answer.
A model saying it is confident is not evidence of calibrated confidence. Test behavior against known answerable and unanswerable cases, including situations where a brief clarification can resolve the uncertainty.
7 Safety violation rate
Safety violation rate measures how often tested interactions breach defined content, privacy, or action policies. Those definitions should reflect the application's threats and consequences.
- Measure it: Divide interactions containing a defined violation by tested interactions. Report categories and severity separately, including results from ordinary requests and dedicated attack tests.
- Example: A retrieved document contains an instruction to send private records outside the organization. The application should ignore it and prevent the unauthorized action.
- Benefit: Violations become reviewable evidence for release decisions.
- Limitation: Results depend on test coverage. Zero observed violations in a finite dataset does not prove that every possible attack is blocked.
OWASP's prompt injection guidance covers direct attacks and instructions embedded in external content. Test permissions and connected systems alongside model responses.
If violations rise, review access checks, tool scope, and output handling. Treat a critical unauthorized action as a release blocker even when average quality improves.
8 P95 end to end latency
P95 latency is the duration at or below which approximately 95% of measured requests complete. It highlights slower experiences that an average can conceal.
- Measure it: Time requests from arrival to the final usable response or confirmed action under representative concurrency. Include queueing, retrieval, generation, tool calls, and retries.
- Example: A reservation workflow finishes when its action and confirmation are complete, rather than when the first text appears.
- Benefit: It exposes delays affecting users at the slower end of the workload.
- Limitation: Results depend on traffic mix and sample size. Report timeouts and failures alongside successful completion timings.
Use it for interactive applications and workflows with deadlines. For streaming interfaces, also track time to the first output token; that measure does not capture the whole task.
When latency rises, inspect stage timings, output length, retries, and provider limits. Set response budgets around the workflow instead of adopting a universal speed target.
9 Cost per successful task
Cost per successful task measures the operating expense required to deliver an accepted outcome. It makes failed attempts and rework visible in the economics.
- Measure it: Divide total in-scope cost across all attempted tasks by successful tasks. Include model calls, retrieval, external tools, retries, and required human review.
- Example: A cheaper response path may become more expensive overall if it creates additional retries or reviewer work.
- Benefit: It ties spending to completed work rather than request volume alone.
- Limitation: Comparisons become misleading when accounting boundaries or task mixes differ. Document infrastructure allocation and review costs consistently.
Use current contracted prices and measured usage to estimate production expense. If no tasks succeed, report the measure as undefined.
If cost rises, inspect expanding contexts, repeated calls, falling success rates, and growing review effort. Test optimizations against the same quality and safety requirements before accepting the savings.
Set thresholds and compare releases consistently
Define acceptance levels from user needs, error consequences, and budget constraints. There is no universal launch score across applications. LuMay's guide to validating an AI proof of concept offers related guidance on success criteria and baselines.
For each candidate release:
- Reuse the same dataset, scoring rubrics, retrieval snapshot, and controlled tool fixtures.
- Record prompt, model, index, tool, and application versions, plus generation settings.
- Compare with a stable baseline and inspect results by task type, language, and severity.
- Repeat important stochastic cases and report sample sizes and variation.
- Investigate regressions before changing a threshold to accommodate them.
Use deterministic checks where possible and calibrate model graders against domain reviewers. The MT-Bench research on LLM judges documents position and verbosity biases. Conceal candidate identities during comparisons, vary presentation order, and review disagreements.
Require specialist review for consequential outputs. Recheck calibration when the judge or rubric changes so a grading change is not mistaken for an application improvement.
Decide what can launch and keep monitoring
Prepare a release record containing threshold results, baseline differences, unresolved failures, and owners. Choose whether to release the supported workflows, narrow their scope with human review, or fix and retest.
Production LLM testing continues after launch. Sample real outputs with privacy controls, monitor latency and costs, and turn incidents into regression cases. Assign escalation and rollback ownership before live users encounter a failure.
NIST's Generative AI Profile provides a voluntary risk-management reference spanning design, development, use, and evaluation. Use it alongside application-specific evidence rather than treating a test score as a guarantee.
LuMay's LLM development services include evaluation, benchmarking, and LLMOps. For projects spanning retrieval, agents, or system integration, explore the relevant options in LuMay's AI engineering services overview.
