Evaluating LLM Output

The previous eight lessons have given you a rigorous foundation: you know what language models are, how they are built, what they can genuinely do, and where they reliably fail. This lesson brings all of that together into a practical discipline: how do you evaluate an LLM's output? How do you decide what to trust, what to verify, and when not to use an LLM at all? These are professional skills — the same ones that distinguish a sophisticated AI user from one who is vulnerable to the technology's failure modes.

A Framework for Evaluation

Evaluating LLM output is not about finding errors after the fact — it is a systematic practice that begins before you even write the prompt. The framework has three stages: before, during, and after. Before: Assess the task. For each task you are considering delegating to an LLM, ask: - Is this task well-matched to LLM capabilities? (Text generation, transformation, synthesis, brainstorming — generally yes. Exact arithmetic, real-time data, verified citations — generally no.) - How serious are errors? (Drafting a blog post: errors are easy to catch and fix. Summarizing a medical document for a clinical decision: errors could be dangerous.) - Do I have the background to verify the output? (If you cannot evaluate whether an LLM's legal analysis is correct, you should not rely on it without independent expert review.) During: Monitor as you work. Read each LLM response critically: - Does it actually answer the question you asked, or does it answer an adjacent, easier question? - Are there specific factual claims — numbers, names, dates, citations — that need verification? - Does the reasoning chain hold together, or does a step slip in that does not follow from the previous? - Is the model expressing appropriate uncertainty, or is it stating uncertain things with false confidence? After: Verify before acting. Any LLM output you intend to act on — publish, submit, use to make a decision — should go through explicit verification of its specific claims.

Specific verification strategies: Fact-checking specific claims: Any specific factual claim — a statistic, a historical date, a scientific finding, a legal rule, an attribution — should be verified against a primary or reliable secondary source. Do not use a different LLM query as verification; that LLM may hallucinate the same false fact. Use authoritative external sources. Code verification: LLM-generated code should be tested, not just read. Code that looks correct may have edge-case bugs, security vulnerabilities, or deprecated API calls. Run it. Test it with inputs it might fail on. Read it line by line for logic errors the tests might miss. Mathematical verification: Do not trust LLM arithmetic. Verify every computation with a calculator or by hand. For more complex mathematical reasoning, check the logic of each step independently. Cross-checking with multiple sources: For general factual claims, verifying against two or three independent authoritative sources provides much stronger confidence than accepting LLM output alone. If multiple sources agree and the LLM's claim matches, the risk of hallucination is lower. If sources conflict or the LLM's claim is not corroborated, treat it as unverified. Knowing your own domain: The most powerful verification tool is your own expertise. An expert reading an LLM's medical explanation can spot implausibilities that a non-expert would miss entirely. This is why domain knowledge remains essential even in an age of powerful LLMs — the LLM's output needs a knowledgeable human in the loop.

When Not to Use an LLM

Not every task is appropriate for LLM assistance. Recognizing when not to use one is as important as knowing how to use one well. High-stakes factual claims without verification infrastructure: If you need accurate, current, specific factual information for a decision where errors have serious consequences — medical, legal, financial — an LLM alone is not an appropriate tool. Use it only if you have the expertise and resources to verify its output rigorously. Private or sensitive information: Many LLM services send your prompts to external servers. Pasting confidential business information, personal medical records, or private communications into a commercial LLM may violate privacy obligations or expose sensitive data. Tasks where you cannot evaluate the output: If you do not have the domain knowledge to assess whether the LLM's output is correct, and the stakes of being wrong are meaningful, do not use the LLM as your primary source. You cannot catch what you cannot recognize. Tasks requiring exact reproducibility: LLM outputs are stochastic — the same prompt may produce different outputs on different runs. For tasks where you need a guaranteed identical output every time (like a legal template that must be verbatim), LLMs are the wrong tool. When attribution is required: LLM output is not citable as a source. If your work requires citing sources — academic papers, journalism, professional reports — the LLM can help you draft, but the underlying claims must be grounded in citable human sources.