Where Agents Struggle
Agents are genuinely powerful, but they are not magic. Every strength has a corresponding weakness, and responsible AI literacy means knowing both. The goal of this lesson is not to discourage you from using agents — it is to give you the clear-eyed understanding you need to design them well, deploy them thoughtfully, and catch problems before they become disasters.
Struggle 1: Vague or Ambiguous Goals
An agent is only as good as its goal specification. When the goal is vague, the agent fills in the gaps with its own interpretation — and that interpretation may not be what the human intended. Example: Tell an agent to clean up the project folder. Does that mean delete old files? Move them to an archive? Rename them with consistent formatting? The agent will make a choice. If it chooses delete and you meant archive, weeks of work could vanish in seconds. This is not a limitation of agents alone — a human assistant given the same vague instruction might do the same thing. But the agent will execute far faster, at scale, and possibly without any checkpoint for a human to catch the mistake. Vague goals are far more dangerous with autonomous systems than with human workers who naturally pause to clarify.
Every ambiguous word in an agent's goal is a potential failure point. Words like improve, clean, optimize, update, and handle all mean different things to different people. Before deploying an agent, replace every ambiguous word with a precise, measurable criterion — or build in an explicit clarification step before the agent acts.
Struggle 2: Irreversible Actions
An agent that can act can also act in ways that cannot be undone. Sent emails cannot be unsent. Deleted files may be unrecoverable. Submitted forms process immediately. Posted social media content is seen by real people the moment it goes up. Charged transactions hit real accounts. The faster and more autonomous an agent is, the larger the potential blast radius of an error before a human notices. A single bad decision repeated 500 times before anyone intervenes is far worse than a single bad decision a human makes and immediately corrects. Good agent design treats irreversibility as a first-class concern. High-stakes, irreversible actions should require explicit human confirmation before execution — even if everything else in the pipeline is fully autonomous.
Struggle 3: Hallucination Carried into Action
Language models sometimes generate information that sounds correct but is not — a phenomenon called hallucination. When a chatbot hallucinates, the worst outcome is that the user is given wrong information. When an agent hallucinates, it may act on that wrong information. Example: an agent tasked with summarizing research papers hallucinates a citation — a paper that does not exist — and includes it in the formatted reference list it saves to a document and emails to ten colleagues. The wrong citation is now in a distributed document, and tracing it back is a mess. Agents need verification steps for high-stakes factual claims, especially anything that will be published, shared, or acted upon downstream.
Struggle 4: Long Chains of Dependencies
Multi-step tasks where each step depends on the previous one have compounding failure risk. If step 3 produces a slightly wrong result and the agent does not catch it, that error propagates into step 4, which amplifies it in step 5, and by step 10 the output may be far off target. Each step in the chain is a potential point where a small error grows. The longer and more complex the agent's workflow, the more important it is to build in checkpoints — intermediate verification steps where the agent or a human can confirm that the outputs so far are correct before continuing.
Struggle 5: Tasks Requiring Deep Human Judgment
Some decisions should not be delegated to an agent, not because agents are incapable of generating a response, but because the consequences of an error are too serious and the task requires accountability, ethical reasoning, or lived human experience that an agent cannot replicate. Examples include: medical diagnoses that determine treatment, judicial decisions, disciplinary actions affecting a person's career or freedom, and any decision where a human being needs to be able to appeal to another human being who can be held responsible. An agent can inform these decisions — gather data, summarize evidence, flag relevant facts — but it should not make them unilaterally.
Match each agent failure mode to the design response that best addresses it.
Terms
Definitions
Drag terms onto their definitions, or click a term then click a definition to match.
An agent is told to handle customer refund requests. It processes 300 refunds overnight, but a bug in its logic caused it to approve refunds for orders that were already refunded — meaning 80 customers were refunded twice. Which agent struggle does this illustrate most clearly?
Why is hallucination more dangerous in an agent than in a chatbot?
Red Team an Agent
- Step 1: Read this agent specification. Agent goal: automatically respond to all incoming customer emails using our company's tone guidelines. Tools: email read, email send. Autonomy level: fully autonomous, no human review.
- Step 2: Identify at least three specific ways this agent could go badly wrong. For each, describe the failure, label which struggle category it falls into, and explain what the real-world impact would be.
- Step 3: Redesign the specification to reduce risk. What would you change about the goal, the tools, the autonomy level, or the oversight structure?
- Step 4: Is there a version of this agent you would deploy with confidence? Describe it.