Who Is Responsible?

When a car's brakes fail and a crash kills someone, we know roughly how to think about responsibility. The driver had a role. The manufacturer had a role. Regulators who set safety standards had a role. Liability law gives us a framework. Now imagine: an AI system gives a patient incorrect drug dosage advice and they are harmed. Who is responsible — the hospital that deployed the system, the company that built the model, the data team that assembled the training set, or the doctor who trusted the AI's output over their own judgment? The answer is not obvious, and that ambiguity is a genuine problem.

The Accountability Gap

AI systems have created what researchers call an accountability gap — a mismatch between the scale of harm an AI system can cause and the clarity of who is answerable for it. Several features of AI make accountability harder than for most technologies. Distributed creation: A large AI system may be built from a foundation model trained by one organization, fine-tuned by another, deployed on infrastructure from a third, and accessed through an application from a fourth. When it fails, each layer can point to another. Emergent behavior: AI systems sometimes behave in ways their creators did not anticipate and cannot fully explain. A model that performs well in testing may fail in a novel real-world situation. Can you be held responsible for behavior you could not predict? Scale: A single AI model might be deployed to millions of users simultaneously. One flaw can cause harms across an enormous population before anyone notices. Opacity: Many AI systems — especially large neural networks — are not easily interpretable. Even their builders cannot always explain why a specific decision was made. This makes it difficult to assign blame to a specific design choice.

Different legal and ethical frameworks distribute accountability differently. Product liability: Manufacturers of physical products are liable for design defects that cause harm. Applying this to AI would make builders responsible for harmful outputs. This creates strong incentives for safety but may be impractical for general-purpose AI that can be used in unlimited ways the builder did not design for. Negligence: A party is negligent if they failed to exercise reasonable care given what they knew. A hospital that deploys an AI medical tool without testing it adequately on the relevant patient population may be negligent. This requires showing the party knew or should have known about the risk. Strict liability: Some activities are dangerous enough that liability attaches regardless of fault or care. Some legal scholars argue AI deployed in high-stakes decisions should carry strict liability — companies would be responsible for harms regardless of how carefully they built the system.

The Human-in-the-Loop Question

One common approach to accountability in AI systems is requiring a human in the loop — a person who reviews AI recommendations before they become decisions. This seems to restore clear accountability: the human who approved the decision is responsible. But as we saw in Lesson 2, automation bias means human reviewers often defer to AI recommendations, especially under time pressure. If a doctor approves an AI diagnosis without genuine independent judgment, is the human 'in the loop' in any meaningful sense? Research suggests that accountability-by-design — building systems that explain their reasoning, make uncertainty visible, and actively flag cases where the AI is less confident — is more effective than simply having a human sign off. A human who understands why the AI reached a conclusion is better positioned to catch errors than one who sees only the conclusion.