Small Stakes and Big Stakes
Not every use of AI is equally critical. An AI that suggests which movie to watch tonight operates in a very different world of consequences than an AI that recommends a cancer treatment or approves a bail decision. Understanding the difference — and why it matters for how much safety work is required — is one of the most practical skills in AI literacy.
Defining Stakes
The stakes of an AI application are determined by two questions: How severe is the worst-case harm if something goes wrong? And how reversible is that harm? A movie recommendation that misses the mark causes mild disappointment — low severity, completely reversible. A medical diagnosis AI that misses a cancer could delay life-saving treatment — high severity, potentially irreversible. The higher the severity and the lower the reversibility, the higher the stakes. A third factor matters too: who is affected. An AI affecting one person in one decision is lower stakes than an AI affecting millions of people across a society. Scale amplifies stakes.
Stakes are determined by severity (how bad is the worst-case outcome?), reversibility (can the harm be undone?), and scale (how many people are affected?). High severity, low reversibility, and large scale together define a high-stakes AI application.
A Spectrum of AI Applications
Consider the following spectrum from low to high stakes: At the low-stakes end: autocomplete suggestions in a text message app, a playlist recommendation engine, a virtual assistant that sets a timer, or a game AI opponent. Mistakes are annoying but harmless. No life changes based on these outputs. In the middle: an AI that grades student essays, a job application screener, a navigation system that routes traffic. Errors can affect important life outcomes — grades, jobs, travel efficiency — but are often correctable and rarely life-threatening. At the high-stakes end: AI that assists medical diagnosis, AI that supports criminal sentencing or parole decisions, AI that controls autonomous vehicles, AI that detects cybersecurity threats on national infrastructure, AI that assists in targeting decisions in military systems. In these contexts, errors can end or severely harm lives, and the harm may be irreversible.
The European Union's AI Act, passed in 2024, explicitly classifies AI applications into risk tiers and requires stricter safety standards for higher-risk applications. This reflects a growing global consensus that not all AI deserves the same level of scrutiny — but high-stakes AI demands exceptional care.
The EU AI Act categorizes AI applications into risk tiers — unacceptable risk, high risk, limited risk, and minimal risk — and applies different rules to each. High-risk AI (such as medical devices and critical infrastructure) faces strict requirements for human oversight, data quality, and transparency.
Rank each AI use case by matching it to the correct stakes level.
Terms
Definitions
Drag terms onto their definitions, or click a term then click a definition to match.
Stakes Determine Safeguards
The point of understanding stakes is not to fear high-stakes AI, but to calibrate the safeguards appropriately. High-stakes applications require more rigorous testing, independent audits, human oversight checkpoints, clear accountability, and appeal processes. Low-stakes applications can often be deployed more quickly with lighter oversight. A useful question when encountering any AI system: What is the worst realistic outcome if this system makes a mistake? If the answer is minor inconvenience, lighter oversight may be fine. If the answer involves serious harm to a real person's life, liberty, or health, robust safeguards are non-negotiable.
An AI system assists judges in sentencing decisions in criminal courts. Using the stakes framework, what best describes this application?
What are the three factors that together determine the stakes of an AI application?
Stakes Assessment Workshop
- Step 1: For each AI application below, rate its stakes on a scale of 1 (minimal) to 5 (extremely high). Write one sentence justifying your rating.
- A) AI that translates languages in a travel app
- B) AI that monitors students' eye movements during online exams to detect cheating
- C) AI that controls insulin dosing for diabetic patients
- D) AI that generates captions for social media photos
- E) AI that scores job candidates' video interviews based on facial expressions and speech patterns
- Step 2: Pick the two applications you rated most differently from each other. Write a paragraph explaining what specific factors account for the large difference in stakes.
- Step 3: For the highest-stakes application you identified, propose three concrete safeguards that should be required before it is deployed.