Risk, Benefit, and Tradeoffs
A complete analysis of AI risk that does not account for AI's benefits is not careful analysis — it is advocacy. The same rigor that requires taking AI risks seriously requires taking AI benefits seriously, because every policy decision about AI involves a tradeoff: restricting a capability that carries risk also forgoes the benefits of that capability. This lesson develops the framework for evaluating tradeoffs honestly — not to dismiss AI risks, but to reason about them with the fullness the subject deserves.
AI's Documented Benefits Are Substantial
The benefits of AI are not hypothetical. They are documented, quantified, and in many cases life-saving. Medical diagnosis and drug discovery: AI systems have demonstrated performance at or above specialist-level accuracy in specific diagnostic tasks — identifying diabetic retinopathy from retinal scans, detecting malignant melanoma from dermoscopy images, identifying pneumonia from chest X-rays. These capabilities matter most in settings where specialist expertise is scarce: rural areas, low-income countries, contexts where the alternative to AI screening is no screening. AI-assisted drug discovery has accelerated the identification of promising compounds for diseases that have resisted treatment for decades. AlphaFold's protein structure prediction, released to the research community, has been described by structural biologists as transformative — enabling research that would previously have required years of experimental work. Accessibility and inclusion: AI-powered speech recognition, image captioning, real-time translation, text-to-speech, and language assistance tools have made information and communication more accessible to people with visual impairments, hearing loss, motor difficulties, and those who speak minority languages. These are not conveniences — for many users they are the difference between full participation in education, employment, and public life and exclusion from it. Scientific research and knowledge production: AI is accelerating discovery across mathematics, materials science, climate modeling, genomics, and astronomy. Climate models are more accurate, new battery materials are being discovered faster, and genomic analysis that previously required months can be performed in hours. The acceleration of science compounds: faster science produces more science, which produces more capability to address challenges including climate change, pandemic preparedness, and poverty. Productivity and economic development: AI tools are making workers across many fields more productive — software developers, writers, educators, researchers, legal analysts, financial analysts. Productivity gains, if broadly shared, raise living standards. The global economic potential of AI productivity improvements is large.
When evaluating an AI risk, always ask: compared to what? The alternative to AI-assisted medical screening in a rural clinic is often no screening, not specialist-level human screening. The alternative to AI content moderation at social media scale is not thoughtful human moderation of every post — it is no moderation or vastly under-resourced moderation. Honest risk-benefit analysis compares AI to the realistic alternative, not to an ideal baseline.
The Structure of Genuine Tradeoffs
Most real AI policy decisions are not between 'AI with risk' and 'safety without risk.' They are between different distributions of risk and benefit, and the choice of which distribution to accept is a question of values as much as analysis. Consider facial recognition technology. Its risks are documented: higher error rates on darker-skinned individuals, wrongful arrests, chilling effects on public assembly, surveillance overreach. Its potential benefits are real: faster identification of missing persons and trafficking victims, more accurate identification of actual perpetrators instead of wrongful convictions of innocent people, efficiency gains in legitimate law enforcement. A policy that prohibits all uses of facial recognition by law enforcement avoids the surveillance and misidentification harms but also forgoes the victim-identification benefits. A policy that permits all uses maximizes the benefits but accepts the harms. Different communities, with different histories of policing and different exposure to the risks and benefits, may have legitimate different preferences about this tradeoff. The same structure applies to AI in hiring, medical diagnosis, content moderation, credit scoring, and many other domains. There is rarely a policy option that captures all the benefits while eliminating all the risks. Tradeoffs are real and they require value judgments, not just technical analysis. Who bears the risks and who receives the benefits is often as important as the aggregate balance. A policy where AI benefits accrue primarily to shareholders and technology workers while risks fall on low-income communities may have a positive aggregate expected value and still be unjust. Distributive analysis — who gets what, who bears what — is essential to honest tradeoff evaluation.
For each AI application, match the primary benefit to the corresponding risk that must be weighed against it.
Terms
Definitions
Drag terms onto their definitions, or click a term then click a definition to match.
Frameworks for Evaluating Tradeoffs
Several analytical frameworks help structure genuine tradeoff evaluation. Risk-risk analysis compares the risk of deploying an AI system to the risk of not deploying it. If an AI diagnostic tool has a 5% error rate on a particular condition but the human-only alternative in that setting has a 15% error rate, deploying the AI is risk-reducing even if it is imperfect. The relevant comparison is always to the realistic alternative, not to a perfect baseline. Stakeholder analysis identifies who bears which costs and who receives which benefits. A complete tradeoff analysis documents these distributions explicitly rather than assuming that positive aggregate expected value justifies any distribution. Some tradeoff configurations are net positive in aggregate but unjust in distribution — and justice has independent moral weight. Safeguards and conditionality can sometimes improve a tradeoff. If the main risk of a technology is that it will be used without oversight in high-stakes settings, a conditional deployment policy — permit use with mandatory auditing, required explanation, and appeal rights — may capture most of the benefit while substantially mitigating the risk. Not all AI deployments are binary deploy/don't-deploy choices. Threshold reasoning is appropriate when some risks are categorically unacceptable regardless of benefits. If a technology has the potential to cause irreversible large-scale harm, a framework that says 'we will not deploy unless we can demonstrate it cannot cause this category of harm' may be more appropriate than expected-value balancing. Nuclear weapons and certain biological research operate under categorical prohibitions for this reason.
Complete the key principles of tradeoff analysis.
A policy advocate argues that AI hiring tools should be banned because they have documented bias. An industry spokesperson argues that AI hiring tools should be permitted because they process applications more consistently than individual human reviewers, who have their own biases. Which analytical step is most important for resolving this dispute rigorously?
An AI diagnostic tool for a rare cancer has 92% sensitivity and 88% specificity when tested on the population of patients in a high-income academic medical center. It will be deployed in rural clinics where no specialist is available. Which consideration is most important before deployment, and why?
Structured Tradeoff Evaluation
- Choose one AI application from the list below (or propose your own with teacher approval):
- - AI-assisted bail and sentencing recommendations in criminal courts
- - Autonomous weapons systems that can identify and engage targets without human authorization
- - AI tutoring systems that adapt to individual student learning patterns
- - AI systems for predicting which patients in an emergency room are at highest risk of deterioration
- Conduct a complete tradeoff evaluation:
- 1. Benefits case: What are the most significant potential benefits? For whom? What is the realistic alternative, and how does the AI compare to it?
- 2. Risk case: Using the taxonomy from Lesson 2, identify misuse, accident, and structural risks. For each, assess magnitude, probability (or uncertainty), and reversibility.
- 3. Distribution analysis: Who bears the primary risks? Who receives the primary benefits? Are these the same people? If not, does the distribution raise justice concerns?
- 4. Conditionality options: Could specific safeguards, oversight mechanisms, or deployment conditions substantially improve the tradeoff without eliminating the benefit?
- 5. Your conclusion: State your policy recommendation — deploy as-is, deploy with conditions, deploy in limited contexts only, or do not deploy — and defend it as the position that best weighs the evidence.
- Present your analysis as a policy brief addressed to a decision-maker. Acknowledge the strongest counterargument to your recommendation and explain why you nonetheless hold your position.