Fairness, Justice, and Power

The formal fairness definitions studied in previous lessons are mathematically precise — but mathematics does not tell us which definition to use. That choice requires a theory of justice. And theories of justice are not neutral: they reflect assumptions about what society owes its members, who has standing to make fairness claims, and what inequalities are acceptable. This lesson connects the technical fairness literature to broader frameworks of justice and examines how power shapes which fairness standards get applied, enforced, and measured.

Three Frameworks of Justice

Distributive justice concerns how benefits and burdens are distributed across society. The central question is: what is a fair outcome? Different theories give different answers. Libertarian theories prioritize voluntary exchange and hold that any distribution resulting from free choices is just, regardless of how unequal it is. Egalitarian theories (such as those of John Rawls) hold that inequalities are just only if they benefit the worst-off members of society. Utilitarian theories hold that the just distribution is the one that maximizes total welfare, potentially at the expense of particular groups. Each of these translates differently into algorithmic choices. A utilitarian approach might accept higher false positive rates for a minority group if the aggregate benefit to the majority is large enough — this is precisely the logic behind optimizing overall accuracy at the expense of subgroup accuracy. A Rawlsian approach would demand that the worst-off group (those most exposed to false positives) be made as well-off as possible — which might correspond to minimizing the maximum false positive rate across groups. Procedural justice concerns whether the process is fair, independent of the outcome. A procedurally just process is one where: decisions are made consistently, individuals can challenge decisions, decision-makers do not have a personal stake, and the basis for decisions is transparent. These are criteria that many algorithmic systems fail: commercial risk assessment tools are often proprietary (not transparent), training data and threshold choices are not disclosed, and affected individuals typically have no meaningful right to contest a score.

Structural justice is particularly important for algorithmic systems because AI is increasingly deployed simultaneously across many domains of life. A person who receives a high risk score from a criminal justice algorithm may also be denied housing by an automated screening system, declined credit by an automated lending model, passed over by a hiring algorithm, and rejected by an automated insurance classifier. Each of these decisions may be individually defensible under some fairness criterion. But if the same groups experience the compounding of adverse algorithmic decisions across all major life domains, the cumulative effect is a form of structural exclusion that is more severe than any individual decision suggests. The sociologist Virginia Eubanks documented this pattern in 'Automating Inequality' (2018), examining automated systems in the child welfare, public assistance, and criminal justice systems in the United States. Her central finding: the same populations — low-income, disproportionately Black and Hispanic — are subjected to the most intensive algorithmic scrutiny in the most consequential domains of life, creating a 'digital poorhouse' that updates the historical function of punitive institutions.

Who Defines Fairness, and Who Enforces It?

Fairness standards do not emerge from mathematics alone. They are defined by people and institutions with interests, and the people and institutions that define them have historically not been the communities most harmed by unfair systems. In the algorithmic fairness research community, the researchers, conference committees, and journal editors who determine which definitions are considered standard are overwhelmingly based at wealthy research universities and technology companies in the United States and Europe. The communities most affected by biased algorithmic systems — people of color in the U.S. criminal justice system, low-income families subject to automated welfare decisions, communities in the Global South targeted by facial recognition — are largely absent from these conversations. This matters because it shapes what gets measured and what gets ignored. The fairness definitions developed and popularized in the academic literature are group-level statistical measures, suited to retrospective audit of large datasets. They are less suited to measuring the harms most salient to affected communities: the experience of being treated as a suspect rather than a person, the loss of dignity in automated rejection, the impossibility of contesting an opaque decision, or the cumulative experience of being targeted by many systems simultaneously. Power also determines enforcement. Algorithmic fairness requirements exist in law (employment discrimination law, the Equal Credit Opportunity Act, the Fair Housing Act) but enforcement has been inconsistent and slow relative to the pace of technological deployment. The companies deploying these systems are typically far better resourced than the regulatory agencies meant to oversee them and the communities seeking to challenge them.

Whose knowledge counts in fairness analysis? The technical literature focuses on quantitative measures — error rates, selection rates, calibration — that can be computed from datasets. But communities affected by biased systems often have qualitative knowledge that statistical audit cannot capture: patterns of mistreatment that do not appear in official records because they are not recorded, experiences of being treated with disrespect that do not show up in outcome data, harms that unfold over years rather than in individual decisions. Researchers in the 'participatory design' and 'community-based participatory research' traditions argue that affected communities should be co-designers of any AI system that will govern their lives — not just subjects of impact assessments after deployment. This is not only a justice claim but a practical one: communities often have contextual knowledge that technical teams lack, and their participation can identify failure modes before deployment rather than after.