Skip to main content
AI Safety, Alignment & Ethics

⏱ About 20 min20 XP

Who Governs AI?

When something goes wrong with an AI system — a hiring algorithm discriminates, a self-driving car injures a pedestrian, a content-recommendation engine amplifies incitement to violence — who is responsible? Who has the authority to investigate, demand changes, impose penalties, and set rules to prevent recurrence? The answer is not simple, because AI governance is not a single institution. It is a layered, overlapping, and often contested set of actors operating at different scales and with different powers.

Technology Companies: First-Mover Governance

In practice, the first governance layer for any AI system is the company that builds it. Long before any regulator acts, companies make consequential decisions: what data to train on, what safety filters to deploy, what uses to permit in their terms of service, what to disclose and what to keep secret. Some companies have published internal AI principles, safety teams, responsible-use policies, and model cards (structured documents describing a model's intended use, limitations, and evaluation). These are genuine governance instruments. They are also unilateral — set by corporate leadership, not by public deliberation — and subject to revision whenever business priorities change. The history of technology platforms shows that voluntary internal governance tends to weaken under competitive pressure or when it conflicts with revenue. Self-governance is not nothing. It is often faster and more technically informed than government action. But it is structurally limited: a company cannot effectively self-regulate when the potential harms affect parties outside the company, when public accountability is demanded, or when competitive dynamics reward cutting safety corners.

Internal vs. External Governance

Internal corporate governance (policies, ethics boards, safety teams) is typically faster and more technically sophisticated than government regulation. External governance (laws, independent audits, regulatory agencies) is more durable, publicly accountable, and applies equally across competitors. Both have essential but distinct roles.

National governments are the second major layer. A national government has the unique power to make binding law — rules backed by the force of the state — that applies uniformly to all actors within its jurisdiction. The European Union's AI Act, adopted in 2024, is the most comprehensive binding AI governance framework enacted by any major jurisdiction to date. It classifies AI systems by risk level (unacceptable, high, limited, minimal), bans certain uses outright (such as real-time biometric surveillance of public spaces by law enforcement), and requires high-risk systems — including those used in hiring, credit, healthcare, and critical infrastructure — to meet specific technical standards, be registered, and undergo conformity assessments before deployment. The United States took a different approach. Rather than comprehensive legislation, the Biden administration issued Executive Order 14110 in 2023, directing federal agencies to develop risk-based frameworks and voluntary standards. The National Institute of Standards and Technology (NIST) published the AI Risk Management Framework. US governance has relied more heavily on sector-specific regulation — the Food and Drug Administration for medical AI, financial regulators for algorithmic trading, the Federal Trade Commission for deceptive AI-generated content — than on a single horizontal AI law. China has enacted a series of targeted AI regulations: rules governing algorithmic recommendation systems (2022), deep synthesis (deepfakes) regulation (2022), and generative AI measures (2023). China's approach emphasizes content control and national security alongside safety, reflecting different governance priorities. The divergence between these approaches creates regulatory fragmentation: companies operating globally must navigate different, sometimes contradictory requirements across jurisdictions.

Flashcards — click each card to reveal the answer

International Bodies and Multistakeholder Governance

AI development is global — a model trained in the United States is deployed worldwide, a data center in Singapore processes requests from Europe, a foundation model produced by one company is fine-tuned and redeployed by thousands of others. National laws have jurisdictional limits. International governance attempts to extend coordination across borders. The United Nations has been increasingly active: the UN General Assembly passed a resolution on safe and trustworthy AI in March 2024, and UN Secretary-General Antonio Guterres convened a High-Level Advisory Body on Artificial Intelligence that published a report calling for global AI governance structures. UNESCO adopted a Recommendation on the Ethics of AI in 2021, ratified by 193 member states — though a recommendation is not binding law. The OECD (Organisation for Economic Cooperation and Development) published AI Principles in 2019, endorsed by 42 countries, which shaped the language of many subsequent national frameworks. The G7 Hiroshima AI Process (2023) produced voluntary codes of conduct. The UK hosted the first AI Safety Summit at Bletchley Park in November 2023, resulting in the Bletchley Declaration signed by 28 countries, which acknowledged that frontier AI poses serious risks and committed signatories to information-sharing on catastrophic risks. International governance instruments are almost always softer than domestic law: they tend to be voluntary, aspirational, and lacking binding enforcement mechanisms. Their value lies in norm-setting — establishing shared language, shared expectations, and the political foundation for future binding agreements.

The Enforcement Gap

Most international AI governance instruments are non-binding — they express shared principles but carry no penalties for non-compliance. The gap between agreeing on norms and enforcing them is the central challenge of international AI governance, as it is for international governance in general.

A company headquartered in the United States trains an AI system on data from EU citizens and deploys it to EU users for making credit decisions. Which governance bodies have a legitimate claim to regulate this system?

Why are civil society organizations (NGOs, advocacy groups, academic researchers) considered important participants in AI governance, even though they lack the power to make binding law?

Governance Actor Mapping

  1. Choose one of the following AI systems: (A) an AI-powered tool used by US immigration agencies to predict whether a visa applicant is likely to overstay; (B) an AI system used by a global bank to set interest rates on consumer loans; (C) a generative AI service used by 500 million people worldwide to create and share synthetic media.
  2. For your chosen system, map every governance actor that has a legitimate claim over it:
  3. - Which national governments and why?
  4. - Which international bodies and with what authority (binding or voluntary)?
  5. - Which regulatory agencies within those governments?
  6. - Which civil society actors have a stake?
  7. - What is the company itself doing or required to do?
  8. For each actor, describe: (1) the nature of their authority, (2) their primary concern, and (3) one limitation of their governance capacity for this system. Present your map to the class and discuss where the gaps are largest.