Privacy, Data Rights, and Consent

Every time you use a smartphone, swipe a loyalty card, search the web, or walk past a security camera, you generate data. That data is collected, stored, combined with other data about you, and used to train AI systems you may never interact with directly. The scale of modern data collection would have been unimaginable to the architects of privacy law fifty years ago. This lesson examines what privacy actually means, why it matters, how consent has become a broken mechanism, and what meaningful data rights might look like.

What Is Privacy, and Why Does It Matter?

Privacy is not primarily about secrecy. It is about control — specifically, the ability to control information about yourself and to present different aspects of yourself in different contexts. This contextual integrity framework, developed by philosopher Helen Nissenbaum, holds that a privacy violation occurs not simply when information is shared, but when information flows in ways that violate the norms of the context in which it was originally disclosed. You share your medical diagnosis with your doctor. Sharing it with your employer — without consent — is a privacy violation even if the information itself has not changed. The violation is about context and control, not secrecy. Why does privacy matter? Several distinct reasons have been articulated: Autonomy: The ability to control information about yourself is bound up with the ability to define your own identity and make choices free from external pressure. Dignity: Being reduced to a data profile — a set of inferred attributes — strips away the complexity of personhood. Algorithmic profiling treats people as bundles of statistics. Power asymmetry: Those who collect and analyze data gain power over those who generate it. This asymmetry is central to the business models of major technology companies and to the capabilities of surveillance states. Chilling effects: When people know they are watched, they change their behavior — avoiding certain searches, certain associations, certain speech. Surveillance therefore constrains freedom even for people who have done nothing wrong.

The aggregation problem is one of the most important privacy phenomena in the age of AI. Individual pieces of data that seem innocuous in isolation can be combined to reveal sensitive facts that no single piece would reveal. Your first name: not sensitive. Your employer's name: not sensitive. Your neighborhood: not sensitive. The time you leave home each morning: not sensitive. Combined: a stalker's dossier. AI systems excel at aggregation. Systems trained on large behavioral datasets routinely infer attributes that users never disclosed: political affiliation from shopping patterns, health conditions from browsing history, sexual orientation from social network structure, mental health status from posting frequency. These inferences are often more accurate than the intuitions of people who know the user personally. The inference problem is particularly difficult from a consent perspective: you cannot consent to sharing information that is inferred about you, because you did not know it would be inferred, and the inference may be invisible to you even after it occurs.

The Problem of Consent

Privacy law in most jurisdictions is built on the concept of informed consent — the idea that collecting and using data about someone requires their meaningful agreement. In practice, modern data consent is largely fictional. Consider a typical terms-of-service agreement. Studies have found that reading all the privacy policies a person encounters in a year would take approximately 76 working days. No one reads them. 'Notice and choice' — the dominant consent model in US privacy law — offers a choice between accepting the policy or not using the service. For services that have become infrastructure — search, email, maps, social media — this is not a meaningful choice. Even when privacy preferences are clearly expressed, they are routinely violated. Dark patterns — interface designs that manipulate users into sharing more than they intend — are endemic in the industry. Cookie consent banners that require twelve clicks to opt out, while accept-all requires one, are a trivial example of a much more systematic problem. The GDPR's approach — requiring specific, informed, freely-given, and revocable consent for each purpose — is more demanding and has changed industry practices in Europe. Critics argue it has also created compliance theater: long consent forms that technically satisfy the standard while remaining practically unintelligible. Some privacy scholars argue that individual consent is the wrong model entirely for population-scale AI systems. When a model is trained on millions of people's data, and you can opt out, the impact on the model is negligible while your individual burden is high. Consent operates individual-by-individual; AI risk operates at population scale. This mismatch may require collective governance mechanisms rather than — or in addition to — individual consent rights.