AI Governance Principles

By 2019 there were already enough AI ethics guidelines to study as a population. Researchers at ETH Zurich collected every set they could find from companies, governments, universities and professional bodies, and counted 84. Nearly nine in ten had appeared since 2017. The documents agreed, at least in their vocabulary: transparency appeared in 73 of them, justice and fairness in 68, responsibility and non-maleficence in 60 each, and privacy in 47. Yet no single principle appeared in all 84, and when the researchers read what the words meant, they found substantive divergence in how each principle was interpreted, why it mattered, what it applied to and how it should be put into practice¹.

That same year, the Oxford ethicist Brent Mittelstadt explained why agreement on words would not be enough. Medicine turned four principles into practice over decades, backed by a shared aim, professional norms, proven methods and real accountability when things go wrong. AI development, he argued, has none of those to lean on, so principles on their own risk giving false assurance².

Seven years later, the test for an executive is simple. Ask your leadership team two questions. Does our organization have AI principles? In many teams, most hands go up. Can anyone name one decision those principles changed last quarter? Expect far fewer. The distance between those two answers is what this chapter is about.

The core idea

A governance principle is a short, high-level expectation of how AI must behave in the organization, written so that it does not depend on any model, vendor or product. “Every AI outcome has a named owner” is a principle. It prescribes no software, and that is why it lasts.

Principles should stay stable as technology changes. What changes is how deeply each one is implemented, and that depth should follow the risk of the use case. A principle earns its place only when it shapes a real decision: it tells people what must be weighed, and who must decide.

As What Is AI Governance? established, governance is the system of decision rights, controls and oversight. As Why Responsible AI Matters showed, responsible AI is the standard that system tries to meet. Principles are the bridge between the two: the standard written down in a form that people can apply and be held to.

Principles outlast rules, and rules outlast procedures

Three words are routinely confused in governance discussions: principle, rule and procedure. They are layers, and they change at different speeds.

The legal profession offers an unusually clean example. In July 2024 the American Bar Association published its first formal ethics opinion on generative AI. It did not invent new principles. It applied rules lawyers already lived by: competence, confidentiality, communication with clients, reasonable fees, candor to the court and the duty to supervise others. It concluded that how much independent checking AI output needs depends on the tool and the task, and that the partners who manage a firm must set clear policies on its use³.

That is the pattern to copy. The principle, that a lawyer answers for what is filed under their name, is decades old and survived the arrival of AI unchanged. The rule beneath it, verify every AI-drafted citation, is new and specific. The procedure, a cite-check step in the filing workflow, will be redesigned many times. When next year’s tool arrives, the procedure changes and perhaps the rule, but the principle does not. Governance written tool by tool expires with the tool; governance built on principles only needs its lower layers rewritten.

Seven principles, one set

Organizations often publish long lists of principles, and long lists fail for a simple reason: nobody can apply what they cannot remember. This course uses seven, and uses the same seven in every chapter that follows.

The 2019 survey of 84 guidelines shows why even a short list needs more than agreement on words¹.

The set is not new, and it should not be. It lines up with the OECD AI Principles, the first intergovernmental standard on AI, adopted in 2019, updated in May 2024 and now backed by 47 adherents⁴. It also maps onto the characteristics of trustworthy AI in the NIST AI Risk Management Framework⁵. Both are widely cited reference points, so a set mapped to them is easier to defend when an auditor, regulator or customer asks where it came from.

Two choices in the set deserve a word. First, several familiar ideas sit inside a principle rather than beside it: traceability inside accountability, contestability inside transparency, reliability inside human oversight. Folding them in keeps the list short without losing them. Second, proportionality does not appear among the eleven values the 2019 study found¹. It is not a value at all. It is the operating rule that stops the other six from becoming either posters or paperwork. An organization may add an industry-specific principle, such as accessibility in public services, but it should add requirements as standards beneath the seven before it adds new principles above them.

Purpose and accountability come first

Every other principle depends on the first two, because neither fairness nor oversight can be judged until someone knows what the system is for and who answers for it.

Purpose asks why AI is being used at all: which problem it solves, for whom, and what outcome is expected. Purpose also limits use. A tool approved to draft replies for customer support is not thereby approved to rate the agents who use it. A change of purpose is a new decision, and it needs a fresh look. The GDPR has long applied the same logic to personal data through purpose limitation⁶.

Accountability says that a named person owns each outcome. AI sits inside a business process, the process has an owner, and that ownership does not dissolve because a model produced the output. Accountability also needs a record: enough about the model version, the input, the output and the approval to reconstruct a material decision later. Collect enough to investigate, not everything. Who holds which role, and how ownership moves through a system’s life, is the subject of AI Roles, Ownership and Accountability.

What transparency, fairness, privacy and oversight ask

The middle four principles are the ones most often misread, usually by being made either too weak or impossibly strong. Each asks for something specific, and no more.

Transparency means meaningful information for the people affected: that AI is involved, what role it plays, its important limits and how to raise a concern. For consequential outcomes it includes contestability, a real route to question a result and have a person review it. Explainability asks whether we can understand a decision; contestability asks whether the person affected can challenge it. Transparency does not require publishing source code or model weights.

Fairness means AI does not create unjustified harmful differences between groups. It does not mean identical outcomes in every case, and no single fairness metric fits every system. The principle is stable; the measurement is chosen per use case, as Bias and Fairness explained.

Privacy and security mean minimum data, minimum access and protection designed in from the start, not added at the end. The mechanics belong to Privacy and Confidential Data and Security and AI Attacks.

Human oversight means that people keep the ability to understand, override and stop a system. A person somewhere in the workflow is not the same thing; Human Oversight and AI Incidents showed what meaningful oversight requires.

Proportionality: same principle, different depth

Proportionality is the keystone. It says that the depth of control should match the impact, likelihood and exposure of a use case, while the principles themselves stay the same for every system. What Is AI Governance? made the case for proportionate governance overall. Here the idea works inside each principle.

Take transparency. For an internal tool that drafts first versions of meeting notes, a short disclosure that AI helped is enough. For a system that decides whether a customer qualifies for a service, transparency means an explanation the customer can understand, a route to appeal and a person who can change the result. The ABA opinion reaches the same conclusion for verification: checking a brainstormed list of arguments and checking citations bound for a court are different jobs³. The law takes the same approach at its own scale. The EU AI Act sets obligations by risk category, a classification taught in AI Inventory and Risk Classification.

Proportionality cuts both ways. Too little control on a high-impact system is the obvious failure. Too much control on a low-risk one is the quieter failure: it slows adoption, pushes people toward unapproved tools and lowers little risk. Two design habits scale in the same way. Least privilege gives AI only the data and permissions its approved purpose needs. Reversibility designs actions so that mistakes can be undone or contained. Both matter more as autonomy rises, as Agentic AI and Autonomous Actions explained.

When principles collide

Principles do not always point the same way, and pretending they do is how organizations end up with decisions that nobody made. The NIST framework is explicit that trustworthiness characteristics involve trade-offs and that addressing them one at a time does not make a system trustworthy. Its own example: privacy-enhancing techniques can reduce accuracy, which in turn affects fairness⁵.

A fraud model’s operators cannot explain its signals in full without teaching fraudsters to avoid them. A team cannot check whether outcomes differ by group without data about group membership that privacy says to minimize. Complete logs make accountability possible and also create a store of personal data that must itself be protected. Requiring a person to review every case sounds like strong oversight until the reviewer is clearing hundreds an hour.

None of these has a universal answer. What a principle set must supply is the route to an answer: the accountable owner decides, with the relevant specialist, writes down the reason and revisits it when the facts change. A principle that cannot guide a hard trade-off is decoration. One that names what must be weighed and who must decide is governance.

Story: the brief everyone assumed was checked

One of the clearest recent post-mortems of a principles failure comes from a profession that already had the principles. In the spring of 2025, two law firms were jointly representing a client in an insurance coverage dispute in federal court in Los Angeles. A court-appointed special master, a retired magistrate judge, was hearing a discovery dispute, and the parties agreed to file short supplemental briefs on one question. What Exactly Is Artificial Intelligence? told the best-known fake-citation case, Mata v. Avianca, as a lesson about mistaking a chatbot for a search engine. This case teaches something different: what happens when everyone assumes someone else owns the check.

A lawyer at the first firm used several AI research and drafting tools to produce an outline of the argument and sent it to colleagues at the second firm, who worked it into the brief. According to the order, no lawyer or staff member at either firm checked the citations before filing, and the lawyers at the second firm did not know AI had been used, nor did they ask. About nine of the 27 citations in the ten-page brief were wrong. At least two of the cases did not exist, and several quotations attributed to judicial opinions were invented⁷.

The special master could not find two of the cases and emailed the lawyers. The same day, the second firm sent a revised brief without those two citations, but with about six other errors still in it, and with no mention of AI. The order that followed was blunt: no reasonably competent attorney should out-source research and writing to this technology without verifying it. The special master struck every supplemental brief, denied the discovery relief the client had asked for, and ordered the two firms to pay 31,100 dollars toward the special master’s and the defense’s costs. He declined to sanction the individual lawyers, calling the episode “a collective debacle”⁷.

Read the case against the seven principles and the gaps are plain. Accountability was diffused across two firms, each apparently assuming the other had checked: a collective debacle is what shared ownership without a named owner looks like. Transparency failed twice, first between the firms and then to the tribunal, even after a warning. Human oversight existed on paper, a team of lawyers, but no one exercised it. And proportionality was inverted: a filing to a court is the highest-stakes output a law firm produces, and it received the lightest check.

Notice what was not missing. The principles were there, in the profession’s rules and in the bar’s guidance from nine months earlier³. What was missing was everything beneath them: a rule that every AI-assisted citation is verified, a procedure that makes that check a step rather than a hope, an owner for the brief as a whole, and an expectation that AI use is disclosed to colleagues. The firms had principles. They did not have a control.

What this means for leaders

The first lesson is to keep the set short and stable. Seven principles that people remember are far more likely to shape decisions than twenty that nobody reads, and a set mapped to the OECD and NIST frameworks is easier to defend to regulators and customers. The second is that every principle needs an owner and at least one control on the systems that matter most; how a principle becomes policy, standard, process, control and evidence is the subject of the next chapter. The third is that control depth should follow risk in both directions, heavier where the stakes are high and lighter where they are not. And the fourth is to decide in advance who settles collisions between principles, because the hard cases are exactly where principles are tested.

Check yourself

1. Transparency means publishing the model’s source code.MythFact
2. Human oversight means a person must approve every output.MythFact
3. Fairness requires identical outcomes for every group.MythFact
4. The same principle can require very different controls in two use cases.MythFact
5. When generative AI arrived, the legal profession needed new ethical principles to govern it.MythFact
6. Published AI ethics guidelines agree on which principles matter but differ on what they mean.MythFact

What comes next

Principles say what responsible AI must stand for. On their own they change nothing, as the law firms found. Many organizations then make a second mistake: they write a policy and believe they have governance. The next chapter, AI Policy vs AI Governance, separates the two and traces how a principle becomes a policy, a standard, a process, a control and the evidence that it works.