The AI Governance Operating Model

Suppose you are the chief risk officer of a company with twenty thousand employees, and two memos reach you on the same morning. The first, from the general counsel, proposes that every AI use case, from a meeting summarizer to a credit model, goes to one central committee for approval. The second, from the head of the largest business unit, proposes the opposite: let each unit decide for itself, because the units know their customers and cannot wait a month for an answer.

Both memos are reasonable, and both describe a familiar failure. Follow the first for a year. The committee meets monthly, the queue grows, routine requests wait as long as dangerous ones, and people quietly stop asking. Follow the second for a year. Five units write five sets of rules, buy overlapping tools and classify the same risk five different ways, and nobody can tell the board how many AI systems are running or which of them matter. One road ends in a bottleneck. The other ends in fragmentation.

Look at what the two memos argue about: who should be in charge. Yet neither failure comes from choosing the wrong person. Each comes from how decisions flow. That is a design question, and the design has a name: the governance operating model.

The core idea

A governance operating model that scales does three things. It centralizes what must be consistent across the organization. It delegates what needs business context. And it lets risk decide how far each decision has to travel before someone makes it.

The third part does most of the work. In a large organization, much AI use is routine: drafting, summarizing, searching documents, help with code. A small share can hurt customers, operations, finances or compliance. Specialist attention from risk, legal, privacy and security is the scarcest thing governance has. A model that spreads it evenly across every request spends most of it in the wrong place, and makes the dangerous cases wait behind the trivial ones.

The earlier chapters supplied the parts. AI Governance Principles set out what governance is for. AI Policy vs AI Governance separated the written rules from the system that applies them. AI Roles, Ownership and Accountability gave each system a named owner and placed the three lines. The operating model is how those parts work together when there are hundreds of systems rather than one.

An org chart is not an operating model

Many organizations believe they have a governance operating model because they have drawn one. The drawing shows boxes and reporting lines. It is the visible part, and the smallest.

Beneath the chart sit the things that actually govern. Decision rights say who may approve what, and up to which limit. Policies and standards say which rules apply. The workflow is the path a request follows from idea to production. Services are what teams can use without inventing their own: templates, approved tools, review slots. Controls run day to day. Escalation says when a decision leaves the team that started it, and where it goes. Monitoring tells anyone whether the whole thing works.

The standards describe governance the same way, as a working system rather than a structure. ISO/IEC 42001, the international standard for AI management systems, requires top management to assign and communicate responsibilities and authorities for the relevant roles, and then to operate, evaluate, audit and continually improve the system¹. The NIST AI Risk Management Framework treats govern as a cross-cutting function that enables the other three, map, measure and manage. It asks that roles, responsibilities and lines of communication be documented and clear to teams throughout the organization². Neither prescribes a chart. Both expect decisions to flow somewhere predictable.

Three designs, three prices

There are three broad ways to arrange the flow, and each has a price.

A centralized model puts most authority in one function: policy, the risk framework, standards, approvals, the inventory and reporting. It buys consistency, specialist depth and a single view. Its price is distance from the work and, as volume grows, a queue. A federated model gives authority to the businesses, which own their use cases, their implementation and their monitoring. It buys context and speed. Its price is inconsistency, duplicated effort and a picture that nobody can add up. A hybrid model sets common rules at the center and leaves execution in the business. Its price is subtler: the line between the two has to be drawn precisely, or it blurs, and blurred lines recreate both failures at once.

In practice, organizations mix. McKinsey’s 2025 global survey found that companies centralize selectively. Risk and compliance and data governance were most often run in a fully centralized way, for example through a center of excellence. Technical talent and the adoption of AI solutions were most often run in a hybrid or partly centralized way³. That is the pattern this chapter recommends: centralize the rules, distribute the use.

Where governance lives is far from settled. In a 2025 survey of more than 670 governance professionals by the IAPP and Credo AI, no single function held primary responsibility for AI governance in more than 22 percent of organizations⁴.

The spread is the point. Whichever function hosts governance, the work crosses all the others, so the home matters less than the route. There is also no universal winner among the three designs. A small company with one line of business may run perfectly well centralized. A group of distinct, separately regulated businesses may need more federation. Hybrid is common in large enterprises, not mandatory.

What the center keeps and what the business owns

The practical question is what goes where. The answer follows from asking where consistency creates value and where context does.

Consistency matters wherever things must be compared or added up. The board needs one view of AI risk, which is only possible if every unit classifies risk the same way, applies the same minimum controls and reports into the same inventory. Approved tools and platforms belong at the center for a similar reason: one well-reviewed platform is cheaper and safer than five unreviewed ones.

Context matters wherever the work happens. The people closest to a process know which use cases matter, how the process should change and what users will actually do. They also run the system day to day. Specialists in security, privacy, legal, risk, data and architecture are drawn in when the risk of a particular system calls for them, not on every request.

This is the Three Lines Model applied to a portfolio, as AI Roles, Ownership and Accountability applied it to a single system: the business owns and manages its risk, a second line sets frameworks and challenges, and internal audit gives independent assurance⁵. The central governance function is an enabler and a coordinator. It does not own every system.

Delegation needs a boundary to be safe. Each delegated decision needs written thresholds, a named owner and a way for the center to see whether it is drifting, such as sampling local approvals and tracking exceptions. Delegation without a boundary is fragmentation with better manners.

Risk decides how far a decision travels

The mechanism that makes delegation safe is routing. Risk is not a label for the quarterly report. It is the rule that decides which path a request takes.

Where risk is lower, the decision stays local. A named business owner approves it, standard controls apply and automated checks run. Where risk is higher, the decision travels: specialists review it, a cross-functional or central body approves it formally, and oversight continues after launch. How to define the tiers is the subject of AI Inventory and Risk Classification. Here the point is only that the tier chooses the path.

The US government shows the design at work. In April 2025 the Office of Management and Budget told federal agencies to cut bureaucratic bottlenecks, let leaders delegate the acceptance of risk, and keep a heavier set of minimum practices for high-impact AI only⁶. The memo binds agencies, not companies.

Some routing is not yours to choose.

Governance that teams can use

If governance means email chains, spreadsheets, manual signatures and five unconnected reviews, teams will go around it. A mature function behaves like a service: one front door, a visible path, and only the steps the risk requires.

The intake asks only what supports a decision: what the use case is, who uses it, who is affected, which data and which model, and what actions it can take. The request is registered and classified, and the route follows. Only the reviews and controls that route requires are triggered. Approval comes from a named person within a stated service level, and the requesting team can see where its request stands. After launch, ownership continues, and a material change in model, data, provider, users or autonomy sends the system back through the route. The details belong to AI Lifecycle Governance and AI Evaluation and Approval Gates.

Reuse is what makes the service fast. Once the center has approved a pattern, for example a drafting assistant on internal documents with a known set of controls, the next request that fits the pattern can be approved against it in days. That is the logic of streamlining approvals for similar risk profiles.

Usability is not a courtesy. When the approved path is slower or worse than the unapproved one, people take the unapproved one. Shadow AI, the subject of Privacy and Confidential Data, is often a verdict on the path as much as on the people.

Four ways operating models fail

Operating models tend to fail in four recognizable ways, and each has a known fix.

The bottleneck sends every request to the center; the fix is risk-based routing and delegation. Fragmentation lets every unit write its own rules; the fix is common principles and minimum standards, with local context on top. Theater produces meetings, policies and dashboards that change no decision; the test is whether governance ever changes what someone does. The spreadsheet failure is the quietest: a spreadsheet works at twenty systems, but at hundreds it loses track of owners, versions and approvals, and it leaves no reliable audit trail.

To see which failure is under way, measure the model rather than its activity. Useful measures include the time to approval on each route, the share of requests decided locally, the number of exceptions, overdue reviews and how complete the inventory is. Technology can automate intake, routing and evidence. People still make the governance decisions and own their outcomes.

Story: one elevator company, two years

The following is an illustrative composite, built from patterns common in building-services firms rather than from one company.

A regional elevator maintenance company, servicing tens of thousands of lifts and escalators, had a central AI governance team of eight: specialists in risk, privacy, security, data and legal, with a lead who reported to the chief risk officer. In two years its inventory grew from about twenty AI systems to more than three hundred. The team did not grow.

In the first year the company ran design A: every system went through the eight. A meeting assistant, a drafting tool for customer letters and a model that recommended taking a lift out of service when its sensors showed a fault all joined one queue, in order of arrival. The eight spent most of their week on low-risk productivity tools. Field technicians and the dispatch center waited weeks for simple answers, then stopped asking, and unapproved tools appeared on laptops in the remote-monitoring center and the billing office. By the end of the year the committee was busy, the queue was long and the inventory was wrong.

In the second year the company changed the route, not the headcount.

Design B kept policy, the risk taxonomy and standards at the center. The center published an approved toolkit and a small library of patterns, each with standard controls, so business owners could approve routine use themselves with automated checks. Specialists joined only when the route required them. The eight now spent their time where the harm could be real: fault prediction, customer billing, alerts for trapped passengers and the shutdown model, where a wrong call can leave a faulty lift running. Lifts fall under EU product rules listed in Annex I of the AI Act, so a model that works as a safety component may be high-risk whatever the company’s own tiers say.

In this illustration, two years on, the company reviewed fewer things, more deeply. Routine approvals took days rather than weeks. Unapproved tools became rarer because the approved path had become the easy one, and the inventory became reliable because registering a system was the fastest way to get it approved. Design A had not failed for lack of effort. It had tried to scale governance by adding manual review, and manual review alone does not scale.

The scale test

Governance teams everywhere feel the pressure that the elevator company felt. In the IAPP survey, only 10 of 671 respondents said they would not need more AI governance staff within the next year⁴. Some growth is justified. But headcount cannot keep pace with adoption that doubles, and an operating model that needs it to has a structural flaw.

The test is simple to ask of any organization. If AI use doubled next year, would the governance team have to double too? If yes, the model is a queue. If the answer is that more patterns would be standardized, more checks automated and more routine decisions delegated, so that the same specialists spend their time on the systems that matter, the model scales.

What this means for leaders

Four lessons follow for anyone who owns or oversees AI governance.

1. Design the flow, not the chart. Decide who may approve what, on which route, before deciding where the team reports.
2. Centralize the rules, distribute the use. Keep policy, taxonomy, standards, approved platforms and the inventory central. Leave priorities, design and operation with the business, inside written boundaries.
3. Make risk the router. Let the tier, and where it applies the law, decide how far each decision travels. Lighter controls for routine use are not an exemption.
4. Run governance as a service and measure it. One intake, visible status, stated service levels, reusable patterns, and measures of the model itself rather than of its activity.

Check yourself

1. A central AI governance team should approve every AI use case.MythFact
2. Federated governance means each business unit can write its own AI rules.MythFact
3. Risk-based governance means low-risk AI needs no governance.MythFact
4. Surveys find that organizations often centralize risk, compliance and data governance while running AI adoption in a hybrid way.MythFact
5. An organization’s internal tiers decide whether a system is high-risk under the EU AI Act.MythFact
6. If AI use doubles, a well-designed governance function needs to double as well.MythFact

Reflection: trace one request

What comes next

The operating model sets the center, the business, the route and the services. Many large organizations run it through two formal structures: a body that makes the hard calls, and a team that keeps the model running every day. The next chapter, AI Governance Committee and AI Governance Office, looks at each of them and at how they divide the work.