AI Roles, Ownership and Accountability

On the night of 18 March 2018, a test vehicle run by Uber’s self-driving unit struck and killed Elaine Herzberg as she pushed a bicycle across a road in Tempe, Arizona. The car was driving itself. A backup safety driver sat behind the wheel, there to take over if the system failed. Its sensors had detected Herzberg 5.6 seconds before impact, but the system never correctly classified her as a pedestrian¹.

Before reading on, make a prediction. Many parties shaped that outcome: the company that designed the system, the engineers who tuned it, the managers who set the testing program, the state that permitted it and the driver in the seat. Who do you predict faced criminal charges?

In March 2019, the prosecutor reviewing the case found “no basis for criminal liability for the Uber corporation”². The backup driver was charged with negligent homicide. In July 2023 she pleaded guilty to endangerment and was sentenced to three years of supervised probation³. The National Transportation Safety Board found that her distraction by her phone was the probable cause. It also named contributing factors that no single driver controlled: an inadequate safety culture at Uber’s self-driving unit, inadequate safety risk assessment, ineffective oversight of vehicle operators, and no adequate way of handling operators’ “automation complacency”¹.

The point is not to retry the case. It is to notice where accountability came to rest. Most of the decisions that shaped the system were made far from the driver’s seat, and the formal blame landed in it.

The core idea

Accountability for an AI system is either designed or it defaults. When nobody designs it, it goes one of two ways, both bad. It spreads so thinly across many contributors that nobody holds it, or it falls on the person nearest the machine, who usually had the least say in how the machine was built.

Designing accountability means two things. First, every governance role has a distinct job: some roles own, some challenge and some give independent assurance. Second, every material AI system has one named owner, a person in a business role who has the competence, the authority and the support to decide about it, including the decision to stop it.

As AI Decision Rights and Accountability, in Module 04, showed, being responsible for a piece of the work is not the same as being accountable for the outcome, and each material decision needs exactly one accountable role. That chapter applied the idea to strategy and to individual decisions. This one applies it to governance: who plays which part, and what makes an owner’s role real rather than nominal.

Two ways accountability goes missing

The first default has a name in political philosophy. In 1980, Dennis Thompson described the problem of many hands: because so many people contribute in so many ways to an organization’s decisions, it becomes hard to hold any one of them responsible for the result⁴. AI makes the problem sharper. A single system can involve a business unit, a product team, data scientists, engineers, security, privacy, legal, risk and an outside vendor. Each can say, truthfully, that its part worked.

The second default is the opposite. The researcher Madeleine Clare Elish called it the moral crumple zone. Responsibility is pinned on the human operator nearest an automated system, even when that person had limited control over how the system behaved. Like the crumple zone of a car, it absorbs the impact, but what it protects is the system and the organization behind it, not the person⁵.

Both defaults are easy to fall into, and they often arrive together. In the incident review, accountability diffuses across the teams that built the system. In the disciplinary file, it concentrates on the employee who clicked approve. Neither outcome improves the system, and both teach people that the safest place to stand is far from the decision.

Survey evidence suggests that many organizations have not settled the question even at the top. In McKinsey’s 2025 global survey, 28 percent of respondents whose organizations use AI said their CEO is responsible for overseeing AI governance, and 17 percent said the board is. On average, respondents reported that two leaders are in charge⁶. Shared oversight at the top can work. Shared ownership of a single system rarely does.

Three lines give every role a different job

One of the most widely used maps of governance roles comes from internal audit. The Institute of Internal Auditors’ Three Lines Model, updated in July 2026, separates three kinds of contribution. Management, the first line, owns and manages risks and is responsible for designing and operating the processes and controls. Second-line roles provide specialized expertise, support, monitoring and challenge. Internal audit, the third line, gives independent and objective assurance without taking on management’s responsibility. Above all three, the board provides oversight and delegates authority to management⁷.

The model was not written for AI, which is its strength. AI governance does not need a new theory of roles. It needs the existing one applied to a new kind of system.

The right-hand column matters as much as the left. Many ownership failures come from a role drifting across its boundary: a board that approves individual tools, a privacy team that becomes the de facto owner because it signed the last review, or engineers who end up accepting business risk because nobody else turned up to the meeting.

Frameworks written for AI say the same. The NIST AI Risk Management Framework asks that roles, responsibilities and lines of communication for managing AI risk be documented and clear to individuals and teams throughout the organization, and that executive leadership take responsibility for decisions about AI risk⁸. ISO/IEC 42001, the certifiable standard for AI management systems, requires top management to make sure that responsibilities and authorities for relevant roles are assigned and communicated⁹. A central governance function may coordinate all of this; how that function is set up belongs to AI Governance Committee and AI Governance Office.

An owner who can decide and stop

Naming an owner is easy. Making the name mean something is harder. A useful test comes from the EU AI Act. For high-risk systems, it requires deployers to assign human oversight to people “who have the necessary competence, training and authority, as well as the necessary support”¹⁰. The article is about oversight, not ownership, but the four words make a good test for any owner of any material AI system.

Authority is the test most easily failed. An owner who must ask three committees before pausing a misbehaving system is not an owner but a spokesperson. The practical check is simple: could this person stop the system tonight, on their own judgment, without being overruled by the team that runs it?

The owner should sit where the outcome lands. If the system decides which customers get a credit limit increase, the owner is in lending, not in data science. If it screens job applicants, the owner is in talent acquisition, not in IT. Specialists can and should be named alongside, for the model, the data and the platform, but they are responsible for their parts, not accountable for the result.

Banking supervisors reached this conclusion for models years ago. The UK Prudential Regulation Authority’s model risk principles, in force since May 2024, expect each bank to give overall responsibility for its model risk framework to a named senior manager under the UK’s senior managers regime¹¹. A name, a role and personal accountability: the same pattern, applied to the framework rather than to one system.

Ownership also has to survive change. People move, teams reorganize and vendors are replaced. A system whose owner left last spring is ownerless until someone notices. A simple rule prevents this: when an owner moves, ownership passes to their manager until it is formally reassigned, and the owner field in the AI inventory is checked whenever an organization chart changes. The inventory itself belongs to AI Inventory and Risk Classification.

Specialists challenge; they do not own

The second line exists to make the owner’s decisions better, not to take them. Risk, compliance, legal, privacy and security set standards, supply expertise, review evidence and, when necessary, escalate. In banking, this has long been called effective challenge. US supervisors made it the guiding principle of their 2011 model risk guidance, SR 11-7: critical analysis by objective, informed parties who can identify a model’s limitations and get it changed. They added that it depends on incentives, competence and influence, so a challenger nobody has to listen to is not one¹². That guidance was replaced in April 2026 by updated interagency guidance¹³.

Two failures are worth naming. In the first, the second line becomes the owner by default: because legal signed off, everyone treats legal as accountable, and the business stops thinking about the risk. In the second, the first line outsources its judgment: the owner treats a clean review as permission rather than as one input to a decision that remains theirs. Both make the review the decision. How evaluation and approval should work in practice is the subject of AI Evaluation and Approval Gates.

The new US guidance also shows why an organization cannot wait for regulators to define its roles. It states plainly that generative and agentic AI models “are not within the scope of this guidance”, and the agencies plan to ask the industry about AI separately¹⁴. For generative and agentic systems, which many banks are now adding, the role map is the bank’s own to draw.

Your legal role is not your org chart

Internal roles are one layer. The law adds another, and the two are easy to confuse. The EU AI Act assigns duties by role in the value chain, chiefly provider, the organization that develops a system or places it on the market under its own name, and deployer, the organization that uses it. Which role you hold depends on what you do with a system, not on what your contract calls you¹⁰.

This matters for ownership in two ways. First, a team can change the organization’s legal role without meaning to, for example by fine-tuning a supplier’s system and releasing it under the company’s brand. The owner of each system should know which legal role it puts the organization in, and should treat any change that could shift it as a decision, not a technical detail. Second, a supplier’s duties never replace your own. The Institute of Internal Auditors puts it in a sentence: “Outsourcing does not change accountability”⁷. The vendor answers for its product under its contract and its own obligations. You answer for how you chose to use it.

Story: the exam algorithm with many hands and no owner

This is a documented case, examined afterward by an official review. It involves a statistical model rather than machine learning, but its accountability lesson carries over unchanged to AI.

In March 2020 the pandemic closed schools in England, and the summer exams were cancelled. Ofqual, the independent exams regulator, had advised that, if possible, exams should go ahead with social distancing. The Secretary of State for Education decided to cancel them and issued Ofqual a formal direction setting out government policy; grades would be calculated instead¹⁶.

Many hands then shaped the result. Ministers set the policy. Ofqual designed the standardization model and governed it through its board, a standards and technical issues group, a policy and implementation group and an external advisory group. The exam boards ran it. Schools and colleges supplied the grade they expected each student to achieve and a rank order of their students. Key quality criteria were that grade distributions should be similar to previous years and that attainment gaps between groups should not widen¹⁶.

A-level results came out on 13 August 2020. In England, 39.1 percent of grades came out lower than schools had estimated: 35.6 percent by one grade, 3.3 percent by two and 0.2 percent by three¹⁷. After four days of public outcry, Ofqual announced on 17 August that students would receive their school’s estimate or the calculated grade, whichever was higher, and said: “we are extremely sorry”¹⁸.

Then accountability came to rest. On 25 August the chief regulator of Ofqual stood down. On 26 August the Department for Education announced that its permanent secretary, its most senior civil servant, would leave, saying the Prime Minister had concluded there was “a need for fresh official leadership”. The same day the Prime Minister told pupils that their grades had been “almost derailed by a mutant algorithm”¹⁹. The Education Secretary who had issued the direction remained in post until a reshuffle in September 2021²⁰. Before a parliamentary committee in September 2020, Ofqual’s chair said that “the blame lies with us collectively”, and that “the fundamental mistake was to believe that this would ever be acceptable to the public”²¹.

The Office for Statistics Regulation reviewed the episode and published its findings in March 2021. It was fair to the people involved: in its view the teams “worked with integrity” to find the best method in the time available, and all the regulators had put clear governance structures in place. But, it added, “it is not clear to us however how risks around public acceptability and what was achievable in the timescales were effectively managed within these governance structures”, and the public acceptability of large changes from schools’ estimates was never tested. Its fourth recommendation is the one for this chapter: wherever a model is used, accountability should be clear, and in particular “the roles of commissioner (typically a Minister) and model developer” should be clear, as should the communication between them¹⁶.

Read it through the ideas in this chapter. Each party could truthfully say its part had worked: that is the problem of many hands. Plenty of governance existed, but no single role owned the outcome that mattered most, whether the public would accept the grades, and nobody had agreed in advance who could stop the system or on what evidence. And when the reckoning came, it fell on two officials and on “the algorithm”, the institutional version of the moral crumple zone. The cost also outlived the summer. Announcing the approach for 2021, the Education Secretary said: “This year, we will put our trust in teachers rather than algorithms.” The review warned that public bodies might become less willing to use statistical models at all¹⁶. Undesigned accountability had cost not just one year’s grades but the confidence to use models again.

What this means for leaders

The lessons are few, and they are about design rather than goodwill. Accountability will settle somewhere whether or not you place it, so place it deliberately: one named business owner per material AI system, with the authority to stop it. Give the specialists a clear challenge role and protect it, without letting a review turn into ownership. Know which legal role each system puts the organization in, and treat anything that could change that role as a decision. And make ownership survive the reorganizations, departures and vendor updates that will certainly come.

Check yourself

1. If the vendor built the model, the vendor is accountable for how your organization uses it.MythFact
2. When accountability is not designed, it tends to fall on the person nearest the system.MythFact
3. A risk or legal team that reviews an AI system becomes its owner.MythFact
4. An owner who cannot pause the system without three approvals still meets the test of a real owner.MythFact
5. Fine-tuning a supplier’s high-risk system and releasing it under your own brand can make you its provider under the EU AI Act.MythFact
6. The revised 2026 US model risk guidance for banks covers generative and agentic AI.MythFact

Reflection: who would be blamed?

What comes next

This chapter placed the roles one system at a time: an owner, the challengers around the owner and the assurance behind them. An organization with dozens or hundreds of AI systems also needs a structure that connects those roles: what is decided centrally, what is decided in the business, and how decisions are routed between them. The next chapter, The AI Governance Operating Model, builds that structure.