Why Responsible AI Matters

In 2025 the University of Melbourne and KPMG published one of the largest studies of public attitudes to AI to date: more than 48,000 people in 47 countries. Two thirds of them used AI regularly. Fewer than half, 46 percent, said they were willing to trust AI systems. At work, the picture was stranger still. Among employees who use AI at work, 66 percent said they had relied on its output without evaluating it, and 56 percent said they had made mistakes in their work because of AI¹. The first two figures describe everyone surveyed; the last two describe only employees who use AI at work.

Read the numbers together and they describe two failures at once. More than half of people are not willing to trust AI, and many of those who do rely on it trust it more than they should. The researchers drew the conclusion that matters for leaders: in their statistical model, trust was a key driver of whether people accepted AI¹. That makes trust a business variable, not a public-relations one. It is the subject of this chapter, and the reason responsible AI belongs on an executive agenda.

The core idea

Responsible AI is the practice of designing and operating AI so that it creates value while protecting people, managing risk and keeping someone clearly accountable. It is not a promise of perfect systems. It is a commitment to systems whose risks are understood, controlled and owned.

The chain runs in both directions. Responsible design and operation earn trust from the people a system touches. Trust leads to adoption, meaning people use the system and rely on it to the right degree. Adoption is where the value comes from. Run the chain backward and you get the familiar failure: careless practice damages trust, low trust stalls adoption, and the business case never arrives, however good the model is.

The frameworks executives are most likely to meet say the same thing in their own language. The NIST AI Risk Management Framework describes trustworthy AI as valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed, and it stresses that these qualities involve trade-offs that people must decide on, not just engineer². The OECD AI Principles, updated in 2024, add that organizations that develop, deploy or operate AI should be held accountable for its proper functioning³. Neither framework treats trustworthiness as a feature of the model alone. It is a property of how the organization builds, uses and answers for it.

As What Is AI Governance? showed, governance is the system of decision rights, controls and oversight that makes AI decisions at scale. Responsible AI is the standard that system is trying to meet. The first defines how the organization decides; the second defines what a good decision looks like.

The law is the floor, not the question

Many leaders first hear “responsible AI” as a legal checklist. The law matters, and in Europe much of it is not optional: the GDPR already governs most AI that touches personal data, and the EU AI Act adds duties by risk. But compliance and responsibility ask different questions.

Compliance asks whether the organization meets its obligations. Responsible AI also asks whether a use is right for the people it affects, whether its outcomes are fair, whether it can be explained and whether someone can step in when it goes wrong. A system can pass every legal review and still lose the trust of the customers or employees it was built for. And law moves more slowly than practice. Many uses of AI fall into areas where the rules are general, untested or still arriving, and an organization that waits for the law to tell it what is acceptable will usually find out from its users first.

The gap also closes over time. Assessing the impact of risky processing before it starts was once voluntary good practice; since 2018 the GDPR has made it a legal duty for high-risk processing (Article 35)⁴. The AI Act is doing the same for AI, turning practices such as human oversight and logging into obligations for high-risk systems. An organization that treats responsibility as more than compliance is better placed when the floor rises, because it has already built to the higher standard.

This is why responsible AI cannot sit with the legal team alone. Legal owns the obligations. The questions of purpose, data use, fairness, ownership and oversight belong to the business that uses the system, with technology, data, security and risk as partners.

Informed trust, not maximum trust

Trust does not mean believing that AI is always right. Researchers in human factors worked this out long before generative AI. In a widely cited review, John Lee and Katrina See argued that what matters is not how much people trust an automated system but whether their trust is calibrated: matched to what the system can actually do. Overtrust leads to misuse, and distrust leads to disuse⁵.

Both ends are expensive. At the blind end, a confident error travels straight to a customer or into a decision. The KPMG finding that two thirds of employees who use AI at work rely on its output without checking it describes exactly this¹. How that over-reliance arises, and how to design oversight against it, was the subject of Operational and Workforce Risk and Human Oversight and AI Incidents.

At the other end, people refuse a system or quietly work around it, and the investment produces nothing. This is not only a matter of attitude. In a well-known series of experiments, people who watched a forecasting algorithm make a mistake lost confidence in it faster than in a human who made the same mistake, and chose the human, even though the algorithm was more accurate⁶. A follow-up study found a simple remedy: when people could adjust the algorithm’s forecasts, even slightly, they became far more willing to use it⁷.

That result is the practical heart of informed trust. People trust a system more when they understand what it does, know its limits, know who is accountable for it and can challenge or correct it. None of those depends on a better model. All of them depend on how the organization designs and runs the system around the model.

What irresponsible use costs

The cost of irresponsible use is rarely the cost of fixing one system. A European enforcement case about data on people at work shows how the bill grows.

In August 2021 Italy’s data protection authority fined Deliveroo’s Italian business 2.5 million euros over the system that managed the shifts of about 8,000 riders. The riders had not been given transparent information about how the algorithm worked, and the app collected far more data than it needed, including their location every 12 seconds. The regulator also ordered the company to fix the violations and to complete changes to its algorithms within months, and to check the algorithm’s results regularly for accuracy⁸.

The same logic has applied to data on workers more broadly: in 2020 the Hamburg data protection authority fined H&M about 35 million euros after team leaders recorded employees’ illnesses and family problems in notes that an IT error exposed⁹.

The fines are the part that makes the news. The larger costs sit underneath: rework done on a regulator’s timetable rather than the company’s, managers’ time, employees who now assume the worst about any new system, and, often, leadership restricting the wider program rather than the one system that failed. As The AI Risk Landscape showed with the Dutch childcare benefits case, the consequences can reach well beyond the organization itself. Trust is a business asset, and it is far cheaper to protect than to rebuild.

The logic works in the other direction too. An organization that has shown its people and customers how it uses AI, and has kept its word, has a lower cost of introducing the next system. That is a capability, and it compounds.

One good question per dimension

Responsible AI is usually described through a set of dimensions. An executive does not need the catalog. One good question for each of six is enough. Fairness: could outcomes differ unfairly between groups? Privacy: can we justify this data to the people it describes? Security: how could the system be attacked or misused? Transparency: do the people affected know AI is involved? Ownership: who answers for the outcome? Oversight: who can review, override or stop it?

Module 06 taught the risks behind four of them, in Bias and Fairness, Privacy and Confidential Data, Security and AI Attacks and Human Oversight and AI Incidents. Two deserve emphasis here. Transparency asks whether the people affected know that AI is involved and what role it plays; the Deliveroo case above began with a failure to tell people. Ownership asks who answers for the outcome, because “the system decided” is not an accountability model. The depth of the answers should match the stakes: a tool that drafts internal meeting notes needs lighter answers than one that affects someone’s job, pay or credit. Turning these questions into a principle set the organization can hold people to is the work of the next chapter.

Designed in, not bolted on

Responsible AI is cheapest at the start. Choices made in design, such as which data to use, what the output is for, who sees it and who can override it, are hard to reverse once a system is live, has users and is shaping decisions.

The GDPR already makes this expectation explicit for personal data. It requires data protection by design and by default (Article 25), and an impact assessment before processing that is likely to create a high risk to people (Article 35), not after⁴. Deliveroo’s case shows the alternative: the transparency and data limits that could have been designed in were instead imposed, with deadlines, by a regulator⁸.

Designing responsibility in does not end at launch. AI behavior changes as data, models, users and providers change, so monitoring has to continue, and buying a system from a vendor moves none of the responsibility. How each stage is governed is the subject of AI Lifecycle Governance. Nor does designing in mean slowing down. Safe experiments with approved tools and non-sensitive data can move quickly; the questions get harder as a use moves toward customers, employees and consequential decisions.

Story: the interview scores that read faces

This is a documented case. The company in it sells AI to employers, so the lesson applies as much to an organization that buys such a system as to one that builds it.

By 2019 HireVue, a US hiring-technology company, ran video interviews for more than 700 employers¹⁰. Candidates recorded answers to set questions on camera, and the company’s algorithms scored the recordings. Employers got what they paid for: a fast, consistent first screen of large applicant pools. By early 2021 the company reported that its platform had hosted more than 19 million video interviews¹¹.

Part of the score came from the candidate’s face. In November 2019 the Electronic Privacy Information Center (EPIC), a US privacy group, asked the Federal Trade Commission to investigate. Its complaint said that HireVue collected facial expressions and movements, along with voice and words, to assess traits such as cognitive ability and emotional intelligence, that it used secret algorithms, and that these were unfair and deceptive practices¹⁰. Whatever the merits of each allegation, the shape of the problem was plain. People applying for jobs were being scored on signals they had no reason to think counted, by a method they could not examine. The product created measurable value for its customers. It was also exactly the kind of system that loses the trust of the people it judges.

The company changed the product rather than defend it. HireVue stopped using facial analysis in its assessments in March 2020 and announced the change in January 2021, together with the results of an outside audit of its assessments for early-career hiring [@shrm-hirevue-2021; @fortune-hirevue-2021].

The reason the company gave is the most useful part of the case. Its own research had found that the visual data contributed little, about 0.25 percent of a model’s predictive power in most cases, as language analysis improved¹². Its chief executive said that, set against the concern it caused, the face analysis “wasn’t worth the incremental value”¹². The audit did not close every question: it recommended looking further at whether candidates’ accents affected results¹². The company kept scoring what candidates said, and kept selling the product¹¹.

Notice what changed and what did not. The service still screened candidates at scale. What changed was the data, the openness to outside review and, with them, the case the company could make to the people it scored. The data that was hardest to justify turned out to be the data the system least needed. That is one case, not a law of nature, but it is why the question “can we justify this data to the people it describes?” is worth asking early. And an employer that used the tool was the one making the hiring decisions. Under the AI Act it would be a deployer, with duties of its own; as the previous section said, buying a system moves none of the responsibility. Sometimes the responsible answer is to redesign, reduce scope, add controls or not deploy at all. Here it was to reduce scope and open the system to review, and the value survived.

What this means for leaders

The first lesson is that value is not a defense. A system can deliver measurable results and still be irresponsible, and the results do not protect it once the people affected find out how it works. The second is that trust should be managed as deliberately as cost: watched through signals such as adoption, overrides, complaints and requests for a human, and protected before launch rather than repaired after. The third is that the law is the floor, not the standard, and building to a higher one is what lets a valuable system keep its license to scale. And the fourth is the distinction this module rests on: responsible AI defines what good looks like, and governance is how the organization makes sure it happens.

Check yourself

1. Responsible AI means AI that makes no mistakes.MythFact
2. If an AI system is legal, it is responsible.MythFact
3. The goal is to make people trust AI as much as possible.MythFact
4. People are more willing to use an imperfect algorithm when they can adjust its output.MythFact
5. If no one complains, the system must be responsible.MythFact
6. An AI system can create measurable business value and still be irresponsible.MythFact

What comes next

Responsible AI gives an organization its reason and a shared sense of what good looks like. A shared sense is not enough for thousands of people making AI decisions every week; they need a short, explicit set of principles they can apply and be held to. The next chapter, AI Governance Principles, sets out those principles and how they become expectations teams can act on.