Understanding AI Total Cost of Ownership

Suppose this line sits in a business case on your desk, waiting for approval. Annual AI cost: 250,000. The note underneath reads: vendor quote, model and API usage. The case is tidy, the number is real, and the vendor is reputable. The question is what the number leaves out.

Finance leaders have good reason to ask. In October 2024 Gartner told its CIO clients that an organization that does not understand how its generative AI costs scale could make a 500 to 1,000 percent error in its cost calculations. More than 90 percent of the 300-plus CIOs it surveyed that summer said managing cost limited their ability to get value from AI¹. Treat the estimate as a forecast, not a measurement. It points at the gap this chapter is about: the cost in the case is not the cost of owning the system.

The core idea

The idea of total cost of ownership is older than most AI programs. It grew out of Gartner’s 1980s research on personal computers, whose training, support and downtime sat in other departments’ budgets, and it defines total cost of ownership as “the holistic view of costs across enterprise boundaries over time”². Two words in that definition do the work: boundaries, because the costs sit in many budgets, and time, because they arrive across the whole life.

AI revives the old problem in a sharper form, and this module uses one definition throughout. Total cost of ownership is build plus run plus operate plus change, over the system’s whole life. Build is what it costs to get to the first real transaction. Run is what the system consumes as it works. Operate is the people and processes that keep it trustworthy. Change is what it costs to keep it current and, one day, to leave. The Economics of AI showed how these costs behave as use grows; this chapter shows how to count them.

Four families, one boundary

Build covers the one-time work before launch: discovery, engineering, integration with the systems the AI must work with, data preparation, building a test set to judge quality, security review, and training the people who will use it. Run covers what the system consumes every month: model and API usage, compute, storage, data pipelines and a share of any shared AI platform. Most of it grows with volume. Operate covers the people and routines that keep the system working and safe: checking outputs, handling exceptions, monitoring quality, security operations, user support and governance. Change covers updates to models and prompts, re-testing after each update, migration to a new provider and, at the end, retirement.

The six cost layers from The Economics of AI map onto these families: model and compute are mostly run, data and integration are build and run, operations and governance are operate. Change is the family that layer maps tend to leave out, because it touches every layer at once.

A definition needs a boundary, or every corporate expense ends up with an AI label. The test is simple. A cost belongs in TCO if it is necessary to build, run, operate or change the capability, and if it would not exist without it. The data platform the company runs anyway does not belong; the extra pipelines and storage this system needs do. Staff time counts even when the salaries are already paid, because that time is no longer available for other work. The scarce attention of senior engineers and leaders is an opportunity cost: it usually stays outside the accounting figure, but it belongs next to it when two investments compete for the same people.

Gartner’s definition adds a second rule that matters as much as the first. “Total” means nothing that should be included is left out, “but there must be no double-counting”². Completeness and consistency are the same discipline seen from two sides.

Below the waterline

Some AI costs arrive as invoices: the model bill and the cloud bill. They are easy to find and easy to argue about, which is why business cases are built around them. Most of the rest never arrive as an AI invoice. They sit in salaries, in the platform team’s budget, in the support desk and in the security function’s backlog.

People are often the largest hidden line, as the Canada Revenue Agency case below shows. The Economics of AI, which opens this module, set the human review rate as a dial; for TCO the point is narrower. Whatever setting is chosen, the people who check outputs and handle exceptions belong in operate, and so do the support staff who answer users’ questions about the new system. If they are missing from the case, the case is not complete.

How costs behave

Counting the families is half the work. The other half is knowing how each cost moves, because a forecast that treats them all the same will be wrong in a predictable direction. Management accounting has four patterns, and AI uses all of them³.

One-time costs are mostly build: integration, the test set, training. Fixed costs recur at the same level each period: platform fees, licenses, a support rota. Variable costs grow with each document, call or query, which is why most of run and part of operate scale with volume. The fourth pattern, the step cost, is the one that surprises.

Step costs: the jump nobody forecasts

A step cost is fixed within a range of activity and then jumps to a new level. The range in which the cost stays flat is called the relevant range, and a fixed-cost forecast is only valid inside it.

AI systems are full of steps. Reserved computing capacity is bought in blocks. A review team handles a certain number of exceptions a person before the next hire. A support desk can absorb a new system’s questions until it cannot. Vendor contracts move between volume tiers. Each of these stays flat while use grows, and then jumps.

Suppose exceptions from an AI system are checked by reviewers who each cost 80,000 a year, or 20,000 a quarter. Three reviewers cost 60,000 a quarter. As volume grows, the spreadsheet adds 5,000 a quarter in a smooth line. Reality stays at 60,000 for three quarters, jumps to 80,000 when the fourth reviewer is hired, and to 100,000 when the fifth arrives. By the eighth quarter the forecast is close to the actual, but in between it was too pessimistic in some quarters and too optimistic in others, and it never told anyone when to hire.

The management question is not “what is the average cost per unit?” but “where is the next step, and what triggers it?” A case that names its steps, with the volume at which each one arrives, can be managed. A case with a smooth line will be surprised. Model, Compute and Infrastructure Costs looks at capacity and utilization, the largest source of steps in run.

Over the whole life

TCO also spans time, and counting only the first year is a common way to understate it. Build happens before the first real transaction. Run and operate then continue every year the system lives. Change arrives on a schedule that the buyer does not control.

The months in the figure are illustrative. Hosted models are retired on the provider’s timetable. On one major cloud platform, the published standard lifecycle retires a generally available model version 18 months after launch, models from several other publishers follow a 12-month lifecycle, and retirement dates cannot be extended⁴. A system adopted a few months into a model’s life therefore faces its first forced change within about a year and a half, and a second before the end of year three. Each change means evaluating the replacement on the system’s own test set, adjusting prompts and workflows, re-testing, and sometimes revalidating with users. A move to a new provider adds re-engineering, data movement and a parallel run, when the organization pays for two systems at once.

Retirement is not too far away to matter. Removing access, handling or deleting data, exiting contracts and archiving records all cost money, and the size of that bill is decided when the architecture and contract are chosen. Model and Third-Party Risk treats lock-in as a risk and AI Lifecycle Governance treats controlled retirement as a duty; here the point is that both have a price, and the price belongs in the case. State the life you are costing, three or five years, and count every family across it.

Shared costs: once, never twice

Many AI systems share a platform: one data team, one evaluation toolkit, one model gateway, one security review process. Someone has to decide how that cost reaches each system’s business case. The FinOps Foundation calls this allocation: assigning and sharing cost and usage so that teams are accountable for what they consume. Its framework names the usual methods: fixed shares, proportional shares or a proxy metric, or deliberately holding a cost centrally as an “informed ignore”⁵.

There are two ways to get it wrong. If nobody allocates the shared platform, each business unit sees only its own line, concludes that AI is cheap, and the central team absorbs the rest until its budget breaks. If both the platform team and the application team charge the same infrastructure to the case, a sound project looks too expensive to approve. Gartner’s rule against double-counting applies exactly here².

The remedy is one agreed rule, written down and applied to every case. Usage-based allocation suits variable costs such as model usage and compute, because each system pays in proportion to what it consumes. Fixed shared costs, such as a platform team, can be shared by a fixed key or held centrally, as long as the choice is visible. The method matters less than consistency: publish one, and do not let each team invent its own. Turning allocation into showback and chargeback is an operating practice that Cost Optimization and AI FinOps covers later in this module⁶.

Story: what Canada’s tax chatbot cost

In February 2020 the Canada Revenue Agency introduced Charlie, a chatbot on its website that answers general questions about taxes and benefits⁷. It cannot deal with a taxpayer’s own account, but the Auditor General noted that it helps reduce pressure on the agency’s call centres⁸. By late 2025 it had been used in more than 7 million conversations and asked more than 18 million questions⁷.

In October 2025 a member of Parliament asked what Charlie had cost to develop, operate and maintain, broken down by year and by type of expense⁹. The agency’s answer, tabled in December, put the spending since the 2018–19 fiscal year at more than 18 million Canadian dollars. About 13.7 million of it was salaries. About 3.2 million went to IT consultants. The figures did not include employee benefits or travel⁷.

Read the breakdown through the four families. The part that looks most like an AI invoice, the outside consultants, was under a fifth of the reported spend. About three quarters was the agency’s own people, the build and operate work that never arrives labeled as AI. That is the waterline at work: had anyone costed Charlie from its technology contracts, they would have missed most of the bill.

The boundary test applies too. Salaries were counted; the benefits that go with them were not. Even the number given to Parliament is therefore not the whole cost of ownership, and the missing piece is the people family, already the largest. A complete count would load each salary with its benefits, once.

Then came change. In its response to the Auditor General, the agency said that from November 2025 a generative AI chatbot in beta would answer more questions on its website⁸. A system launched in 2020 was being rebuilt on a new kind of model within six years. Whatever that switch costs belongs in the change family, and a three- or five-year case for any chatbot should expect a change of the same kind.

Quality is part of the operate bill as well. When the Auditor General’s team tested Charlie with six tax questions, it answered two accurately, while other public AI tools answered five⁸. Keeping a system accurate, and knowing whether it is, takes people and testing, year after year. The agency’s own figures make the chapter’s case better than any composite could: the technology line was the small one, the people line was the large one, and the change line arrived on schedule.

What this means for leaders

The discipline is short to state. Insist on one definition of total cost of ownership, the four families over a stated life, and refuse cases that report a vendor quote as the cost. Apply the boundary test: a cost belongs if the capability needs it and it would not exist otherwise. Ask where the steps are and what volume triggers each one. Put the provider’s model lifecycle into the change family, because change is scheduled, not hypothetical. And require one published allocation rule, so shared costs are neither hidden nor counted twice.

TCO describes cost only. Approval also needs value, risk and strategic fit, and AI ROI and Value Realization in Module 03 sets the full cost against value to calculate return.

Check yourself

1. The vendor’s quote for model usage is a fair estimate of an AI system’s annual cost.MythFact
2. Staff time spent checking AI outputs belongs in total cost of ownership even when the salaries are already paid.MythFact
3. If usage grows steadily, a straight-line cost forecast is accurate enough.MythFact
4. Model change is a cost a hosted AI system can expect within its first two years.MythFact
5. Charging a shared platform to both the platform team and the application team makes the case more prudent.MythFact
6. Total cost of ownership is enough to approve an AI investment.MythFact

Reflection: find the missing family

What comes next

This chapter drew the whole footprint: four families, counted over the whole life, with steps and shared costs made visible. The next chapter zooms into the most visible and most volatile part of the run family, and asks why two systems on the same model can have very different bills. That is the subject of Model, Compute and Infrastructure Costs.