How Generative AI Works

On the morning of 22 May 2023, a photograph of black smoke rising beside the Pentagon spread across Twitter. A verified account posing as a Bloomberg news feed shared it, and so did accounts with hundreds of thousands of followers¹. Just after 10 a.m. in New York, the S&P 500 slipped about 0.3 percent to its low for the day. Within minutes the image was exposed as a fake and the index recovered².

No camera had taken that picture, and no archive had stored it. It was widely judged to be the work of an image generator: a model that had learned what buildings, smoke and news photos tend to look like, and composed a new picture that matched a description. Nothing in the image was retrieved. All of it was made.

The same is true of every paragraph a chat assistant writes. Leaders who understand how that composing works can answer three questions their organizations face every week: where does the output come from, why does the same request give a different answer each time, and why does wrong output look exactly as convincing as right output?

Composed, not retrieved

A generative model produces new content in response to an instruction and whatever context the application supplies. What it holds is not a library of finished answers but patterns, learned in training, about how text, images, sound and code tend to fit together. At the moment of a request it uses those patterns to build an output through a long run of small choices, each one a matter of probability.

This is what separates generative AI from the predictive systems described in The Major Types of AI. A predictive model returns a judgment about its input: a risk score, a category, a forecast. Its output is small, and the world eventually shows whether it was right. A generative model returns something new: a letter, a summary, an image, a block of code. There are many acceptable versions and no single correct one. The practical consequence is about checking. A score can be measured against outcomes. A composition has to be reviewed.

Two pairs of words are worth keeping apart from here on. Composed is not retrieved: the output was never stored anywhere, even when parts of it closely resemble something that was. And fluent is not verified: the model’s skill is producing output that fits the patterns, not checking it against the world.

Two ways to build an output

Modern generative systems compose in one of two main ways, and the difference explains much of their behavior.

Text is built from left to right. As Large Language Models showed, a language model predicts the next small piece of text, a token, adds it, and predicts again, so that a paragraph is hundreds of choices made in sequence³. Each choice is shaped by everything before it, including the choices the model has already made. An unlucky early choice therefore shapes every token after it.

Many image and video generators work differently. They use a method called diffusion, set out in its modern form by Jonathan Ho, Ajay Jain and Pieter Abbeel in 2020⁴. In training, the model is shown images that have been progressively buried under random noise, and it learns to remove a little noise at a time. To generate, the process runs in reverse. It starts from pure static and, over many steps, removes noise in a direction that matches the prompt: first the rough layout, then shapes and light, then fine detail. The whole picture is worked on at once rather than piece by piece.

The original method used a thousand refinement steps for every image, which made it slow and expensive⁴. Later techniques produced images of similar quality 10 to 50 times faster⁵, which is a large part of why image generation became cheap enough for everyday use. Video applies the same idea across space and time. One leading developer describes its video model as a diffusion model that takes noisy patches of video and learns to predict the clean ones⁶. Newer systems increasingly blend the two approaches, so treat the table as the mental model rather than a rule about any product.

Every step is a weighted draw

Return to text and zoom into a single choice. Suppose an order assistant has written: “Your order is scheduled to arrive on”. The model does not know the arrival date. It assigns a probability to every possible next token.

Two things follow from an illustrative chart like this one. First, the favorite is only the most likely continuation given the patterns. It is not the arrival date, which lives in the order system. Second, the system then has to choose, and in most applications it does not simply take the top bar. It makes a weighted draw.

There is a good reason for that. Ari Holtzman and colleagues showed that always choosing the most likely text “leads to text that is bland and strangely repetitive”; models that pick only the favorite tend to loop. Their remedy, now widely used, is to draw from the likely core of the distribution and ignore the long tail of improbable options⁷. The result reads far better. The price is that the same request, on the same model, with the same settings, can come back in different words or with different content.

Even with sampling turned down to its most predictable setting, outputs can still differ between runs, because of how servers batch requests together. AI vs Automation showed one test in which 1,000 identical requests gave 80 different answers, until engineers changed the serving software⁸. Variation is a property of how these systems are built and run. It can be reduced. It should not be a surprise.

Match the settings to the task

The most common control is a setting called temperature. Lower values concentrate the draws on the likeliest options, so output is more conservative and repeatable. Higher values spread them out, so output is more varied and sometimes more surprising. Image generators have an equivalent: the random starting noise, often called a seed. Keep the seed and the settings fixed and the image can usually be reproduced; change the seed and you get a new image.

The business rule is simple: the setting should match the task. Pulling fields from an invoice wants predictable output in a strict format. Generating fifty names for a new service wants variety. Teams get into trouble when one configuration is used for everything, or when a low temperature is treated as a guarantee. It is not. A lower setting makes the model say its likeliest answer more consistently. If the likeliest answer is wrong, it will now be wrong consistently. Where the words must be identical every time, such as a legal disclosure or a safety instruction, the reliable design is a fixed template that the model does not rewrite.

From noise to picture, and why it looks real

Diffusion explains a property of generated images and video that executives need to understand: they are refined toward what looks typical, not toward what exists. Each step makes the picture more consistent with the patterns the model learned and with the prompt. Nothing in the process consults a map, a floor plan or a photo archive. That is why a generator can produce a convincing photograph of an event that never happened, as it did beside the Pentagon, and why it can render a product, a building or a face in confident detail that matches nothing real.

The same mechanism produces the characteristic failures. Physical rules are learned only as visual habits, so they break in the details. The developer’s own report on that video model noted that it “does not accurately model the physics of many basic interactions, like glass shattering”⁶. Hands with extra fingers, wheels that do not turn and lettering that is almost words are the same phenomenon: locally plausible, globally wrong.

Because generated media can be mistaken for real records, the law now treats it as something to be disclosed.

Fluent is not verified

For text, the equivalent of the convincing fake photograph is the confident wrong sentence. A model produces the likeliest continuation, so an invented figure, deadline or citation arrives in the same polished prose as a correct one. Researchers call fluent but false or unsupported output hallucination, and a widely cited survey of the field treats it as a general property of language generation, not a defect of one product⁹. Large Language Models showed why training and benchmarks reward confident guessing. The question here is why people so readily believe the result.

The answer is that fluency is what readers use as a shortcut for credibility, and generation is very good at fluency. In a study published in Science Advances, Giovanni Spitale and colleagues found that people could not tell tweets written by an AI language model from tweets written by people. The model’s accurate tweets were easier to understand than the human ones, and its false tweets were more compelling¹⁰.

The remedy is design, not exhortation. In a well-built application the facts come from authoritative sources at the moment of the request, such as the order system, the policy store or a deterministic calculation, and the model writes the words around them. Supplying documents at question time is called retrieval-augmented generation¹¹, and RAG and Enterprise Knowledge covers how it works. Grounding reduces invented facts. It does not remove the need to check what matters.

Story: seventy thousand clips for one commercial

In 1995 Coca-Cola first aired “Holidays Are Coming”, its commercial of illuminated red trucks rolling through snowy towns, and it became one of the brand’s best-known films. In November 2024 the company released two AI-generated reimaginations of it, made with specialist AI studios. The music was performed by real artists¹². Marketing Week reported that the 2024 version was produced for about a tenth of the cost of a traditional shoot¹³. Critics were less impressed by what they saw: stiff, uncanny human faces, and truck wheels that turned in odd directions or not at all¹⁴.

For the 2025 version, the Wall Street Journal reported, five AI specialists generated and refined more than 70,000 video clips over about a month, and around 100 people were involved in the campaign as a whole¹⁴. That is roughly 470 clips per specialist for every day of that month, for films that use a small fraction of them. The new films replaced human faces with animals. Viewers still spotted Santa’s fingers briefly distorting into impossible shapes¹⁴. Coca-Cola’s head of generative AI put the division of labor plainly: creative direction “has and always will be human-led”, and AI is the engine for execution and production¹³.

Read the case through the mechanics of this chapter. Every clip was a draw: the same prompt, run again, gave a different truck, a different snowfall, a different polar bear, so the team generated tens of thousands and kept the few that worked. Every clip was plausible rather than correct: the wheels and fingers failed because the model had learned how trucks and hands usually look, not how they work. And the cost moved. Coca-Cola said its 2024 remake cost about a tenth of a traditional shoot; the 2025 clip counts show where the effort went instead. The work that remained was deciding what the brief should be, which samples were good enough, what had to be repaired by hand, and whether the result was worthy of the brand. Whether the films succeeded is a matter of taste and of reaction, which was divided. What the case settles is where generative AI’s costs go: away from making, toward choosing and checking.

What this means for leaders

The mechanics translate into a short set of design decisions. Start by asking of any proposed use whether it needs a composition or a fact. Drafting, rewriting, summarizing, translating and visual concepts are compositions, and generation is well suited to them, although the gain in any one process has to be measured there. Dates, prices, balances, eligibility and anything a regulator or customer will rely on are facts, and they should come from systems of record and deterministic software, with the model writing the words around them.

Then decide how much variation the task should have, and test for it before launch. Run the same requests many times and look at the spread. Where the answer must be identical, use templates and fixed content, not a lower setting and hope. Where variation is the point, plan for selection: someone has to choose among the samples, and that someone is part of the cost.

Finally, measure cost per useful outcome rather than cost per output. Suppose, with illustrative figures, that 100 drafts cost 1,000 in total and 70 of them are usable. The figure that matters is about 14.29 per usable draft, not 10 per draft, plus the time of the person who sorted them. As the commercial showed, generation makes the first draft nearly free and makes judgment the scarce input.

Check yourself

1. A generative AI system finds a stored answer and sends it back.MythFact
2. The same prompt with the same settings can produce a different answer.MythFact
3. Turning the temperature down makes an answer more likely to be correct.MythFact
4. Many image and video generators start from random noise and refine it toward the prompt.MythFact
5. A fluent, confident answer is more likely to have been checked.MythFact
6. Generation makes producing content cheap, so the main cost shifts to choosing and checking it.MythFact

Reflection: compositions and facts

What comes next

You now have the mechanics: generative AI composes, it draws on chance at every step, and its fluency says nothing about its accuracy. Of everything that shapes the output, the instruction and the context you supply are where you have the most control. The next chapter, Prompting and Context Engineering — Executive Mental Model, explains how to communicate intent to these systems, and why prompting has become much more than writing a good sentence.