Deceit

The argument that produced one of the most consequential ideas in modern artificial intelligence happened in a bar, over beer, among people who were mostly trying to talk a friend out of a bad plan.

It was the spring of 2014, in Montreal, at a brewpub called Les Trois Brasseurs where the city’s machine-learning graduate students drank. The occasion was a going-away party for one of them. Ian Goodfellow, a doctoral student in Yoshua Bengio’s lab, was there with a clutch of colleagues, and the conversation had drifted, as it tended to among that crowd, toward an unsolved problem. They wanted a computer to generate photographs. The field already had machines that could retrieve an image, classify it, label what was in it. This was a different ambition: to make new ones, from nothing, that looked like the real thing.

The conversation had a specific shape. The friends at the table had a scheme. It involved a statistical accounting of everything that makes a photo look like a photo: the way light falls, the texture of skin, the geometry of a face. If you could enumerate enough of those properties and force a program to satisfy all of them at once, the reasoning went, you would coax it into producing something convincing. It was the kind of plan that sounds reasonable until you try to write down the list, at which point you discover the list has no end. Goodfellow thought it would never work. He said so. The number of things that make an image look right is effectively infinite, and any one you forget becomes the tell that gives the whole thing away.

This was, in its way, a familiar standoff. For decades the people who wanted machines to produce believable output had been trying to specify, in advance, what believable meant. The history of the field was a graveyard of such specifications. Every rule you wrote down was a rule the world would eventually violate. The thing about reality is that it does not come with a checklist.

What Goodfellow proposed instead, half-arguing, was to stop writing the checklist and let a second program write it for you. Build two neural networks, he said. The first one generates images. It starts out terrible, producing noise. The second one is a critic, a detector, whose only job is to look at an image and decide whether it came from the real world or from the first network. Then you set them against each other. The generator tries to fool the detector. The detector tries not to be fooled. Each one’s failures become the other one’s lessons. You never tell the generator what a real photograph looks like. You only ever tell it whether it got caught. Over enough rounds, the argument runs, the generator gets so good at not getting caught that its forgeries become indistinguishable from the genuine article, and at that point, by definition, it is producing photographs.

His friends were unconvinced. It was too simple, and worse, it was the kind of idea that sounds clean at a bar and falls apart at a keyboard. Training one neural network was hard enough. Two networks locked in mutual opposition, each constantly changing the ground the other stood on, sounded like a recipe for a system that would never settle, just chase its own tail forever. Goodfellow had had a few beers. He went home that night convinced his friends were wrong and equally convinced there was only one way to prove it.

He coded it. By his own account the whole thing went together in a single evening, not because he was a genius working in a fever, but because the idea, once you accepted it, required almost nothing he did not already have lying around. The pieces were standard. A generator network, a discriminator network, the ordinary machinery of backpropagation that the field had spent thirty years perfecting. The novelty lay entirely in the wiring, in pointing two familiar tools at each other. He ran it. It more or less worked the first time, which is the kind of thing that almost never happens and that he would spend years gently insisting was luck as much as anything. The images it produced were small and crude. But they were not retrieved from anywhere. The machine had made them.

He called the idea generative adversarial networks. The adjective in the middle was the whole point. The two networks were adversaries. The training signal was not a lesson handed down from a human supervisor; it was a fight.

To explain it, Goodfellow reached for a metaphor he would repeat for the rest of the decade, until it became the standard way the field described the thing. Imagine a counterfeiter and a cop. The counterfeiter wants to print fake money good enough to pass. The cop wants to catch every fake. In the beginning the counterfeiter is hopeless and the cop catches everything, which teaches the counterfeiter exactly what to fix. As the counterfeiter improves, the cop has to get sharper to keep up, which forces the counterfeiter to get sharper still. Neither side is ever allowed to rest, because the moment one of them stops improving, the other one wins. If the process runs to its conclusion, you arrive at a counterfeiter whose bills are perfect, perfect meaning that even the best cop in the world can do no better than flip a coin.

There is something worth pausing on in that metaphor, because the field mostly did not pause on it at the time. The entire apparatus is organized around deception. The generator’s whole purpose is to lie convincingly. Its measure of success is the moment a competent observer can no longer tell its lies from the truth. The mathematics underneath was respectable. Goodfellow framed it as a minimax game, the kind of two-player zero-sum contest that economists and game theorists had studied for half a century, with a well-defined equilibrium the two networks were chasing. But strip away the equations and what you had was a machine engineered, from the ground up, to produce falsehoods indistinguishable from fact. That was not a side effect. That was the objective function.

The paper, “Generative Adversarial Nets,” went to the Neural Information Processing Systems conference in 2014. Goodfellow was the first author; the list ran through a roster of the Montreal lab, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and ended with Bengio. Tucked into the acknowledgments was a thank-you to Les Trois Brasseurs, the bar where the argument had happened, which is the kind of joke researchers permit themselves exactly once in a career. The images in the paper were not impressive to look at. Tiny handwritten digits. Blurry little faces and grainy snapshots, each a smear of color in which you could just barely make out the suggestion of a cheekbone or a window. If you had shown those pictures to a stranger and said this is the future of synthetic media, the stranger would have laughed. They looked like photographs developed in a flood.

But the people who understood what they were looking at did not laugh. They understood that the resolution was a detail and the mechanism was the breakthrough. A blurry face today, with a better network and more data and a few years of engineering, was a flawless face tomorrow. The trajectory was the thing. Yann LeCun, who had spent his career being right about which unglamorous ideas would matter, called adversarial training the most interesting idea in the last ten years in machine learning. Coming from a man not given to praising his colleagues’ work for sport, that was close to a coronation.

What made the idea spread so fast was the same thing that had made it possible to code in a night. It demanded nothing exotic. Any lab with the standard toolkit could try it, and almost immediately everyone did. Within a year a pair of researchers had figured out how to build the two networks out of the convolutional architectures that worked so well on images, and the faces got sharper. The technique acquired the texture of a movement: a steady cadence of papers, each one taking the blurry output of the last and sanding it down, a community racing to make the lie a little more perfect. The acronym, GAN, entered the language of the field the way a few ideas a decade do, as a thing you no longer had to explain.

The progress over the next five years was almost grotesque in its speed. The faces went from thumbnail smudges to portraits you would not look at twice on a passport. By 2018 a team at the chip company Nvidia had built a version that produced human faces at high resolution, rendered with pores and stray hairs and the faint asymmetries that make a real face real, and the people in them did not exist. They had never been born. There was no photograph behind the photograph. The following year a software engineer used the same system to build a website that did nothing but generate a new one of these phantoms every time you refreshed the page: a fresh stranger, smiling or neutral, conjured and discarded in the time it took to load. You could sit and refresh it for an hour and meet a thousand people, none of whom had ever drawn breath, each of whom your eyes accepted without hesitation as a person.

The machine had won the counterfeiter’s game. The cop in the metaphor was supposed to be another neural network. But the cop that mattered, in the end, was the human eye, and the human eye had been beaten.

It did not take long for someone to notice that a technology designed to produce convincing fakes might have uses beyond academic curiosity. In late 2017 a user on the message board Reddit, posting under the handle “deepfakes,” began sharing videos in which the faces of famous actresses had been grafted onto the bodies of performers in pornography. The seams were visible if you looked, but you had to look. A journalist named Samantha Cole, writing for the Vice site Motherboard, reported on it on December 11, 2017, and the handle became a common noun almost overnight. Deepfake. Within weeks there were tools that let anyone with a decent computer and a folder of someone’s photographs do the same thing, no machine-learning degree required. The barrier that had always protected people from having their likeness puppeted, the simple fact that faking a video convincingly was hard and expensive, had quietly dissolved while everyone was admiring the pretty faces.

The architecture under the hood was not always literally a GAN, and the people who built specific deepfake tools borrowed from several techniques. But the conceptual permission slip was Goodfellow’s. He had shown the field, and through it the world, that you could train a machine to manufacture a convincing falsehood without ever teaching it what truth looked like, simply by rewarding it every time it slipped one past a critic. Once that was possible for tiny grayscale digits, it was a matter of engineering before it was possible for a sitting head of state appearing to say words he had never said.

Goodfellow himself was not naive about any of this, and to his credit he did not pretend to be. He had been clear from early on that the same equilibrium that made the generator a flawless forger also implied a permanent contest. The cop never gets to retire. If you could build a network that detected fakes, you could turn around and use it to train a better faker, which would then require a better detector, and so on, the arms race that lived inside every GAN now scaled up to the level of society. He talked, in the years after, about a coming world in which the default assumption about any image or recording might have to flip, where the burden of proof would shift. The question would stop being whether you could prove a thing was fake. It would become whether you could prove it was real. He did not present this as a prophecy he relished. He presented it as the structure of the thing he had built.

There is a temptation, looking back, to cast the whole episode as a fall, a clever young man who loosed something he should have left in the bottle. That reading is too tidy, and it misreads what actually happened in the bar. Goodfellow did not set out to build a lie machine. He set out to solve a problem that had defeated the field for decades, how to make a computer generate something genuinely new, and the solution, the one that finally worked after all the checklists had failed, ran on deception as its engine. The deceit was not a moral choice. It was a mathematical one. The only way anyone had found to teach a machine to create was to teach it to fool, and the only way to teach it to fool was to put a skeptic in the room and let the two of them go to war.

That was the discovery hiding inside the party argument, and it was darker than the cheerful counterfeiter story let on. For thirty years the people in this book had been trying to make machines perceive: to recognize a cat, transcribe a sentence, name a face. Perception is a kind of agreement with reality. You are right when your answer matches the world. Generation inverted that contract. A generative model is rewarded not for agreeing with the world but for being mistaken for it, and the whole field had just been handed a recipe for building such machines at industrial scale. The same mathematics that let a computer dream up a face that had never existed let it dream up evidence of things that had never happened.

Goodfellow would leave Montreal and become one of the most sought-after researchers alive, passing through Google and OpenAI and eventually Apple, his name attached forever to the four letters he had coined over beer. The technique would seed a thousand others. And the broader machinery of generation that he had helped legitimize, the idea that you could train a model to produce convincing new artifacts rather than merely sort existing ones, would, within a decade, escape the narrow world of images entirely and become the engine of the chatbots that put this whole story on the front page.

But that was a decade off. The nearer consequence was simpler and more political. A technology that could fabricate a convincing face, or a convincing video of a head of state, was not going to stay in a Montreal lab, and it was not going to stay in any one country. The method was published, which meant it belonged to everyone, and the question of who, exactly, was racing to build this technology everywhere was about to become the largest one in the field. The board that the previous chapter had described as set was, in fact, about to grow a second half a world away.