The Man Who Didn't Sit Down

The conference room at Harrah’s was too small for the crowd it drew. People stood three deep along the back wall and spilled into the doorway, the way they do when a talk has stopped being a talk and become an event. It was December 2012, and the annual meeting of the neural information processing community had taken over two casino hotels straddling the Nevada line at Lake Tahoe, where slot machines chimed in the lobbies below and the air smelled of carpet cleaner and snow. Inside the room, the subject was a piece of software that had, two months earlier, looked at a million photographs and learned to tell a leopard from a lifeboat better than anything anyone had ever built.

Geoffrey Hinton was somewhere near the back. He was sixty-four years old, British, the son and grandson and great-great-grandson of accomplished people, and he was standing because he could not do otherwise. A back injury that traced to his youth had left sitting reliably painful, and at some point he had simply stopped. He worked at a standing desk. He ate his meals kneeling on a foam pad. When he had to travel any real distance by car, he lay flat across the back seat. He had built a life around the vertical, and he carried it with him, a tall man in a rumpled jacket who never took a chair.

What the people in that room knew, and what most of the technology industry did not yet know, was that the software being discussed had won a contest. The contest was called ImageNet, and it was a yearly test of how well a computer program could name the contents of a photograph: this is a dog, this is a mushroom, this is a container ship. For years the scores had crept upward by fractions, the way scores do in a mature field where everyone uses roughly the same methods and competes at the margins. Then a program from the University of Toronto had arrived and cut the error rate by something close to half. This was not an incremental gain. It was a different order of result, the kind that makes the people who understand it go quiet and then start talking all at once.

The program had a plain name, drawn from the first name of the student who had written most of it. He was not in the front of the room basking in any of this. Alex Krizhevsky disliked attention and was bad at hiding it, and the machine he had used to produce the result that was reordering the field was not a supercomputer or a server farm but two gaming graphics cards he had installed in a computer in his bedroom at his parents’ house. He had spent months coaxing the cards to do work they were never designed for, writing the low-level code by hand, restarting the training runs when they failed, watching the error numbers fall week after week until they fell off a cliff. The whole apparatus had cost a few thousand dollars. It now belonged, in a sense, to whoever could buy the three people who had made it.

The third person was Ilya Sutskever, who believed. That was the quality people remembered about him first, before the intensity and the strange, abrupt way he had of cutting to the center of a problem. Where others in the field hedged, Sutskever did not. He had decided, earlier than almost anyone, that the methods their small community had been ridiculed for were not a curiosity but the future, and that the future would arrive faster than the cautious expected, and that the way to get there was to make the networks larger and feed them more. The result on the screen was, to him, confirmation of something he had already known.

Two months after the contest, the three of them had done something that professors and graduate students did not ordinarily do. They had formed a company. It had no product, because the result was the product. It had no revenue and no customers and no real plan beyond the obvious one, which was to be acquired. The University of Toronto, where all three worked, had given its blessing and taken a stake. The company existed mostly as a name and three signatures. The name was DNNresearch, for deep neural networks, and it was perhaps the purest acquisition target in the history of the field: nothing but the people and the idea they had proven.

Hinton understood that he was holding something valuable and that he had no idea what it was worth. He was a scientist, not a dealmaker, and the situation had no precedent he could consult. So he did the reasonable thing, which was to run an auction. He arranged for the interested parties to submit bids by email, and he set rules. There would be a borrowed hotel room. There would be a laptop. Each bidder would name a number, and the numbers would be shared in turn, and the price would climb in agreed increments until someone stopped. He would set the deadlines, and he would decide when it was over. He was, in effect, conducting from a casino hotel a sealed competition among some of the most powerful technology companies on earth, for a company that was a few weeks old and had three employees, none of whom had ever sold anything.

The bidders were a map of where the money in computing was about to go. Google was in, with deep pockets and a research operation that had already begun to take neural networks seriously. Microsoft was in. Baidu, the Chinese search giant, was in, represented through a researcher named Kai Yu who had seen what was coming and wanted it for his company. And there was DeepMind, a small London outfit founded by a former chess prodigy and game designer named Demis Hassabis, which believed in this future as fervently as anyone and could not remotely afford to win. The presence of DeepMind in the bidding was its own kind of signal. The people closest to the idea wanted it most and had the least cash, and the people with the cash were only beginning to grasp what they were buying.

The numbers climbed. They started in a range that would have been a generous outcome and kept going past it. An email would arrive; Hinton would relay the new figure; the others would decide whether to follow. The price moved through twenty million, then higher, and as it rose it took on the unreal quality that large numbers acquire when they are attached to something that, on paper, was almost nothing. There was no factory. There were no patents that mattered yet. There were no users. There were two graphics cards in a bedroom in Toronto and a result that everyone in the field now suspected was the beginning of the rest of their careers. The bidding crossed thirty million and kept rising, and Hinton, standing in the borrowed room, watched the figures arrive and understood that what was being valued was not the software itself. The software could be rewritten in a month. What was being valued was the conviction, the long bet that these three had made when it was unfashionable to make it, and the proof they had finally produced that the bet was correct.

It is worth pausing on how strange that was. Hinton had spent most of his working life on the wrong side of his own discipline. The approach he championed, building crude mathematical imitations of brain cells and letting them learn from examples rather than following rules a programmer wrote down, had been declared a dead end more than once, by people with credentials and reasons. Students were warned away from it. Grants were hard to get. For long stretches the entire enterprise had survived on the stubbornness of a few dozen people scattered across a handful of universities, who kept publishing papers that the mainstream of artificial intelligence regarded, when it regarded them at all, as a quaint detour. Hinton had moved countries partly to find institutions that would tolerate the work. He had watched promising people leave the field for something more respectable. And now, in a casino hotel in the mountains, the largest companies in technology were bidding against each other in millions of dollars for the right to employ him and two of his students, because the detour had turned out to be the road.

He stopped the auction. He did not let it run to whatever ceiling the bidders might have reached; later he would say he could probably have gotten more, and that he chose not to. The price stood at forty-four million dollars, and he chose Google. There were reasons beyond money. There were the people, the resources, the sense that this was where the work could continue at the scale Sutskever already dreamed about. And there was also, characteristically, the matter of his back. Whatever else Google offered, the deal would let him keep doing the thing he could do standing up. The three of them would join the company. The bedroom graphics cards had become, by way of a hotel-room email auction, one of the more consequential purchases the industry would make that decade.

Almost no one outside that room saw it happen. There was no stage, no press release timed to a stock market open, no executive in a black turtleneck holding a glowing object aloft. There was a man who would not sit down, a laptop, and a string of emails counting upward into the dark over a frozen lake. And yet if you wanted to mark the moment when the modern artificial intelligence industry began, the moment when the idea stopped being an academic embarrassment and became the most valuable thing in technology, the thing every large company would soon reorganize itself around, you could do worse than this. The money had arrived. Where money goes, attention and talent and ambition follow, and they followed fast. Within a few years the salaries of the people in that small community would rival those of professional athletes, and the companies bidding that December would be locked in a competition that reshaped the world’s economy.

The obvious question is the one this book exists to answer. How did this happen? Not the auction itself, which was merely the visible spike, but the long buried thing underneath it. How did an idea that had been dismissed, defunded, and pronounced dead settle for half a century on the far margins of computer science, kept alive by a stubborn few, and then, almost overnight, become the engine of the most powerful machines humans had built? The result that drew the crowd to Harrah’s did not come from nowhere, and it did not come from genius alone. It came from a long chain of people, most of them now forgotten, who believed something that the evidence of their own era did not support, and who kept working on it through ridicule and failure and the deaths of their own careers, because they could not let it go.

Hinton himself was a link in a much older chain than the technology suggests. His family tree ran back through generations of mathematicians and eccentrics and explorers; he was a descendant of George Boole, whose nineteenth-century algebra of logic became the grammar of every digital computer, and he was connected to the family that gave its name to the world’s tallest mountain. He came by his obstinacy honestly. But the idea he carried to that hotel room was not Boole’s clean logic. It was almost the opposite. It was messy, statistical, biological, a machine that learned the way a child learns, by being shown things and corrected, rather than by being told the rules. That idea, too, had a beginning. It had a first machine and a first true believer, and they belonged to an era when the men who imagined thinking machines wore narrow ties and worked under fluorescent light, and the press treated every demonstration as a herald of the apocalypse or the millennium.

To understand the man who would not sit down, and the fortune that changed hands above the lake, you have to go back more than half a century, to a summer morning in Washington, D.C., and a thirty-year-old psychologist with a gift for showmanship who had built a machine that could learn from its mistakes, and who was about to tell the world that it would one day be conscious of its own existence.