Does the Data Deluge Make the Scientific Method Obsolete?

“All models are wrong, but some are useful.”

So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if imperfectly, explain the world around us. Until now. Today companies like Google, which have grown up in an era of massively abundant data, don’t have to settle for wrong models. Indeed, they don’t have to settle for models at all.
…
Speaking at the O’Reilly Emerging Technology Conference this past March, Peter Norvig, Google’s research director, offered an update to George Box’s maxim: “All models are wrong, and increasingly you can succeed without them.”
…
There is now a better way. Petabytes allow us to say: “Correlation is enough.” We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.

In this talk we will see that a computer might not learn in the same way that a person does, but it can use massive amounts of data to perform selected tasks very well. We will see that a computer can correct spelling mistakes, translate from Arabic to English, and recognize celebrity faces about as well as an average human—and can do it all by learning from examples rather than by relying on programming.

My guess is that this emerging method will be one additional tool in the evolution of the scientific method. It will not replace any current methods (sorry, no end of science!) but will compliment established theory-driven science.

Yesterday, Google’s director of research Peter Norvig let visitors at the Emerging Technology conference in San Diego look into the technology that his firm uses in search and translation functions. As Norvig put it, a lot of the time Google does not rely on complex models and theories, but simply on large amounts of data.

One example is Google’s translation function, which allows Chinese texts to be translated into English, for example. In Chinese, multiple symbols that mean something on their own can also be combined to create a single word. Google segments Chinese texts by comparing a large amount of Chinese and English versions of the same content to increase the probability that the Chinese characters will match the English words.

Google’s mission is to organize the world’s information and make it universally accessible and useful.

I recently had a run-in with the fact-checkers for Wired magazine. They wrote and asked me:

Is it true that at your ETech presentation in March, you said, in a direct homage to George Box, “All models are wrong, and you don’t need them anyway”? Is that accurate?

Great, I thought–Wired is a publication with integrity and wants to get the facts right. I wrote back:

The quote I used was “essentially all models are wrong, but some are useful”.

The point I was making — and I don’t remember the exact words — was that if the model is going to be wrong anyway, why not see if you can get the computer to quickly learn a model from the data, rather than have a human laboriously derive a model from a lot of thought.

I figured they would either use the quote I gave them, paraphrase it, or drop it completely if it didn’t fit with the point of the story. But when Chris Anderson’s story The End of Theory: The Data Deluge Makes the Scientific Method Obsolete came out in June 2008, there was a fourth possibility that I hadn’t even counted upon: they attributed to me a made-up quote that actually contradicts the reply I gave them:

Peter Norvig, Google’s research director, offered an update to George Box’s maxim: “All models are wrong, and increasingly you can succeed without them.”

To set the record straight: That’s a silly statement, I didn’t say it, and I disagree with it.

The ironic thing is that even the article’s author, Chris Anderson, doesn’t believe the idea. I saw him later that summer at Google and asked him about the article, and he said “I was going for a reaction.” That is, he was being provocative, presenting a caricature of an idea, even though he knew the idea was not really true. That’s a mode I expect from other publications, but it’s not what I want from Wired, and I don’t expect Wired to make up facts to support their caricature.