The End of Theory: The Data Deluge Makes the Scientific Method Obsolete by Chris Anderson
So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if imperfectly, explain the world around us. Until now. Today companies like Google, which have grown up in an era of massively abundant data, don’t have to settle for wrong models. Indeed, they don’t have to settle for models at all.
Speaking at the O’Reilly Emerging Technology Conference this past March, Peter Norvig, Google’s research director, offered an update to George Box’s maxim: “All models are wrong, and increasingly you can succeed without them.”
There is now a better way. Petabytes allow us to say: “Correlation is enough.” We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
see update, below. Norvig was misquoted, he agrees with Box’s maxim
I must say I am not at all convinced that a new method without theory ready to supplant the existing scientific method. Now I can’t find peter Norvig’s exact words online (come on Google – organize all the world’s information for me please). If he said that using massive stores of data to make discoveries in new ways radically changing how we can learn and create useful systems, that I believe. I do enjoy the idea of trying radical new ways of viewing what is possible.
Practice Makes Perfect: How Billions of Examples Lead to Better Models (summary of his talk on the conference web site):
Related: Will the Data Deluge Makes the Scientific Method Obsolete? – Pragmatism and Management Knowledge – Data Based Decision Making at Google – Seeing Patterns Where None Exists – Manage what you can’t measure – Data Based Blathering – Understanding Data – Webcast on Google Innovation