(And with the amount of traffic those sites receive, individual tests can be completed in seconds.) They are helped, of course, by the fact that the web is particularly conducive to rapid data acquisition and product iteration. Instead they conduct controlled experiments by showing different versions to different groups of users until they have iterated to an optimal solution.
Online companies like Amazon and Google don't anguish over how to design their websites. In some domains this is already starting to happen. What if businesspeople and policy-makers were to spend less time relying on instinct or partially informed debate, and more time devising objectives ways to identify the best answers? I think that would often lead to better decisions. What a shame then that experiments are, by and large, used only by scientists. Instead it has freed us to appreciate the universe in terms that are well beyond our abilities to derive by intuition alone. Our embrace of truth as defined by experiment (rather than by common sense, or consensus, or seniority, or revelation or any other means) has in effect released us from the constraints of our innate preconceptions, prejudices and lack of imagination. The superiority of the latter approach is demonstrated not only by the fact that science has uncovered so much about the world in which we live, but also, and even more powerfully, by the fact that such a lot of it - from the Copernican principle and evolution by natural selection to general relativity and quantum mechanics - is so mind-bendingly counter-intuitive. The scientific method dictates that wherever possible we should instead conduct a suitable controlled experiment. When required to make a decision, the instinctive response of most non-scientists is to introspect, or perhaps call a meeting. The scientific concept that most people would do well to understand and exploit is the one that almost defines science itself: the controlled experiment.