Google does a great job of using statistical and engineering principles to improve. It is amazing how slow we are to adopt new ideas but because we are it provides big advantages to companies like Google that use concepts like design of experiments, experimenting quickly and often… while others don’t. Look Inside a 1,024 Recipe Multivariate Experiment
…
While we could have hypothesized which elements result in greater conversions (for example, the color red is more eye-catching), multivariate testing reveals and proves the combinatorial impact of different configurations. Running tests like this also help guide our design process: instead of relying on our own ideas and intuition, you have a big part in steering us in the right direction. In fact, we plan on incorporating many of these elements in future evolutions of our homepage.
via: @hexawise – My brother has created a software application to provide much better test coverage with far fewer tests using the same factorial designed experiments ideas my father worked with decades ago (and yet still far to few people use).
Related: Combinatorial Testing for Software – Statistics for Experimenters – Google’s Website Optimizer allows for multivariate testing of your website. – Using Design of Experiments
Google Optimizer is a great tool and should really be used whenever you create a new page. It's nowhere near as easy a sell as it should be alas.
Pingback: Curious Cat Management Blog » Combinatorial Testing – The Quadrant of Massive Efficiency Gains