Experimentation Works: The Surprising Power of Business Experiments
p-value is the probability that the observed difference (or one that’s more extreme) would occur by chance,
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
In innovation, many tasks are unique, project requirements constantly change, and the output—thanks, in part, to the widespread use of computer-aided design and simulation tools—is information, which can reside in multiple places at the same time.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
ask an endless stream of “what if” questions.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
a business analytics program is incomplete without controlled experiments.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
the greater the novelty of an innovation, the less likely it is that reliable data will be available.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
experimentation typically begins by the selection or creation of one or more possible testable hypotheses, which may or may not include the “best possible” solutions—since no one knows what these are in advance.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
experimentation plan required a more holistic approach that considered how hypotheses informed each other, the sites on which they were run, goals and metrics linked to business outcomes, projected sample sizes, implementation steps, and so on. Most of all, a good plan allowed for iterations and led to the exploration and optimization of much bolde
... See moreStefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
Today, designed experiments are being used for the optimization of processes, products, store layouts, websites, and business models in both online and offline business settings.
Stefan H. Thomke • Experimentation Works: The Surprising Power of Business Experiments
When test results conflict with prior experiences, one should rerun the experiment, preferably with larger sample sizes and a low significance threshold.