When setting up experiments…

Ask yourself these questions:

What will be the metric to measure (pieces, percentages, time, dollars)? Pick one or multiple of these options.

How can we assess the current state? Sometimes this step requires weeks of preparation. 

How will we measure what has changed after the test? If we are not able to measure the difference later, the experiment will be a waste of our time. 

How will you change your actions based on the experiments different outcomes? If you do nothing, you don’t need the experiment.

How, based on the results of your experimentation, will we raise revenue, or cut costs, or improve control? If we don’t know how, why are we interested in this test?

Needless to say, if you know the results of your experiment up front, this is not an experiment.