NUMBAT OER - Open Educational Resources

6. Experimental design

From the discussion of hypothesis testing, error and power, it is clear that the effectiveness of testing depends ultimately on the data. Where data have been collected before the hypothesis is defined, there is often little that can be done to improve matters. However, where an experiment is designed specifically to test a hypothesis, it is often possible to optimise the measurements so that the test is effective, that is to increase the power of the test (see Section 5).

Good experimental design is often quite complex, especially when it comes to isolating the source of variation under study and being able to exclude other sources of variation. Agricultural field trials are a good example, where differences in test plots have to be attributable to crop variety or fertilizer treatment, not to soil conditions or exposure to wind. But even in the more controlled conditions within a laboratory, it is important that an experiment is thought through and matches the prediction and the hypothesis used to test it. There are various techniques to optimise the effectiveness of experimental design, including power analyses.