ExplanationWhy Specify Experimental Design?
At level one of this topic, we introduced the two types of experimental design and at level two we explained the advantages of each. At this stage, we will explain why it matters which type of experimental design you choose.
The key thing to remember is that the sample values collected in one sample of a paired design can paired with corresponding values in the other condition's sample. This fact allows us to be much more specific when we make comparisons between the two conditions. In an independent experimental design, no pairings can be made so the best we can do is compare the average (the mean) of each group.
Sampling Error and Variance
You can understand the improved accuracy that a paired design provides by thinking about sampling error. Sampling error is likely to be far smaller when the same subject is measured twice.
Here is an example. Imagine you measured your height on a monday morning, spent a week stretching, and measured your height again. Even the smallest change in height would suggest that the stretches had made you grow. Now imagine you spent a week stretching, and then measured your height and the height of another person who had not spent the week stretching. You would expect there to be a difference in your heights because of natural variation.
It is obviously important to take into account these differences in variation when comparing samples. You will see how this is done when we move on to how to choose a t-test.
At level two, we introduced the concept of confounding variables. These are variables other than your independent variable, which might have an effect on your dependent variable. Confounding variables increase the variance of the measurements of the dependent variable.
Imagine an example where you want to test whether men or women are taller. The independent variable is gender and the dependent variable is height. Anything else that effects height is a confounding variable in this study. An obvious example would be age. The more different age groups you measure, the more variation you will see in heights. Nationality would be another. If you are lucky, confounding variables will only increase variance but if you are unlucky, they could introduce bias. In our current example, bias might be introduced if the men you measured were all jockeys (and so quite short) and the women were all basketball players. In this case, the confounding variable (sport) would eclipse the dependent variable (gender).
It is important to consider what confounding variables there might be when you carry out an experiment so that you can avoid their ill effects. Here are the most common ways to reduce the effects of confounding variables once you have identified them:
- Within subjects designs have a naturally lower risk of confounding variables introducing bias as the same subject is tested in each case. Of course, anything could happen to the subject between measurements, which could influence the second measurement, so such experiments are not immune to confounding variables;
- Counter Balancing can be used to reduce fatigue or practice effects for paired designs. Counter balancing controls the order in which subjects are placed in each of the two conditions. Half are measured first under condition 1 and then under condition 2, while the other half are placed under condition 2 for their first measurement and then measured in condition 1 afterwards. This is not always possible, particularly in 'before and after' type experiments that test to see whether or not an intervention (a medical treatment, or a diet, for example) have an effect.
- Random Sampling reduces the risk of counfounding variables introducing bias. If samples are picked completely at random, then any confounding variables that are present should be spread evenly across both groups.
- Controlling confounding variables can reduce their effect greatly. In our example on height above, you could control for age by only measuring 25 year olds. You could control for nationality by only measuring people from your own country (or race). Another way of controlling for confounding variables is to make sure that each group contains the same spread of values of each confounding variable (the same number of people of each age, for example). This obviously gets difficult if there are more than two confounding variables.