Explanation

Imagine taking many samples from a population, two at a time, and calculating the difference between the means of each pair (that is, one mean minus the other). Just as the means themselves would be normally distributed, so would be their differences. This distribution of differences is called the sampling distribution of differences and it can be treated in the same way as a sampling distribution of means. For example, it will have a mean itself, and you can calculate confidence intervals around that mean.

Confidence Intervals and the Null Hypothesis
If the null hypothesis were true and there were no difference between your samples, you would expect the two means to be equal, so the null hypothesis is that one mean minus the other equals zero. The difference between the means of two samples from the same population is not always zero. As we saw above, they form a normal distribution, with small differences being more likely than large ones.

You can calculate a confidence interval for the difference between the means of two samples. Look at the formula for the independent t-test at level two of this topic and you will recognise the structure of the formula for confidence intervals. For example, if you found a mean in one sample of 3 and a mean in the other sample of 4, then the difference would be 1 (4-3). Simply reporting this difference is not very useful. Reporting confidence intervals around that difference (-1 to 3, for example) tells us the range into which the difference is most likely to fall.

The points below summarise the use of confidence intervals for comparing means:

• If the lower confidence level is below zero (i.e. it is negative) and the upper level is positive, then the null hypothesis cannot be rejected as the population mean of the differences could be zero;
• If both confidence levels have the same sign (both positive or both negative), then the null hypothesis can be rejected as a mean difference of zero is not in the confidence interval.

The horizontal lines show the range of values you might find for the difference of means, with a difference of zero shown in the centre. The coloured areas show two different confidence intervals around the same mean difference. Notice that the top, blue, interval does not cross zero, and so allows us to be 95% confident that the population difference is not zero. The red area shows the same mean difference, but with a wider confidence interval (due to a larger standard deviation in differences). This interval does cross zero, which means that the population mean difference could be zero, so we cannot reject the null hypothesis.

Exploration

The graph in the game shows the distribution of differences between pairs of means from a sampling distribution. You can increase or decrease its standard deviation using the two buttons. You can choose whether you see the 10%, 5%, or 1% confidence level, which will be coloured red. Any difference inside this confidence interval supports the null hypothesis. The horizontal axis shows the size of the difference in numbers of standard deviations.

To reject the null hypothesis (that there is no difference between the samples), where must the difference between the means fall? Choose..In the red areaIn the white area   As the standard deviation of differences gets larger, where does the area that rejects the null hypothesis move? Choose..Further from the meanCloser to the mean   what happens to the size of the red area as the confidence level increases? Choose..It gets widerIt gets narrower

( You need to enable Java to see this applet. )

Application

In other words, we can be 95% confident that the difference between the means of the populations behind your two samples is somewhere between 2.67 and 4.63.

Does this range cover zero? Choose..YesNo