The Ponzo Illusion

Choosing a T-Test

Tutorial Navigation

Getting Started	General Instructions \| Introduction to Your Study \| Experimental Design \| Stating a Hypothesis
Descriptive Statistics	Histograms \| Central Tendency \| Standard Deviation \| Confidence Intervals
Comparing Two Samples	Samples and Populations \| Choosing a T-Test \| Independent T-Test \| P-Values and T-Tables
Important Concepts	The Normal Distribution \| Z Scores \| Probability Distributions
Levels	You are currently on Choosing a T-Test at level 3. Level 1 \| Level 2 \| Level 3
Next Topic	Samples and Populations \| Independent T-Test

Explanation

Why Are There Two Types of T-Test?
At level one, we learned to choose the right t-test for each kind of experimental design. This theory section explains why we need a different test for each.

First, a quick recap:

With paired (dependent) samples, it is possible to take each measurement in one sample and pair it sensibly with one measurement in the other sample. This might be because measurements were taken from the same group twice (repeated measures) or because there is some other way to join measurements, for example, comparing the IQ of older and younger brothers;
With independent samples, there is no sensible way to pair off the measurements.

A paired t-test is specifically designed to see if the same subjects differ in each condition. The formula looks specifically at the differences between paired measurements, rather than comparing the two means (as the independent t-test does).

Why Do We Choose the Number of Tails?

If you picked two small samples from the same population, they would be different from each other because of sampling error. Some pairs of samples would be very different from each other, but most would be quite close. All these possible differences create a distribution.

The distribution of differences that you would expect from pairs of samples from a population is normal in shape. You saw in the section on probability distributions that you can calculate the probability of any value in a normal distribution occuring. That means you can take the difference between your two samples and calculate the probability of that difference being produced by two samples from the same population. That is what a t-test does.

If a t-test tells you that the same population would produce two samples as far apart as yours only 5% of the time, you can be very confident that there is a true difference in your data. Of all the samples that have a bigger difference than yours, half will have a difference in one direction and half will have a difference in the other direction. That is to say that half will be such that sample 1 is greater than sample 2, and half will be the other way around. These large differences make up the extremes (or tails) of the distribution described above.

If you are only interested in the samples that differ in one particular direction, then you are only interested in one of the tails, so the proportion of pairs of samples that would be further apart than yours is suddenly halved. For a one-tailed test you ignore the differences that fall in one direction, which means that you have to account for twice as many in the direction that interests you. This makes a one-tailed test less strict as you can find significance from a smaller difference.

The graph above shows the tails of the distribution described above. It shows how often you would expect to find a difference of a given size between two samples from the same population. Notice that the highest (and most likely) difference is zero. The further from zero you move (i.e. the bigger the difference in the sample means), the smaller the probability becomes.

The red area covers 5% of the distribution and shows the largest differences in either direction. The whole of the blue and red area on the right hand side shows the 5% of the data that is furthest from the zero in one direction. Differences in the blue area would be significant in a one tailed test, but not in a two tailed test.

Exploration

The game below simply demonstrates how the choice of tails in a t-test affects the interpretation of its results. The chart in the game shows all of the possible differences between two samples from the sample population. The horizontal axis (from -4 to +4) indicates the size of the difference between two samples and the height of the curve indicates the proportion of random samples that you would expect to have each difference.

The curve is highest where the difference is zero, indicating that the average difference between two random samples from the same population is zero. As the differences get bigger, the probability of two samples showing such a difference gets smaller. The red shaded area on the curve shows you where the largest differences lie. You can choose to see the largest 1%, 5% or 10% of possible differences and you can choose whether you want to see that percentage in one direction (one-tailed) or two.

( You need to enable Java to see this applet. )
To accept the experimental hypothesis, should the difference between your samples be in the red zone or the white?
For a given confidence level (1%, 5% or 10%), what happens to the total size of the red area as you swap between 1 and 2 tails?
If you change from two-tailed to one-tailed, what happens to the size of the red area in the tail of the expected difference?

Application

Your data is from independent samples, which means that you will perform an independent t-test. If you had performed a paired t-test by mistake, would the probability of accepting the null hypothesis have been higher or lower?

Your experiment is one-tailed, which means that you expect a change in a certain direction. If you accidentally looked up the values for a two tailed test, what effect would that have on your findings?

Samples and Populations | Independent T-Test