Sampling Distributions and the Central Limit Theorem

Three distributions to keep apart

Questions about sampling go wrong when distributions get mixed up, so name them carefully.

• The population distribution holds the values of the variable for every individual in the population: the lengths of all 10,000 calls the help line took last month.
• The distribution of one sample's data holds the values in a single random sample, such as the lengths of 40 calls.
• The sampling distribution of a statistic is the distribution of the values of that statistic, such as the sample mean, for all possible samples of a given size from a given population.

The first two describe individual calls. The third describes a statistic, one number per sample.

Every possible sample

A sampling distribution is easiest to see when the population is tiny. Five plants in a classroom are 12, 15, 20, 24 and 34 centimeters tall, a population mean of 21 cm. There are exactly 10 different samples of 2 plants. The table lists each one with its sample mean, and those 10 means are the sampling distribution of $\bar{x}$ for samples of size 2 from this population.

Simulating a sampling distribution

Real populations are far too big for that. The help line's 10,000 calls can form about ${10}^{112}$ different samples of 40, more than the number of atoms in the observable universe. So you simulate. Assume what the population looks like, or assume values for its parameters. Generate a large number of random samples from it, compute the statistic for each sample, and record it. The distribution of the recorded values approximates the sampling distribution.

The three histograms below come from one such simulation: the population of 10,000 call lengths, then the means of 1,000 random samples of size 2, then the means of 1,000 random samples of size 40.

The central limit theorem

The pattern in those histograms has a name. The central limit theorem (CLT) says that the sampling distribution of a mean of a random sample has a shape that can be approximated by a normal distribution, and the larger the sample, the better the approximation.

Read that carefully. It is about the distribution of the sample mean across many samples. It does not say the population becomes normal, and it does not say the data inside one sample become normal. How large a sample is large enough depends on how skewed the population is, and 4.1 Sampling Distributions for Sample Means gives the working rule.

Randomization distributions

Experiments assign treatments at random instead of sampling at random, and they have their own version of this simulation. Twenty volunteers were randomly assigned, 10 to learn a list of words with a new memory technique and 10 with their usual method. The technique group recalled a mean of 15.7 words and the usual group 13.0, a difference of 2.7 words.

Could random assignment alone produce a gap that big? To find out, pool the 20 responses, reassign them at random to two groups of 10, and record the difference in means. Repeat many times. The resulting randomization distribution shows how the statistic varies from one random assignment to another, and it approximates the sampling distribution of the difference.

Topic 3.6 turns this kind of comparison into a p-value.