Bootstrap Resampling: Squeezing Certainty From One Sample
You measured the heights of 40 students and got a mean of 162 cm. But if you had grabbed a different 40 students, you would have gotten a slightly different mean. So how sure can you be about that 162? Normally you would need a formula for the standard error, and for some statistics (like the median) those formulas are ugly or don't exist. In 1979, statistician Bradley Efron published a wild idea: pretend your one sample IS the whole population, then draw thousands of fake samples FROM it, with replacement. He called it the bootstrap, after the phrase 'to pull yourself up by your own bootstraps.' It sounds like cheating, but it works, and today it powers everything from medical trials to machine learning.
What You'll Learn
- What 'resampling with replacement' actually means, step by step - How to build a bootstrap distribution from a single data set - How to read a 95% confidence interval off the percentiles - Why the bootstrap works even when no clean formula exists
Sampling With Replacement
Say your sample is just five values: [4, 8, 8, 15, 16]. To make one bootstrap resample, you draw five values from this set, but after each draw you put the value back. So the same value can appear more than once, and some may not appear at all. One resample might be [8, 8, 16, 4, 8]; another might be [15, 4, 4, 8, 16]. Each resample has the same size as the original (n = 5) but a slightly different makeup. Because 8 appears twice in the original, it is more likely to show up often, which is exactly how the data's real shape gets preserved.
Building the Bootstrap Distribution
Here is the full recipe: 1. Take your original sample of size n. 2. Draw a resample of size n, with replacement. 3. Compute the statistic you care about (mean, median, correlation, whatever) on that resample. 4. Record it. 5. Repeat steps 2-4 a large number of times, typically B = 1,000 to 10,000. You now have thousands of recorded statistics. Their spread estimates how much your statistic would bounce around if you could actually re-run the whole study many times. The standard deviation of these bootstrap statistics is your bootstrap standard error.
The bootstrap assumes your original sample is representative of the population. If your sample was biased (say, only tall students volunteered), the bootstrap will faithfully reproduce that bias. It quantifies sampling variability, not sampling bias.
Reading a Confidence Interval
The simplest bootstrap confidence interval is the percentile method. Suppose you ran B = 2,000 resamples and computed the mean each time, giving you 2,000 bootstrap means. Sort them from lowest to highest. For a 95% interval, you chop off the bottom 2.5% and the top 2.5%. The value at the 2.5th percentile (the 50th smallest of 2,000) and the value at the 97.5th percentile (the 1,950th) are your interval bounds. If those come out to 158.3 cm and 165.7 cm, you report: 'We are 95% confident the true mean height is between 158.3 and 165.7 cm.'
Match each bootstrap term to its meaning.
Terms
Definitions
Drag terms onto their definitions, or click a term then click a definition to match.
In a bootstrap resample of size n = 5 drawn with replacement, which outcome is possible?
You have 2,000 bootstrap means. How do you get a 95% percentile confidence interval?
Bootstrap by Hand, Then by Code
Start with the sample [12, 15, 15, 22, 30]. Using dice or a random number generator, draw 5 resamples of size 5 with replacement and compute the mean of each by hand. Then write a short Python or spreadsheet script that does B = 1,000 resamples and prints the 2.5th and 97.5th percentiles of the resampled means. Compare your hand-computed means to the code's distribution and write two sentences on what the interval tells you about the true mean.
Flashcards — click each card to reveal the answer
Want to keep learning?
Sign up for free to access the full curriculum — all subjects, all ages.
Start Learning Free