How populations, samples, observational studies, and experiments affect what conclusions we can draw
Video Follow-Along Worksheet
↻ Already did this one? It stays open even after it’s checked off — just change any answer and click ✓ Check Answers again (no need to Reset). ✨ Grade with AI only ever raises your grade. Make sure you’re signed in so your changes save.
Learning Objectives:
Distinguish between a population and a sample.
Identify whether a study is observational or experimental.
Describe when it is appropriate to generalize from a sample to a population.
The entire group of individuals or items we want to learn about.
Sample
A subset of the population from which data are collected.
Representative Sample
A sample that reflects important features of the population.
Observational Study
A study in which researchers observe or collect data without imposing treatments.
Experiment
A study in which researchers impose treatments or conditions on subjects.
Treatment
A specific condition applied to subjects in an experiment.
Experimental Unit
The individual or subject that receives a treatment.
Retrospective Study
An observational study that examines existing or past data.
Prospective Study
An observational study that follows individuals into the future.
Confounding Factor
An outside factor that makes it hard to separate causes in an observational study.
Generalize
Use sample results to draw conclusions about a population.
Causal Relationship
A relationship in which one variable directly causes a change in another.
Video 1: Populations, Samples, Observational Studies, and Experiments
[0:00 – 8:03]
1.[0:00]
Topic 3.2 focuses on planning and on the scope of different studies allow us to draw.
2.[0:08]
We will learn the difference between a and a , and between an and an .
Returning to Wald's Airplane Example[0:26 – 1:53]
The video revisits Abraham Wald's World War II airplane problem to introduce populations and samples. Wald realized that the data only came from planes that returned, which completely changed how the bullet-hole pattern should be interpreted.
3.[1:11]
Because the planes that were charted made it , Wald said the extra armor should go where the fewest hits appeared, on the .
4.[1:53]
A is all individuals or items of interest.
5.[1:53]
A is always a of the population.
6.[1:53]
We often cannot measure the whole , so we measure a and try to make inferences.
7.[2:39]
In the airplane example, the sample was not of the population because planes that were shot down were probably hit in spots.
8.[2:39]
The returning planes formed a sample because only planes that made it back safely were included.
Generalization Rules[3:18 – 3:58]
Generalize to a population only when the sample was randomly selected or is otherwise representative.
A sample can be generalized only to the population from which it was selected.
A representative sample from one population does not automatically describe a different population.
9.[3:18]
It is only appropriate to from a sample when it is randomly selected or otherwise .
10.[3:18]
A sample is only generalizable to the from which it was selected.
11.[3:18]
A representative sample of lima beans does not automatically let us infer about because that is a different population.
The Racial Income Gap Example[3:58 – 5:38]
The video next uses survey data on household income to show how observational studies work. The data can reveal patterns worth analyzing, but the observed differences may have multiple possible explanations.
12.[4:56]
Possible explanations such as school inequity, family connections, and direct discrimination are called .
13.[5:38]
The income-gap survey was an because no treatments were imposed.
14.[5:38]
In an observational study, are not on individuals.
15.[5:38]
A study examines existing or past data, while a study follows individuals into the future.
The Resume Experiment[6:13 – 7:27]
To test whether hiring discrimination could directly affect callback rates, researchers sent identical resumes to employers. The only difference was the type of name on the resume, and that treatment was randomly assigned.
16.[6:13]
In the resume study, researchers which employers received which names on otherwise resumes.
17.[7:26]
This resume study was an because treatments were imposed on .
18.[7:27]
In this experiment, the treatment was the placed on the resume.
19.[7:27]
A well-designed experiment can determine a .
Key Takeaways from the Video[7:47 – 8:03]
Population = all individuals of interest; sample = a subset of that population.
Observational studies can describe or compare groups, but they cannot prove cause and effect.
Experiments impose treatments and, when well designed, can support causal conclusions.
20.[7:48]
We cannot infer from observational studies, but we can generalize from samples.
21.
Reflection: Why is it risky to generalize from the returning planes in Wald's sample to all hit planes?
22.
Reflection: How do observational studies and experiments differ in the kinds of conclusions they allow statisticians to make?
Exit Ticket
A principal wants to study whether students who participate in sports tend to have higher GPAs. She uses school records to look at 120 students chosen at random from her high school and compares the GPAs of athletes and nonathletes.
(a) Identify the population and the sample.
(b) Identify whether this is an observational study or an experiment, and explain why.
(c) Explain whether it is appropriate to generalize the results to the whole high school.
(d) Explain whether the principal can conclude that playing sports causes higher GPAs.
Class Responses (Esc to close)
Click a "Class" button next to a question to see responses.