Topic 3.1: Introducing Statistics: Do the Data We Collected Tell the Truth?

Learning Objectives:

Identify questions to ask about how data were collected.
Explain how the way a sample is collected affects how we analyze the data.
Recognize when a sample may not be representative of the whole population.
Use the Wald airplane example to explain why missing cases can change a conclusion.
Describe why data collection methods that do not rely on chance can lead to untrustworthy conclusions.

Essential Knowledge:

Given that variation may be random or not, conclusions are uncertain.
A proper analysis of data must take into account how the data were collected.
Samples may not always be representative of the population of interest.
We should consider how sampled individuals may differ from those who were not sampled.
Methods for data collection that do not rely on chance result in untrustworthy conclusions.

Key Vocabulary

Population	The entire group we want to learn about.
Sample	The smaller group from which data are actually collected.
Representative Sample	A sample that reflects the important features of the population.
Data Collection Method	The process used to decide who or what is included in the sample.
Chance	Random selection rather than a method based on convenience or volunteer response.
Untrustworthy Conclusion	A conclusion that may not reflect the truth about the whole population.
Context	The situation in which data were collected and the conditions attached to the sample.
Biased Sample	A sample that systematically leaves out or overrepresents part of the population.
Analysis	The interpretation we make after looking at the data.
Vulnerable Spot	In the Wald example, an area where planes may have been hit and failed to return.

Video 1: The Wald Airplane Problem and Why Data Collection Matters

[0:00 – 2:15]

1. [0:00] Today we begin Unit and ask whether the data we collect tell the .

2. [0:07] We will learn what problems may arise when we collect and how the way we collect informs our analysis.

3. [0:07] We should be wary of how data are collected because that affects our .

Abraham Wald [0:13 – 0:51] Abraham Wald was a Hungarian Jewish statistician who worked for the Statistical Research Group during World War II. In this lesson, his wartime airplane example helps show why the way data are collected can change the conclusion we draw.

4. [0:13] Abraham Wald was a who worked during .

The Wald Airplane Problem [0:51 – 1:29] British bombers were returning from missions with bullet holes, and military leaders wanted to know where to place a limited amount of heavy, bullet-resistant armor. Soldiers made bullet-hole charts only for the planes that came back from the bombing runs.

5. [0:51] The British wanted to know where to put extra bullet-resistant on their planes.

6. [0:51] They could not armor every part of the plane because that would make it less .

7. [0:51] Soldiers made bullet-hole charts only for planes that came from the mission.

8. [1:29] Wald's key insight was to notice the in which the data were collected.

9. [1:29] The sample included planes that actually made it , not the planes that were shot down.

10. [1:29] The bullet holes we do see show places where a plane can be hit and still .

11. [1:29] So the best place for extra armor is probably where we do see many bullet holes.

12. [1:29] The planes that were actually shot were not included in the sample.

Key Rules from the Video [1:29 – 2:15]

Always ask how the data were collected before deciding what the data mean.
Ask who was included in the sample and who was left out.
If a data collection method does not rely on chance, the conclusions may be untrustworthy.

13. [2:04] A proper analysis of data must take into account how the data were .

14. [2:04] Sometimes our sample may not be of the whole .

15. [2:04] We should think about how the individuals in the sample may from those that were not sampled.

16. [2:15] Methods for data collection that do not rely on can lead to conclusions.

17. [2:15] Statisticians should be critical, cautious, compassionate, and avoid bad .

18. Reflection: Why does the Wald airplane example show that the context of data collection matters?

19. Reflection: What questions should a statistician ask about a sample before trusting conclusions drawn from it?

Exit Ticket

A principal wants to know whether students at a high school support a later school start time. She posts a survey link on the student council Instagram page, and 82% of respondents say they support the change.

(a) Identify the population and the sample.

(b) Explain why this data collection method may not tell the truth about the whole school.

(c) Describe one way the students who responded might differ from students who did not respond.

(d) Suggest a better way to collect the data using chance.

Topic 3.1: Introducing Statistics: Do the Data We Collected Tell the Truth?

Key Vocabulary

Video 1: The Wald Airplane Problem and Why Data Collection Matters

Exit Ticket

Class Responses (Esc to close)