Topic 3.4: Potential Problems with Sampling

Learning Objectives:

Identify potential sources of bias in sampling methods.
Recognize undercoverage, nonresponse, and volunteer response bias.
Describe how bias can lead to an overestimate or underestimate.
Recognize question wording bias and self-reported response bias in surveys.
Write a complete AP-style explanation of bias using the population, sample, and direction of the bias.

Essential Knowledge:

Bias occurs when certain responses are systematically favored over others.
Undercoverage bias occurs when part of the population has a reduced chance of being included in the sample.
Nonresponse bias occurs when selected individuals do not respond and may differ from those who do respond.
Volunteer response bias occurs when people choose themselves to participate and those volunteers may not represent the population.
Problems with survey wording or self-reported answers can also create biased results.
Good bias explanations describe how the sample differs from the population and the likely direction of the bias.

Key Vocabulary

Bias	A systematic tendency to favor certain responses over others.
Undercoverage Bias	Bias that occurs when part of the population has a reduced chance of being included in the sample.
Nonresponse Bias	Bias that occurs when chosen individuals do not respond and may differ from those who do respond.
Volunteer Response Bias	Bias that occurs when people choose themselves to participate and volunteers differ from nonvolunteers.
Population	The entire group the study wants information about.
Sample	The individuals from whom data are actually collected.
Overestimate	An estimate that is systematically too high.
Underestimate	An estimate that is systematically too low.
Question Wording Bias	Bias caused by confusing or leading survey questions.
Self-Reported Response Bias	Bias caused when people inaccurately report their own traits or behaviors.
Leading Question	A question worded in a way that pushes respondents toward a certain answer.
Representative	Accurately reflecting the characteristics of the population.

Video 1: Potential Problems with Sampling

[0:00 – 6:57]

1. [0:00] Topic 3.4 discusses potential sources of in .

2. [0:12] We will learn which methods lead to estimates and how they can cause overestimates or .

College Pamphlet Example [0:27 – 1:41] A college advertisement claimed that 99.7% of former students were working full-time in careers of their choice. The catch was that the statistic was based only on recent graduates, not all students who started at the school.

3. [1:31] The pamphlet sampled recent , so students who did not were excluded.

4. [1:41] Only percent of the incoming freshman class graduated within six years.

5. [2:23] Bias is a systematic tendency to favor certain over others.

6. [2:23] Excluding part of the population can create bias.

7. [2:23] Undercoverage happens when part of the population has a reduced chance of being in the sample.

8. [2:44] When selected individuals do not respond, the result may be bias.

9. [2:44] In the internship survey example, only percent of sampled students responded.

How to Explain Bias on an FRQ [3:27 – 4:12]

Identify the population and the sample.
Explain how the sample may systematically differ from the population.
State whether the method is likely to cause an overestimate or underestimate.

10. [3:30] When explaining bias, first identify the population and the .

11. [3:30] Then explain how the sampled individuals might differ from the general .

12. [3:46] If students without paid internships respond less often, the reported 85% is likely an of the true percentage.

13. Reflection: A university randomly selects students and emails them a survey asking whether they found a paid summer internship. Only 10% respond. Explain why this could lead to nonresponse bias. Identify the population, the sample, and whether the result is likely too high or too low.

Running Study Volunteers [4:55 – 5:41] The video gives an example of a study about whether people enjoy running. An advertisement asks for volunteers for a running study, which may attract people who already like running more than the general population does.

14. [4:55] response bias occurs when invitations are sent to all individuals in the population and only choose to participate.

15. [4:55] Advertising a running study may attract people who already running.

16. [4:55] Because of that, the study may the true proportion of people who enjoy running.

17. [5:43] If you are unsure of the exact vocabulary term, the most important thing is to describe how the bias .

18. [5:43] You should also explain whether the bias causes an overestimate or an .

Other Survey Biases [6:12 – 6:57]

Question wording bias occurs when a question is confusing or leading.
Self-reported response bias occurs when people inaccurately report their own traits or behaviors.
These problems come from the data collection instrument or response process itself.

19. [6:12] Confusing or leading survey questions create question bias.

20. [6:12] When people inaccurately report their own traits, that is self-reported response .

21. [6:57] Bias occurs when certain responses are systematically over others.

22. Reflection: Describe the difference between undercoverage bias and volunteer response bias. Explain how each one makes the sample differ from the population and how either could lead to a distorted estimate.

Exit Ticket

A city wants to estimate the proportion of all residents who support adding more bike lanes. Officials mail every household a card that says, "Complete our survey if you support this important plan to make our streets safer by adding more bike lanes." Residents may choose whether to respond online.

(a) Identify the population and the sample.

(b) Identify one source of sampling bias and explain whether it is likely to create an overestimate or underestimate of support.

(c) Identify one source of response bias in the wording of the survey question.

(d) Describe one change that would improve the study.

Topic 3.4: Potential Problems with Sampling

Key Vocabulary

Video 1: Potential Problems with Sampling

Exit Ticket

Class Responses (Esc to close)