Topics 3.6–3.7: Selecting an Experimental Design & Inference

↻ Already did this one? It stays open even after it’s checked off — just change any answer and click ✓ Check Answers again (no need to Reset). ✨ Grade with AI only ever raises your grade. Make sure you’re signed in so your changes save.

Learning Objective (VAR-3.D): Explain why a particular experimental design is appropriate.

Learning Objective (VAR-3.E): Interpret the results of a well-designed experiment.

Essential Knowledge: A randomized block design separates natural variability from treatment effects. Statistical inference allows us to attribute conclusions from sample data to the population. Statistically significant differences are evidence that treatments caused the observed effect.

Key Vocabulary

Randomized Block Design	Subjects are grouped into blocks by a characteristic, then randomly assigned to treatments within each block.
Matched Pairs Design	A special case of blocking that uses pairs of similar experimental units, with one receiving each treatment.
Blinding	Subjects and/or researchers are unaware of which treatment is being administered.
Statistical Significance	Observed differences are too large to be reasonably attributed to chance alone.
Statistical Inference	Using sample data to draw conclusions about a population or treatment.

Video 1: Blocking & Blinding (Topic 3.6)

5:54

Part A: Introduction & High Cholesterol Example

[0:00–0:53]

1. In this video, we will learn how can improve the design of an experiment and under what conditions an experiment can be .

2. The "High Cholesterol" example is a from a previous AP exam.

Part B: Completely Randomized Design

[0:53–2:29]

3. A well-designed experiment includes four key components: of treatments, , replication, and .

4. To randomly assign 200 volunteers to two groups, we can number them 1 to 200, then use a to select 100 numbers without repeat.

5. The 100 randomly selected volunteers receive the drug, and the remaining 100 receive the drug.

6. After treatment, we measure and compare the for both groups.

Part C: Randomized Block Design

[2:51–4:16]

7. From the prompt: "High cholesterol level can be reduced by or by drug treatment."

8. This suggests that could be a blocking variable that affects cholesterol levels.

9. In a randomized block design, we first separate volunteers into based on a characteristic (here: high vs. low exercise), then perform randomization .

10. We then compare cholesterol levels and overall.

Part D: Double Blinding

[4:16–5:27]

11. If asked a yes/no question on the AP exam, make sure you answer somewhere in your response.

12. Double blinding is possible when both the and the are unaware of what treatments are being given to each subject.

13. This works best if the treatments are similar in , taste, or administration method.

Part E: Key Takeaways

[5:27–5:46]

14. A randomized block design helps to separate from differences due to the blocking variable.

15. Blinding is possible when subjects and/or researchers are of the treatment being administered.

Video 2: Matched Pairs & Replication (Topic 3.6)

6:48

Part A: Tractor Plots Introduction

[0:00–1:22]

16. This example is from the AP exam (Form B).

17. In this study, "draft" refers to the needed to pull a plow.

18. Identifying components of this experiment:

• Response variable:

• Treatments:

• Experimental units:

Part B: Evaluating Randomization

[2:10–2:41]

19. Was randomization properly used?

20. Evidence: "It was which plot was to be plowed using the standard hitch."

21. The two (treatments) were randomly assigned to the two (experimental units).

Part C: Evaluating Replication

[3:09–4:02]

22. Was replication properly used?

23. Key insight: Replication means applying each treatment to a sufficient number of , not just taking multiple measurements.

24. Problem: Each treatment was applied to only experimental unit (plot). There were 25 measurements per plot, but this is not replication.

Part D: Confounding Variable

[4:17–5:11]

25. The draft is affected by environmental conditions such as , terrain, and .

26. Because each hitch was used on only one plot, we cannot determine if the difference in draft is due to the or the characteristics of that specific .

27. The treatments (hitches) are with the plots.

Part E: Improving the Design

[5:11–6:20]

28. Improvement 1: Split each plot in and use random assignment to determine which hitch is used on either half. This reduces the effect of the plot.

29. Improvement 2: Use more than experimental units.

30. Don't forget to mention that you will measure the after applying the treatments!

Part F: Key Takeaways

[6:20–6:48]

31. Proper replication requires that multiple receive the same treatment.

32. A design involves blocks of two similar experimental units, one receiving each treatment. OR each experimental unit receives treatments in random order.

Video 3: Inference & Experiments (Topic 3.7)

5:50

Part A: Introduction & Learning Goals

[0:00–0:36]

33. Three learning goals for this video:

• What is ?

• How does help determine statistical significance?

• Under what conditions can results be to the population?

Part B: The Résumé Experiment

[0:36–2:19]

34. In this study, two résumés were identical except for the (Greg Baker vs. Jamal Jones).

35. Résumés were sent to employers in and .

36. Each employer was a résumé with either a commonly White name or commonly African-American name.

37. Check if this is a well-designed experiment:

• Comparison? — Two groups (White vs. African-American names)

• Random assignment? — Each company given a random name

• Replication? — résumés for each group

• Control? — All résumés equal except for name

Part C: Results of the Résumé Experiment

[2:19–3:00]

38. Callback rate for résumés with White names: %

39. Callback rate for résumés with African-American names: %

40. Difference between groups: %

41. The key question: Is this difference due to alone, or is it because of the ?

Part D: Statistical Significance

[3:00–4:44]

42. Random assignment allows us to conclude that very large observed changes are not merely by (statistically significant).

43. Statistically significant differences between treatment groups are evidence that the caused the effect.

44. Two possibilities for the 3.38% difference:

• Option 1: Type of name does not affect callback rate; the difference happened by .

• Option 2: Type of name callback rate.

45. The probability of observing an outcome this extreme due to chance was p ≈ .

46. Conclusion: We have evidence that the type of name on a résumé affects the likelihood of a callback.

Part E: Statistical Inference & Generalization

[4:44–5:23]

47. Statistical inference: Decisions from the can be attributed to the from which the sample was drawn.

48. If experimental units are of the population, then results can be generalized to the population.

49. of individuals gives a better chance that the sample will be representative.

Part F: Key Takeaways

[5:23–5:50]

50. Statistical inference allows us to make about populations or treatments based on sample data.

51. Observed changes larger than can be attributed to chance are considered .

52. of experimental units allows results to be generalized to the population of interest.

Post-Video Reflection

53. Random Selection vs. Random Assignment: Explain the difference and when each is used.

54. Evaluating an Experiment: A researcher wants to test whether a new fertilizer increases crop yield. She has 10 fields available, each with different soil types. She applies the new fertilizer to 5 fields and the old fertilizer to 5 fields, then measures yield.

(a) What is a potential confounding variable in this study?

(b) Propose an improved design using blocking.

55. Interpreting Statistical Significance: In a study comparing two medications, researchers found a statistically significant difference in recovery times (p = 0.02). What does this mean, and what can we conclude about causation?

56. Generalization: Under what two conditions can the results of an experiment be generalized to a larger population?