This page is built to train the distinctions that matter in intro statistics: what counts as an experiment, what random assignment does that random sampling does not, how confounding breaks causal claims, and how design choices like placebo, blinding, matched pairs, and blocks improve inference.
Progress
0 of 42 questions marked correct.
Observational studies versus experiments
The first and most important split: observing versus imposing.
Observational studies versus experiments
The first and most important split: observing versus imposing.
Core ideas
- An observational study measures variables without trying to change the subjects.
- An experiment deliberately imposes a treatment and observes the response.
- Experiments are the preferred design when the goal is a cause-and-effect conclusion.
- Observational studies can describe associations well, but causal claims are much weaker because lurking variables may still drive the pattern.
Fast test
- If researchers only watch, measure, or ask, think observational.
- If researchers assign or impose a condition, think experiment.
- Cause-and-effect needs imposition plus good design, not just measurement.
Question 1 — Classify the study
Researchers record how many hours students sleep and compare it to GPA, but they do not tell anyone how much to sleep. Enter one answer: observational or experiment.
Question 2 — Classify the study
Researchers assign some patients to receive aspirin and others to receive no aspirin, then compare heart attack rates. Enter one answer: observational or experiment.
Question 3 — Cause-and-effect support
Type yes or no: A well-designed experiment gives a much stronger basis for cause-and-effect conclusions than an observational study.
Question 4 — Missing word
Type the missing word: Experiments deliberately impose some ______ on subjects to observe the response.
Question 5 — Observation only
A wildlife researcher watches wolves in the wild and records hunting behavior without interfering. Enter one answer: observational or experiment.
Question 6 — Exact distinction
Type the one word that best completes this sentence: Observational studies mainly help describe or detect association, while experiments are the stronger design for ______.
Subjects, factors, and treatments
Learn the exact pieces of experimental vocabulary.
Subjects, factors, and treatments
Learn the exact pieces of experimental vocabulary.
Core ideas
- The individuals studied in an experiment are the subjects.
- The explanatory variables are the factors.
- A treatment is a specific experimental condition applied to subjects.
- If an experiment has more than one factor, a treatment is a combination of specific values across those factors.
How to parse a design
- Ask who receives the condition: those are the subjects.
- Ask what is being changed: those are the factors.
- Ask what exact condition each subject gets: that is the treatment.
Question 7 — Identify the subjects
In an experiment on blood pressure medication, 120 adults are assigned to treatment conditions. Type the term for the 120 adults: subjects, factors, or treatments.
Question 8 — Identify the factor
In that experiment, the dosage level of medication is being changed. Type the term for dosage level: subjects, factor, or response.
Question 9 — Count treatments
One factor has 3 levels: low dose, medium dose, high dose. How many treatments are there if this is the only factor? Enter only the number.
Question 10 — Multiple-factor count
An experiment has 2 factors. Factor A has 2 levels and Factor B has 3 levels. If each treatment is one combination of levels, how many treatments are there? Enter only the number.
Question 11 — Exact term
Type the term for a specific experimental condition applied to subjects.
Question 12 — Identify what changed
Researchers compare Pepsi versus Coke in a blind taste test. Type the factor being manipulated in one or two words.
Confounding and bad experiments
Why uncontrolled comparisons fail even when they look persuasive.
Confounding and bad experiments
Why uncontrolled comparisons fail even when they look persuasive.
Core ideas
- Two variables are confounded when their effects on the response cannot be separated.
- Uncontrolled experiments can be just as misleading as observational studies if other influences differ across groups.
- If subjects choose their own condition, the treatment effect can be mixed up with preexisting differences.
- To study a treatment cleanly, the design has to control other influences rather than letting them ride along.
Fast confounding test
- If treatment groups differ for reasons other than treatment, suspect confounding.
- If self-selection occurs, treatment differences and subject differences are often mixed together.
- When effects cannot be distinguished, causal interpretation breaks.
Question 13 — Define confounding
Type the term for when the effects of two variables on a response cannot be distinguished from each other.
Question 14 — Self-selection problem
Students choose whether to take a course online or in person, and then the instructor compares their scores. Type yes or no: Is self-selection likely to create confounding here?
Question 15 — Pick the flaw
A school gives a new tutoring method only to students who volunteered for extra help. Later, their scores improve more than everyone else’s. Enter one answer: confounding or clean comparison.
Question 16 — Exact word
Type the missing word: A bad experiment can fail because the treatment is mixed up with ______ variables.
Question 17 — Comparative need
Type true or false: To pin down cause and effect, experiments should usually compare two or more treatment conditions rather than studying only one condition in isolation.
Question 18 — Why uncontrolled fails
Type the one word that best completes this statement: Uncontrolled experiments are weak because treatment effects can be ______ with other influences.
Random assignment and comparative experiments
Where experimental design becomes statistical rather than anecdotal.
Random assignment and comparative experiments
Where experimental design becomes statistical rather than anecdotal.
Core ideas
- A control group provides a basis for comparison.
- A randomized comparative experiment uses both comparison and random assignment of subjects to treatments.
- A completely randomized design assigns all subjects at random among all treatments.
- Random assignment aims to make the treatment groups similar before treatment begins.
What random assignment does
- It does not sample from a population. It allocates available subjects to treatments.
- It spreads subject differences across groups by chance.
- That makes treatment comparison cleaner and less biased.
Question 19 — Exact design term
Type the term for an experiment that uses both comparison of two or more treatments and random assignment of subjects to those treatments.
Question 20 — Completely randomized count
A completely randomized design has 90 subjects and 3 treatments with equal group sizes. How many subjects should be assigned to each treatment? Enter only the number.
Question 21 — Purpose of a control group
Type the one word that best completes this statement: A control group provides the main basis for ______.
Question 22 — Random assignment versus self-choice
Type better or worse: Allowing subjects to choose their own treatment is usually ______ than random assignment for drawing causal conclusions.
Question 23 — Random assignment target
Type the two-word phrase: Random assignment aims to create treatment groups that are ______ ______ before treatment begins.
Question 24 — Sham treatment idea
Type yes or no: A control treatment can sometimes be a sham treatment rather than literally “no treatment.”
Control, randomization, replication, and statistical significance
The three core principles plus the logic behind “statistically significant.”
Control, randomization, replication, and statistical significance
The three core principles plus the logic behind “statistically significant.”
Core ideas
- Control restricts the effects of lurking variables, often through comparison.
- Randomization uses chance to assign subjects to treatments.
- Replication uses enough subjects to reduce chance variation.
- An effect is statistically significant if it is so large that it would rarely occur by chance alone.
Logical chain
- Random assignment makes groups similar before treatment.
- Control helps equalize outside influences across groups.
- Replication reduces the impact of random noise.
- If the observed difference is still unusually large, chance alone is a weak explanation.
Question 25 — Principle name
Type the principle that uses chance to assign subjects to treatments.
Question 26 — Principle name
Type the principle that uses enough subjects to reduce chance variation.
Question 27 — Principle name
Type the principle that restricts the effects of lurking variables, often through comparison.
Question 28 — Why tiny groups fail
Type true or false: An experiment with one subject in each treatment group is highly vulnerable to chance variation.
Question 29 — Significance idea
Type the adjective used for an observed effect so large that it would rarely occur by chance alone.
Question 30 — Three principles count
How many basic principles of experimental design are emphasized here: control, randomization, and replication? Enter only the number.
Placebo, blinding, and realism limits
Why careful details matter even after random assignment is in place.
Placebo, blinding, and realism limits
Why careful details matter even after random assignment is in place.
Core ideas
- A placebo is a sham treatment with no active ingredient or intended effect.
- People can respond favorably simply because they believe they are receiving treatment.
- Blinding helps keep expectations from distorting the response or the measurement.
- Double-blind means neither the subjects nor the people directly evaluating outcomes know which treatment each subject received.
- Even strong experiments may have limited generalizability if the setting lacks realism.
Why this matters
- If subjects know what they received, beliefs can alter responses.
- If evaluators know what subjects received, measurement can shift.
- Blinding protects the comparison from these extra influences.
Question 31 — Exact term
Type the term for a sham treatment that has no active ingredient.
Question 32 — Why placebo is used
Type yes or no: Some patients can respond favorably even to a placebo.
Question 33 — Double-blind definition
Type true or false: In a double-blind experiment, neither the subjects nor the evaluators know which treatment was received.
Question 34 — Main risk if not blinded
Type the one word that best completes this statement: Without blinding, subjects’ or researchers’ ______ can influence results.
Question 35 — Generalization limit
Type true or false: Statistical analysis alone can always tell us exactly how far experimental results generalize to other settings.
Question 36 — One-word purpose
Type the one word most associated with making treatment identity unknown to subjects or evaluators.
Matched pairs and block designs
When full randomization is not the most precise design.
Matched pairs and block designs
When full randomization is not the most precise design.
Core ideas
- A matched pairs design compares treatments within closely matched subjects or within the same subject under two conditions.
- It uses randomization, but only within each pair rather than across the entire pool at once.
- A block is a group of individuals known in advance to be similar in a way expected to affect response.
- In a block design, random assignment is carried out separately within each block.
- Blocking is another form of control because it binds an outside variable into the design rather than letting it blur the result.
Fast distinction
- Matched pairs is a special case of blocking.
- Blocks group similar subjects before random assignment.
- This often improves precision by reducing variation not caused by treatment.
Question 37 — Exact term
Type the design in which two similar subjects are paired and randomization occurs within each pair.
Question 38 — Exact term
Type the term for a group of individuals known before an experiment to be similar in a way expected to affect response.
Question 39 — Randomization location
Type within or across: In a block design, random assignment is carried out separately ______ each block.
Question 40 — Relationship of designs
Type yes or no: Matched pairs can be viewed as one kind of block design.
Question 41 — Main advantage
Type the one word that best completes this: Matching and blocking often improve ______ by reducing irrelevant variation among subjects.
Question 42 — Count the blocks
If 24 subjects are organized into matched pairs, how many pairs are there? Enter only the number.