Unit 1 · Topic 1.10 · about 25 minutes
The Investigative Question Revisited and Data Collection
Identify how a study collected its data, name its variables and treatments, say who its results apply to, and write a complete investigative question for it.
Predict first
A large survey finds that teens who eat breakfast every day have higher grades than teens who usually skip it. Does this show that eating breakfast raises grades?
Three ways to collect data
How the data were collected decides what you are allowed to conclude.
- A census records information from every item or individual in a population, instead of from a sample.
- An observational study records the values of variables without imposing any treatment. The researcher only observes and records.
- An experiment assigns conditions, called treatments, to the units in the study, then measures what happens.
A census is about who is measured, not about treatments, so a census that imposes nothing is also observational. When every member of the population is measured, call it a census.
To tell an experiment from an observational study, ask whether the researcher decided which treatment each individual got. If so, it is an experiment. If the individuals arrived with their values already, as breakfast eaters and breakfast skippers did, it is an observational study.
Sort it
Census, observational study or experiment? Pick the most specific label: tap a study, then tap its bin.
Census
Observational study
Experiment
The language of experiments
- The experimental units are the individuals or objects that the treatments are assigned to. When they are people, they are often called subjects or participants.
- An explanatory variable, also called a factor, is a variable whose categories, or levels, are imposed on the experimental units.
- The treatments are the conditions actually applied. With one explanatory variable, each level is a treatment. With two or more, each combination of levels is a treatment.
- The response variable is the outcome measured on each experimental unit after its treatment.
Suppose a psychologist studies memory with two explanatory variables: background sound (silence, white noise or music) and study time (10 or 20 minutes). Three levels of one and two levels of the other make six treatments. After studying, each participant writes down as many words from a 30-word list as possible, and the number recalled is the response variable.
| Background sound | Study for 10 minutes | Study for 20 minutes |
|---|---|---|
| Silence | Treatment 1 | Treatment 2 |
| White noise | Treatment 3 | Treatment 4 |
| Music | Treatment 5 | Treatment 6 |
Observational studies and confounding
Observational studies come in a few forms. A prospective study selects its units at one point in time and gathers data from then on into the future, such as following 1,000 newborns for ten years. A retrospective study selects its units at one point in time and gathers data from their past, such as reviewing medical records. A survey is an observational study that collects data from people with a standard set of questions.
Observational studies share one weakness. In an observational study, the explanatory variable is the one you suspect explains the response, such as eating breakfast. Its values are recorded, not imposed. A confounding variable offers another explanation for the relationship a study observes between the explanatory variable and the response. To confound, a variable must be associated with both of them. In the breakfast survey, a regular bedtime fits: it is linked to whether a teen eats breakfast, and on its own it is linked to grades. When people choose their own groups, a confounding variable can never be ruled out, so an observational study by itself cannot show cause and effect.
Who do the results describe?
A sample is random when every unit in it was selected by a random mechanism, such as a random number generator. When the units in a study were randomly selected from a population, it is appropriate to generalize the results to that whole population.
A sample is not random when its units were deliberately chosen or volunteered themselves. Then the results apply only to individuals similar to those in the study. A study of 60 volunteers from one gym speaks for people like those volunteers, not for all adults.
Cause and effect is a separate question with a separate answer. It depends on random assignment of treatments, which is the center of 1.13 Experimental Design.
The investigative question, in full
1.1 Introducing Statistics: What Can We Learn from Data? set two rules for an investigative question: its purpose is fixed before the data come in, and it asks for data that can be collected and analyzed. A complete investigative question also has three components.
- The variables. It names the variable or variables of interest, which guides what data to collect.
- The parameter and the goal. It guides the analysis. If the plan is a significance test, which you will meet in Unit 3, the question names the parameter and the direction of the claim: greater than, less than, not equal to, or an association between two variables. If the plan is a confidence interval, it names the parameter and makes clear that the goal is to estimate it within a range of plausible values.
- The conclusion. It names the population the conclusion will apply to and, for an experiment that uses random assignment, makes clear that the conclusion is about cause and effect.
Worked exampleWriting an investigative question
A cafeteria manager believes that more than half of the students at Westview High would buy a breakfast burrito if it were sold before first period. She plans to survey a random sample of Westview students. Write an investigative question with all three components.
Variable. Whether a student would buy a breakfast burrito sold before first period. It is categorical: yes or no.
Parameter and direction. The parameter is the proportion of all Westview students who would buy the burrito. Her belief is more than half, so the direction is greater than 0.50.
Population and conclusion. The population is all students at Westview High. A random sample with no treatment imposed supports a conclusion about that population, not a cause-and-effect claim.
Put it together. Is there convincing evidence that the proportion of all Westview High students who would buy a breakfast burrito sold before first period is greater than 0.50?
The question names the variable, the parameter (a proportion) with its direction (greater than 0.50), and the population (all Westview High students).
Check your understanding
Researchers select 2,000 adults today, look through old school records to find how many sports each one played in high school, and compare that with their current health. What kind of study is this?
A baker tests how oven temperature (325, 350 or 375 degrees) and type of flour (all-purpose or whole wheat) affect the height of muffins. How many treatments are there?
An observational study finds that people who own a dog walk more steps per day than people who do not. Which variable could be a confounding variable?
A university recruits 150 student volunteers through an ad and records each one's hours of sleep and exam scores. To whom can the results be generalized?
A coach believes that the mean mile time is different for students on her team who train in the morning and students on her team who train after school. Which investigative question matches her belief and includes all three components?
Course alignment, for teachers
AP Statistics topic 1.10, Unit 1: Exploring One-Variable Data and Collecting Data.
- Skill 1.A: Determine a valid investigative question that requires a statistical investigation.
- Skill 2.A: Identify information to answer a question or solve a problem.
- Skill 2.B: Justify an appropriate method for ethically gathering and representing data.