2.1: What's Different About These Questions? (5 minutes)
This warm-up prompts students to compare four questions. It gives students a reason to use language precisely (MP6) and provides the opportunity to talk about characteristics of the items in comparison to one another. Monitor for students who mention numerical and categorical data as well as those who refer to the variability of the data to determine whether the question is statistical.
Arrange students in groups of 2–4. Display the questions for all to see. Give students 1 minute of quiet think time and then time to share their thinking with their small group.
For each question, determine whether it is a statistical question. If it is a statistical question, determine whether an experimental study, observational study, or survey would be best at providing data to answer the question. Explain your reasoning.
- Do dogs who eat only Brand A of dog food have more health problems than those who eat a variety of food brands?
- Do people who sit for at least 8 hours per day have more health problems than those who sit for fewer than 8 hours per day?
- Which brand of dog food has the most protein per serving?
- Do people who eat a low-fat diet feel healthier than those who eat a variety of foods?
Record and display the responses for all to see. There is no single correct answer to the questions, so attend to students’ explanations and ensure the reasons given are correct.
During the discussion, ask students to explain the meaning of any terminology they use, such as survey study, statistical question, observational study, or experimental study. Also, press students on unsubstantiated claims.
Here are some questions for discussion.
- “What is a statistical question?” (A question that requires data to be collected and in which variation is expected in the data collected.)
- “What is the difference between an observational study and an experimental study?” (In an observational study, you collect the data without influencing things directly to answer your question. In an experimental study, you directly influence things and collect data to answer your question.)
- “What makes a survey a special type of observational study?” (In a survey, you ask people questions and their responses are the data. In other observational studies, the researcher observes people and the information they record about the observations becomes the data.)
2.2: Study Type Matching (15 minutes)
In this activity, students take turns with a partner recognizing the purposes of and differences among sample surveys, experiments, and observational studies. Students trade roles explaining their thinking and listening, providing opportunities to explain their reasoning and critique the reasoning of others (MP3).
Arrange students in groups of 2. Tell students that, for each question, one partner determines if a survey, observational study, or experimental study would be the best way to collect data to answer the question. The partner’s job is to listen and make sure they agree. If they don’t agree, the partners discuss until they come to an agreement. For the next question, the students swap roles. If necessary, demonstrate this protocol before students start working.
Display the statements for all to see.
- Surveys are sets of questions given to people to seek their responses.
- Observational studies collect data without influencing things directly.
- Experimental studies collect data by directly influencing something to determine how another thing is changed.
Design Principle(s): Support sense-making; Maximize meta-awareness
Supports accessibility for: Language; Organization
Take turns with your partner to determine whether a survey, observational study, or experimental study would be the best way to collect data to answer the question.
- For each study type that you match, explain to your partner why you think this is the best type of study.
- For each study that your partner matches, listen carefully to their explanation. If you disagree, discuss your thinking and work to reach an agreement.
- Do smokers get in more car accidents than non-smokers?
- What is the students’ favorite type of sport at this school?
- Do people who chew gum while studying do better on tests when they chew gum while taking the test than when they don’t chew gum while taking the test?
- How has the percentage of the world’s wealth owned by the top 1% of individuals changed over the past 300 years?
- Do strawberry plants produce more fruit when growing in a greenhouse or outside?
- What are the most important issues for voters in a district at the moment?
Some students may be confused about when to use an experiment or observational study. Ask students to recall the definition of the studies and point out that experiments require dividing the subjects into 2 groups and directly influencing one of the groups.
The mathematical purpose of this discussion is for students to recognize the purposes of and differences among sample surveys, experiments, and observational studies. Here are some questions for discussion:
- “How did you determine which type of study would be best for each situation?” (I really had to look at the question of interest and determine what would be needed to answer it. If observations would be enough, then I chose an observational study. If I needed to get people’s opinions, then a survey made sense. Finally, if I wanted to know one variable’s effect on another variable, then I chose an experimental study.)
- “What type of study is most appropriate for measuring one variable’s effect on another variable?” (Experimental)
- “What type of study are you likely to be conducting if you collect information about populations to learn about the population, compare populations, or compare variable influence within a population?” (An observational study)
- “Why is a survey a useful type of study?” (A survey is useful for collecting opinions or thoughts which cannot be measured otherwise. It can also be useful to find out more about the population you are studying by asking questions.)
2.3: Relaxing Television (15 minutes)
The mathematical purpose of this activity is for students to recognize the purposes of and differences among sample surveys, experiments, and observational studies, and to apply that knowledge in context. Students also have the chance to discuss how to interpret the results from a survey, including whether a causal relationship is likely present based on the study’s methods.
Tell students that they will be answering questions about a given study and will then design their own study.
Design Principle(s): Support sense-making
A study of 1,000 people aged 20–30 asked how much television each person watches each night and how each person would rate their energy level in the evenings. The study showed that people who watch television for at least 2 hours every night have lower energy in the evening than people who do not watch as much television.
- Is this study a survey, observational study, or experimental study? Explain your reasoning.
- Does this mean that watching television for at least 2 hours every night lowers energy in the evening? Explain your reasoning.
- If you were to do your own experiment to determine if watching television for at least 2 hours every night lowers energy in the evening, how would you set up the experiment?
Are you ready for more?
Sometimes it is not feasible or ethical for a researcher to directly influence factors as required for an experimental study. In those situations, a researcher may try to find what is called a natural experiment. While technically a form of observational study, in a natural experiment, the group’s assignment to treatments seems random so that it resembles an experimental study.
For example, in Oregon in the early 2000s, the state did not not have enough money to fund a health insurance program for everyone who would have been eligible for it, so after people applied, they randomly selected who would be allowed to enroll. Researchers could not ethically have given some people health insurance and denied it to others, but they followed up on the state’s actions to research the effect of having health insurance on a person’s health and well-being.
What are some statistical questions that researchers may be interested in studying, that could be answered with an experimental study, but for which a natural experiment may be necessary? Explain why an experimental study wouldn’t be feasible or ethical.
The goal of this discussion is to discuss the difference between association and causation, and the reasoning that goes into designing an experimental study. Here are some questions for discussion.
- “The given study showed that people who watch television for at least 2 hours every night have lower energy in the evening than people who do not watch as much television. Is this statement describing an association or causation? Explain your reasoning.” (It is describing an association. Using the survey data, we are able to see that people who watch more television report having less energy in the evening, but we do not know if that is caused by some other factor. Maybe people watch television because they have less energy or something else entirely.)
- “What did you include in your experimental design?” (I included two different groups, I assigned people to the groups at random, I recorded their energy levels at the start of the study, and I recorded the energy of each group at the end of the month.)
- “What are some ways to categorize the different factors that our class included in their experimental design?” (Treatment groups, control groups, time, sample size, pre-test, post-test, random selection, random assignment to groups.)
Here are some questions for discussion:
- “What type of study would you perform to test if one weight-lifting routine results in building strength faster than another weight-lifting routine? Explain your thinking.” (I think you would use an experimental study because you want to compare two different routines, the first variable, to how fast strength builds, the second variable. You might also want to use a group that does not do any weight lifting as a control group. If you already had two groups using each routine you could potentially do an observational study to see which group gains strength faster.)
- “What type of study would you conduct if you wanted to know how many hours high school students spend gaming on school nights? Explain your reasoning.” (You could do a survey and just ask students how much time they spend gaming on weeknights.)
- “Describe an example of an observational study.” (An example of an observational study would be to investigate the percentage of people who come to a complete stop at stop signs. You could do this by standing at an intersection that has a stop sign and observe the number of cars that come to a complete stop.)
- “Could the example given for the observational study be investigated with a survey? Explain your thinking.” (It could be, but you would probably have to change the question to something more like the percentage of people who say they come to a complete stop at stop signs.)
- “Could the example given for the observational study be investigated with an experimental study? Explain your thinking.” (An experimental study does not make sense here because you cannot compare how one variable relates to another variable.)
2.4: Cool-down - Beauty Is In The Eye Of The Beholder (5 minutes)
Cool-downs for this lesson are available at one of our IM Certified Partners
Student Lesson Summary
Three of the best ways to collect data are through surveys, experimental studies, and observational studies. Each method has advantages and disadvantages depending on the question you are trying to answer.
A statistical study begins with a research question, which describes what you want to know clearly and simply. Most research questions are questions about characteristics of a population or about the effect of one variable on another.
Sometimes researchers have a question about a population, like: “What percentage of the fish in Lake Erie are toxic for humans to eat?” Sometimes they have a question about how two or more populations compare, like: “Which lake, Lake Erie or Lake Ontario, has a higher percentage of fish that are toxic for humans to eat?” And sometimes, researchers want to change one variable and see how a population responds. For example, “Does taking a fish oil supplement daily help older adults maintain brain function?” After the research question is created, the researcher needs to collect some data.
There are three methods for collecting data: observational studies, surveys, and experimental studies. In an experimental study, the researcher deliberately does something to one or more groups of individuals, such as giving them access to tutoring or giving them a vitamin, and then measures their responses in comparison to another group that does a different thing, such as not going to tutoring or taking a different type of vitamin. This is different from observational studies and surveys, in which the researcher collects data about individuals as they are.
In an observational study, the researcher records values for one or more variables, like ZIP code or height, for each individual participating in the study. These values can be obtained by observation, measurements, or taken from existing data that has already been collected (like the U.S. Census).
A researcher can also collect data with a survey, in which they ask each participant to answer one or more questions.
Each method comes with advantages and limitations that the researcher must understand before planning a study. The method used to collect data also affects what kind of conclusions can be drawn. Choosing the best method for a research question takes careful thought and practice.