Lesson 13
Experimenting
13.1: Satisfaction Test (5 minutes)
Warmup
The mathematical purpose of this activity is for students to compare two distributions. Students may or may not claim there is an important difference between the two distributions for this activity. In the following activity, students will further analyze the data to determine whether there is a significant difference. Monitor for students who use the means (whether calculated or estimated) to compare the distributions.
Making spreadsheet technology available gives students an opportunity to choose appropriate tools strategically (MP5).
Launch
Arrange students in groups of 2. Identify students who calculate the mean and standard deviation or mention margin of error.
Student Facing
The dot plots represent the satisfaction ratings for two similar products resulting from a survey given to 5 randomly selected people who use product A and to 5 randomly selected people who use product B. The satisfaction rating is based on a scale of 1 to 5, where 1 is not satisfied, 2 is somewhat satisfied, 3 is satisfied, 4 is very satisfied, and 5 is extremely satisfied.
 Which product has a higher overall satisfaction rating? Explain your reasoning.
 Do you think that 2 different random samples of 5 people would lead you to the same conclusion?
Student Response
Student responses to this activity are available at one of our IM Certified Partners
Activity Synthesis
The purpose of this discussion is to introduce the idea of variability by chance when comparing two different groups. Ask previously identified students, “How did you use mean, standard deviation, and margin of error to explain the difference between the groups?” (I found out that the mean for product A was 4.4 and that product B had a mean of 3.6. The standard deviation for product A was slightly lower than the standard deviation for product B. I felt like that was enough information to show that product A had a higher product satisfaction rating, but then I started thinking about standard deviation and the margin of error and I wondered if 4.4 and 3.6 were representative of the population mean.) Ask students to discuss their answers to question 2 with their partner.
Here are some questions for discussion:
 “Do you think the results of the survey would be the same if it was repeated?” (I am not sure. The sample size is small and I have no idea about the composition of the larger population.)
 “What could you do to get a better idea of how the two products compare using the same survey?” (You could increase the sample size or take multiple samples.)
 “Do you think some of the differences between the two data sets is due to chance? Explain your reasoning.” (Yes, I think that some of it is due to chance because there are only 5 values in each data set so there is a chance that the 5 values are not representative of the larger population.)
13.2: Randomizing Satisfaction (20 minutes)
Activity
The mathematical purpose of this activity is for ten people in the class to perform a simulation to analyze the data from the warmup. Students reorganize the original data into two groups using a chance process to determine how likely it might be that the original results are due to the way the original groups were organized.
Making statistical technology available gives students an opportunity to choose appropriate tools strategically (MP5).
Launch
Tell students, “We are going to create a randomization distribution and I will assign 10 students to represent the two groups of 5 people surveyed in the previous activity.” Select 10 students and distribute a slip from the blackline master representing a data point to each one. Arrange students so that the students with slips with an A are on one side of the room (the first group) and those with a slip with a B are on the other side of the room (the second group). Ask each group to find the mean for the values in their group. Display the means and the difference between the means for all to see.
Remind students to hold on to their slips. Tell students, “I have a bag containing 5 slips that say ‘first group’ and 5 slips that say ‘second group.’” Rearrange students by having each one select one of the slips from the opaque bag at random go to the appropriate side of the room. Find the mean of the first group and the second group. Tell students to record these values in the table. Collect the slips that assign the groups and return them to the bag. Repeat this process 9 more times for a total of 10 trials. Tell students to complete the activity.
Student Facing
Your teacher will select 10 of your classmates to create a randomization distribution using the data from the warmup.

Complete the table using the data from the activity.
trial group 1's mean group 2's mean (group 1's mean) minus (group 2's mean) actual 4.4 3.6 0.8 1 2 3 4 5 6 7 8 9 10 
Complete the dot plot to display the distribution of the differences of the means.
 What information is represented in the dot plot?
 In what percentage of the trials are the means from the two groups at least as far apart as the actual groupings from the warmup?
Student Response
Student responses to this activity are available at one of our IM Certified Partners
Student Facing
Are you ready for more?
Here are 2 lists of scores 5 individuals got on a memory test. List \(A\) contains the scores of the 5 individuals when taking the test after listening to pop music. List \(B\) contains the scores of the same 5 individuals (in the same order) when taking the test after listening to classical music.
 List A: 59, 28, 73, 58, 44
 List B: 75, 93, 13, 21, 70
 Find the mean of list \(A\), the mean of list \(B\), and the difference mean of list \(A\) minus mean of list \(B\).
 What does this difference of means represent in our context?
 Create a third list, list \(C\) which has 5 numbers each of which are the differences of a number in list \(A\) and its corresponding number in list \(B\). So for example, the first element of list \(C\) is 16 since \(\text 16=5975\). Find the mean of list \(C\).
 What does this means of differences represent in our context?
 What is the connection between the difference of means and the mean of differences? Explain why this is true.
Student Response
Student responses to this activity are available at one of our IM Certified Partners
Anticipated Misconceptions
Students may subtract the means so that the differences are always positive. Tell students that the difference between the means can be negative. If the differences are only recorded as positive, some features of the distribution may be missed.
Activity Synthesis
The purpose of this discussion is for students to understand and interpret a randomization distribution. Ask, “Do you think that the difference between the means of product A and product B happened by chance? Explain your reasoning.” (I am not sure if the difference is due to chance. If it occurred 30% of the time by chance in our simulation I am not feeling so confident that it did not occur by chance. However, 10 feels like it was not enough trials to know how often we would really expect to see 0.8 occur.)
Here is a dot plot and table showing the same simulation repeated 200 times. Display the dot plot and table for all to see and give students quiet think time.
difference between the means  frequency 

1.6  3 
1.2  9 
0.8  24 
0.4  44 
0  49 
0.4  36 
0.8  22 
1.2  12 
1.6  1 
Here are some questions for discussion:
 “In what percentage of the trials are the means from the two groups 0.8 or more apart?” (35.5% of the time since \(\frac{3+9+24+22+12+1}{200} = 0.355\)).
 “Based on the percentage of trials, is it reasonable to say that the original difference between the groups is due to chance rather than the actual products?” (Yes, that seems reasonable. Since more than one third of the trials are at least as different as the original grouping, it is reasonable to suspect that the difference in average rating is due to chance rather than the products themselves.)
 “If the difference between the satisfaction ratings between product A and product B had been 1.2, would you have been more or less confident that this was due to chance? Explain your reasoning.” (I would have been much less confident because a difference of 1.2 happens 12.5% of the time.)
 “How do you think this data would change if the number of trials was 500 instead of 200?” (I think that the data would change a little, but 200 seems like enough trials to understand the likelihood of a difference of at least 0.8 occurring at random. 500 trials might yield a more accurate percentage, but would also take longer to do.)
Design Principle(s): Support sensemaking; Cultivate conversation
Supports accessibility for: Language; Socialemotional skills
13.3: Get Ready to Experiment (10 minutes)
Activity
The mathematical goal of this activity is to understand the importance of randomness in experimental design. In a future lesson, students will collect data and analyze the results of the experiment. In this activity, students assess methods for putting subjects into groups and design an experiment to test whether a treatment will affect the results. Students should learn that a treatment is the variable that is changed between two groups in an experiment.
Launch
Tell students that they will do an experiment involving heart rates in a later lesson. When students finish the question asking about methods for dividing the class, ask students to pause. Select students to share their responses and reasonings for how to divide the class. Discuss the drawbacks of the methods.
Design Principle(s): Support sensemaking
Student Facing
Does counting while exercising affect your heart rate? Let’s think about how to design an experiment to find out.
 For another lesson, the class will be divided into 2 groups. One group will do an exercise silently. The other group will count out loud while they do the exercise. Which of these methods would be good for dividing the class so the results are based only on the counting and heart rate rather than other factors? Explain your reasoning for each method suggested.
 The athletes in the class are assigned to the counting group and the nonathletes are assigned to the silent group.
 The teacher puts everyone’s name in a bag and draws half of the names. The names that are drawn are in the group that counts, and the others remain silent for the exercise.
 The tallest half of the students are put in the counting group and the shortest half of the students are assigned to the silent group.
 Do you think counting out loud will have an effect on heart rate? Explain your reasoning.
 A treatment is the value of the variable that is changed between the two groups in an experiment. What is the treatment in this experiment?
 How would you design an experiment to answer the question, “Does counting while exercising affect your heart rate?”
Student Response
Student responses to this activity are available at one of our IM Certified Partners
Activity Synthesis
The goal of this activity is to get students thinking about the importance of randomness in an experimental design. Here are some questions for discussion:
 “Why is it important to divide students into separate groups using a random process?” (If you don’t use a random process, you won’t be sure if the differences between the groups are due to the treatment or the way you divided the groups.)
 “What other methods could be used to randomly assign subjects to two groups?” (Each student could grab an item from a bag in which half of the items are labeled “count” and the other half “silent.”)
Write student names on papers, put them in a bag, and shake the bag to mix the papers. Draw half of the names to be included in one group. If time permits, do the drawing in front of the students rather than beforehand so they can see the random process used to divide students into groups (although the bag could be set up before class). Record the names for the counting group and the silent group to be used in the next lesson.
Ask, “Do you think that the way the two groups were chosen was done using a random process?” (Yes, because we chose our names out of a bag without looking.)
Supports accessibility for: Conceptual processing; Language
Lesson Synthesis
Lesson Synthesis
Here are some questions for discussion:
 “If someone asked you to explain the importance of assigning groups at random in an experiment, how would you explain it to them?” (The random assignment helps focus the experiment on the treatment variable rather than other factors. For example, imagine you want to test if people feel happier after drinking a smoothie by creating a group that drinks a smoothie and another group that drinks ice water. You would not want to assign people who drink smoothies regularly to one group and people who do not drink smoothies to another group because then you would not know if the results were due to the differences in the groups or due to the type of beverage they drank. When the people are assigned to treatment groups randomly, other factors like gender, weight, drink preference, and so on are distributed among the groups by chance to reduce their effect on one group over the other.)
 “Why do we perform simulations? What can they tell us about the data?” (Simulations allow us to determine how likely it is that the observed difference in means occurred by chance. It can help the data analyst provide evidence that the difference between the treatment groups is actually due to the treatment rather than how the participants were assigned.)
 “How do you interpret the results of a simulation?” (Look at the distribution of differences between the means of the groups. If the original difference between the actual treatment groups is unusual based on the distribution, then it is likely due to the treatments rather than the group assignment.)
13.4: Cooldown  Speedy Ladybugs (5 minutes)
CoolDown
Cooldowns for this lesson are available at one of our IM Certified Partners
Student Lesson Summary
Student Facing
Experiments provide a way to assess the effect of different experimental conditions on a response variable. It is important to design the experiment carefully to ensure that other variables that might have an effect on the response are accounted for.
One important aspect in experimental design is the use of randomness for separating subjects into groups for the experiment. The random assignments are used in order to create groups that are likely to be similar with respect to other variables that might affect the response.
For example, a researcher wants to study the question, “Does the size of the aquarium in which frogs are kept affect the size of the frogs?”
The researcher selects 20 young frogs to be used in the experiment. The frogs are then numbered and a random number generator is used to separate the frogs into two groups of 10. One group will be put in a 10 gallon aquarium to grow while the other group is placed in a 100 gallon tank to grow. The researcher will measure the size of the frogs based on their weight at the end of a year.
The size of the tank is one of the variables and a treatment is the value of the variable that is changed between the two groups in an experiment. In this experiment, there are two treatments—a smaller tank and a larger tank.
The researcher finds that the frogs in the smaller tank have a mean weight of 111.2 grams and the frogs in the larger tank have a mean weight of 169.3 grams. The difference of 58.1 grams seems large, but is it enough to say the tank size is the cause of the difference? Even if all the frogs were in the same tank, we expect there to be some variability in the weight of the frogs. A simulation can be used to estimate how much of the difference between the means occurs by chance.
In this kind of simulation, all the data is grouped together, then data is randomly redistributed among two groups, and the difference between the means of these new groups is computed. This process is repeated several times to create a randomization distribution: a distribution of the differences between the means for the treatment groups containing randomly redistributed data. The mean difference from the experiment is then compared to this distribution to determine whether or not it likely occurred by chance.
To investigate whether the tank size is the important factor in the difference or whether it is due to the chance ways the frogs were separated into treatment groups, we can use a simulation to examine what results we might expect from chance. For the moment, we assume the tank does not play a part in the size of the frogs, and we put all 20 frog weights into one group. We can separate the weights into two groups in a random way and determine the difference in mean weights between the groups.
Doing this many times will produce a distribution of weight differences that could be the result of how the frogs happened to be divided into groups. We can then compare the actual difference in mean weights from the treatment groups to the randomization distribution to determine if the 58.1 gram difference is unusual (which would suggest the tank size played an important role) or whether this is typical of what we might see from random assignment to groups.
Breaking the data into two groups randomly 30 times produces the histogram here.
Notice that, based on this distribution, the original difference of 58.1 grams would be very unusual based on what might be seen due to random assignment to groups. The researcher has evidence to support the hypothesis that tank size has an impact on the weight of frogs growing in it.
If the original difference had been about 15 grams, there would have been a case to be made that the difference observed might be due to random chance since 11 out of the 30 differences found from mixing the data randomly had a difference of at least 15 grams (in one direction or the other).