Lesson 13

Experimenting

  • Let’s do an experiment.

13.1: Satisfaction Test

The dot plots represent the satisfaction ratings for two similar products resulting from a survey given to 5 randomly selected people who use product A and to 5 randomly selected people who use product B. The satisfaction rating is based on a scale of 1 to 5, where 1 is not satisfied, 2 is somewhat satisfied, 3 is satisfied, 4 is very satisfied, and 5 is extremely satisfied.

Dot plot from 0 to 5 by 1’s. Satisfaction rating for product A. Number of dots above each increment is 0, 0, 0, 1, 1, 3.
Dot plot from 0 to 5 by 1’s. Satisfaction rating for product B. Number of dots above each increment is 0, 0, 1, 1, 2, 1.
  1. Which product has a higher overall satisfaction rating? Explain your reasoning.
  2. Do you think that 2 different random samples of 5 people would lead you to the same conclusion?

13.2: Randomizing Satisfaction

Your teacher will select 10 of your classmates to create a randomization distribution using the data from the warm-up.

  1. Complete the table using the data from the activity. 
     

    trial group 1's mean group 2's mean (group 1's mean) minus (group 2's mean)
    actual 4.4 3.6 0.8
    1      
    2      
    3      
    4      
    5      
    6      
    7      
    8      
    9      
    10      
  2. Complete the dot plot to display the distribution of the differences of the means.

    A dot plot from negative 1 point 6 to 1 point 6 by point 2’s. Differences of the means. No dots have been plotted yet.
  3. What information is represented in the dot plot?
  4. In what percentage of the trials are the means from the two groups at least as far apart as the actual groupings from the warm-up?


Here are 2 lists of scores 5 individuals got on a memory test. List \(A\) contains the scores of the 5 individuals when taking the test after listening to pop music. List \(B\) contains the scores of the same 5 individuals (in the same order) when taking the test after listening to classical music.

  • List A: 59, 28, 73, 58, 44
  • List B: 75, 93, 13, 21, 70
  1. Find the mean of list \(A\), the mean of list \(B\), and the difference mean of list \(A\) minus mean of list \(B\).
  2. What does this difference of means represent in our context?
  3. Create a third list, list \(C\) which has 5 numbers each of which are the differences of a number in list \(A\) and its corresponding number in list \(B\). So for example, the first element of list \(C\) is -16 since \(\text- 16=59-75\). Find the mean of list \(C\).
  4. What does this means of differences represent in our context?
  5. What is the connection between the difference of means and the mean of differences? Explain why this is true.

13.3: Get Ready to Experiment

Does counting while exercising affect your heart rate? Let’s think about how to design an experiment to find out.

  1. For another lesson, the class will be divided into 2 groups. One group will do an exercise silently. The other group will count out loud while they do the exercise. Which of these methods would be good for dividing the class so the results are based only on the counting and heart rate rather than other factors? Explain your reasoning for each method suggested.
    1. The athletes in the class are assigned to the counting group and the non-athletes are assigned to the silent group.
    2. The teacher puts everyone’s name in a bag and draws half of the names. The names that are drawn are in the group that counts, and the others remain silent for the exercise.
    3. The tallest half of the students are put in the counting group and the shortest half of the students are assigned to the silent group.
  2. Do you think counting out loud will have an effect on heart rate? Explain your reasoning.
  3. A treatment is the value of the variable that is changed between the two groups in an experiment. What is the treatment in this experiment?
  4. How would you design an experiment to answer the question, “Does counting while exercising affect your heart rate?”

Summary

Experiments provide a way to assess the effect of different experimental conditions on a response variable. It is important to design the experiment carefully to ensure that other variables that might have an effect on the response are accounted for.

One important aspect in experimental design is the use of randomness for separating subjects into groups for the experiment. The random assignments are used in order to create groups that are likely to be similar with respect to other variables that might affect the response.

For example, a researcher wants to study the question, “Does the size of the aquarium in which frogs are kept affect the size of the frogs?”

The researcher selects 20 young frogs to be used in the experiment. The frogs are then numbered and a random number generator is used to separate the frogs into two groups of 10. One group will be put in a 10 gallon aquarium to grow while the other group is placed in a 100 gallon tank to grow. The researcher will measure the size of the frogs based on their weight at the end of a year.

The size of the tank is one of the variables and a treatment is the value of the variable that is changed between the two groups in an experiment. In this experiment, there are two treatments—a smaller tank and a larger tank.

The researcher finds that the frogs in the smaller tank have a mean weight of 111.2 grams and the frogs in the larger tank have a mean weight of 169.3 grams. The difference of 58.1 grams seems large, but is it enough to say the tank size is the cause of the difference? Even if all the frogs were in the same tank, we expect there to be some variability in the weight of the frogs. A simulation can be used to estimate how much of the difference between the means occurs by chance.

In this kind of simulation, all the data is grouped together, then data is randomly redistributed among two groups, and the difference between the means of these new groups is computed. This process is repeated several times to create a randomization distribution: a distribution of the differences between the means for the treatment groups containing randomly redistributed data. The mean difference from the experiment is then compared to this distribution to determine whether or not it likely occurred by chance.

To investigate whether the tank size is the important factor in the difference or whether it is due to the chance ways the frogs were separated into treatment groups, we can use a simulation to examine what results we might expect from chance. For the moment, we assume the tank does not play a part in the size of the frogs, and we put all 20 frog weights into one group. We can separate the weights into two groups in a random way and determine the difference in mean weights between the groups.

Doing this many times will produce a distribution of weight differences that could be the result of how the frogs happened to be divided into groups. We can then compare the actual difference in mean weights from the treatment groups to the randomization distribution to determine if the 58.1 gram difference is unusual (which would suggest the tank size played an important role) or whether this is typical of what we might see from random assignment to groups.

Breaking the data into two groups randomly 30 times produces the histogram here.

Histogram from negative 45 to 45 by 10’s. Differences in mean weights for 2 groups. Height of each bar is 2, 2, 1, 5, 8, 4, 3, 3, 1.

Notice that, based on this distribution, the original difference of 58.1 grams would be very unusual based on what might be seen due to random assignment to groups. The researcher has evidence to support the hypothesis that tank size has an impact on the weight of frogs growing in it.

If the original difference had been about 15 grams, there would have been a case to be made that the difference observed might be due to random chance since 11 out of the 30 differences found from mixing the data randomly had a difference of at least 15 grams (in one direction or the other).

Glossary Entries

  • treatment

    In an experiment where you are comparing two groups, one of which is being given a treatment and the other of which is the control group without any treatment, the treatment is the value of the variable that is changed for the treatment group.