This is pages 285- 287 from the text (referring to the activities)

In an experiment, participants are selected from one population, then randomly assigned to groups.

The Use of the Test Statistic

Once participants have been assigned to groups, we conduct the experiment and measure the same dependent variable in each group. For example, suppose we test the hypothesis that music can inspire greater creativity. Studies are quite common in this area of research (see He, Wong, & Hui, 2017; Kokotsaki, 2011; H. Newton, 2015). To test this hypothesis, we can select a sample of participants from a single population and randomly assign them to one of two groups. In Group Music, participants listen to classical music for 10 minutes; in Group No Music, different participants listen to a lecture about music for 10 minutes. Listening to classical music versus a lecture is the manipulation. After the manipulation, participants in both groups are given 5 minutes to write down as many uses as they can think of for a paper clip. If the hypothesis is correct, then Group Music should come up with more practical uses for a paper clip than Group No Music. The number of practical uses for a paper clip, then, is the dependent variable measured in both groups.

To compare differences between groups, we will compute a test statistic, which is a mathematical formula that allows us to determine whether the manipulation (music vs. no music) or error variance (other factors attributed to individual differences) is likely to explain differences between the groups. In most cases, researchers measure data on an interval or a ratio scale of measurement. In our example, the number of practical uses for a paper clip is a ratio scale measure. In these situations, when data are interval or ratio scale, the appropriate test statistic for comparing differences between two independent samples is the two-independent-sample t test. This test statistic follows a common form:

A two-independent-sample t test, also called an independent-sample t test, is a statistical procedure used to test hypotheses concerning the difference in interval or ratio scale data between two group means, in which the variance in the population is unknown.

𝑡=Mean differences between groups

    Mean differences attributed to error

The numerator of the test statistic is the actual difference between the two groups. For example, suppose that participants in Group Music came up with five practical uses for a paper clip on average, and Group No Music came up with two practical uses on average. The mean difference, then, between the two groups is 3 (5 − 2 = 3). We divide the mean difference between two groups by the value for error variance in the denominator. The smaller the error variance, the larger the value of the test statistic will be. In this way, the smaller the error variance, or the less overlap in scores between groups, the more likely we are to conclude that the manipulation, not factors attributed to individual differences, is causing differences between groups. To illustrate further, we will work through this example using SPSS.

10.7 SPSS in FocusTwo-Independent-Sample t Test

In Section 10.4, we used data originally given in Figure 10.4 to illustrate that the more overlap in scores between groups, the larger the error variance. We will use these same data, reproduced in Table 10.3, and assume that they represent the number of practical uses for a paper clip from the classical music and creativity study. We will use SPSS to compute a two-independent-sample t test for each data set given in Table 10.3: one test for the no-overlap example and one test for the overlap example.

1. Click on the Variable View tab and enter Groups in the Name column. In the second row, enter No-Overlap in the Name column. In the third row, enter Overlap in the Name column. We will enter whole numbers in each column, so reduce the value to 0 in the Decimals column in each row. We can also define the scale of measurement for each variable. Go to the Measure column to select Nominal from the dropdown menu for Groups (because this is a categorical variable), then select Scale from the dropdown menu for No-Overlap and for Overlap.
2. In the first row (labeled Groups), click on the Values column and click on the small gray box with three dots. To label the groups, in the dialog box, enter 1 in the value cell and No Music in the label cell, and then click Add. Then enter 2 in the value cell and Classical Music in the label cell, and then click Add. Select OK.
3. Click on the Data View tab. In the first column (labeled Groups) enter, 1 five times, then 2 five times, which are the codes we entered in Step 2 for each group. In the second column (labeled NoOverlap), enter the scores for the No Music group next to the 1s and enter the scores for the Classical Music group next to the 2s for the no-overlap data given in Table 10.3 (left side). In the third column (labeled Overlap), enter the scores for the No Music group next to the 1s and enter the scores for the Classical Music group next to the 2s for the overlap data given in Table 10.3 (right side). Figure 10.7 shows how the data should appear.
4. Go to the menu bar and click Analyze, then Compare Means, and Independent-Samples T Test to bring up a dialog box, which is shown in Figure 10.8.
5. Use the arrows to move the data for NoOverlap and Overlap into the Test Variable(s): cell. SPSS will compute a separate t test for each of these sets of data. Select Groups and use the arrow to move this column into the Grouping Variable: cell. Two question marks will appear in that cell.
6. To define the groups, click Define Groups . . . to bring up a new dialog box. Enter 1 in the Group 1: box, and enter 2 in the Group 2: box, and then click Continue. Now a 1 and 2 will appear in the Grouping Variable box instead of question marks.
7. Select OK or select Paste and click the Run command.

Table 10.3 ⦁ Data to Enter Into SPSS

The data are reproduced from those given in Figure 10.4 (no-overlap example) and Figure 10.5 (overlap example).

The output table, shown in Table 10.4, gives the results for both data sets; key results are circled and described in the table. Read the first row of each cell because we will assume that the variances were equal between groups. In the Mean Difference column, notice that the mean difference between the two groups is the same for both data sets; the mean difference is −3.0. However, notice in the Std. Error Difference column that the error variance is much smaller for the no-overlap data. The mean difference is the numerator for the test statistic, and the Std. Error Difference (or error variance) is the denominator. If you divide those values, you will obtain the value of the test statistic, given in the t column. The Sig. (2-tailed) column gives the p value, which is the likelihood that individual differences, or anything other than the music manipulation, caused the 3-point effect. The results show that when scores do not overlap between groups, the likelihood that individual differences explain the 3-point effect is p = .001; however, when scores do overlap between groups, this likelihood is much larger, p = .105.

Description

Figure 10.7 ⦁ SPSS Data View for Step 3

The criterion in the behavioral sciences is p ≤ .05. When p ≤ .05, we conclude that the manipulation caused the effect because the likelihood that anything else caused the effect is less than 5%. When p > .05, we conclude that individual differences, or something else, caused the effect because the likelihood is greater than 5% that something else, typically attributed to individual differences, is indeed causing the effect. In this way, the smaller the error variance or overlap in scores between groups, the more likely we are to conclude that differences between groups were caused by the manipulation and not individual differences.

Description

Figure 10.8 ⦁ SPSS Dialog Box for Steps 4 to 6

Also given in Table 10.4, the two-independent-sample t test is associated with N − 2 degrees of freedom, in which N is the total sample size. We report the results of a t test in a research journal using guidelines given in the Publication Manual of the American Psychological Association (APA, 2020). Using these guidelines, we report the results computed here by stating the value of the test statistic, the degrees of freedom (df), and the p value for each t test as shown:

A two-independent-sample t test showed that classical music significantly enhanced participant creativity when the data did not overlap, t(8) = −4.743, p = .001; the results were not significant when the data did overlap, t(8) = −1.826, p = .105.

Description

Table 10.4 ⦁ SPSS Output Table for the Two-Independent-Sample t Test

To read the table, assume equal variances. Notice that the error variance is smaller when scores do not overlap between groups, thereby making the value of the test statistic larger.

Adding groups can allow for more informative conclusions of observed results.

This is pages 291- 293 in the text (referring to the activities)

The Use of the Test Statistic

Once participants have been assigned to groups, we conduct the experiment and measure the same dependent variable in each group. For example, suppose we want to test the hypothesis that gym patrons will crave more high-fat foods after an intense workout, compared with an easy or moderate aerobic workout. To test this hypothesis, we could create the three exercise levels (easy, moderate, or intense) and randomly assign patrons to each group.

To compare differences between groups, we will compute a test statistic, which allows us to determine whether the manipulation (easy, moderate, or intense workout) or error variance (other factors attributed to individual differences) is likely to explain differences between the groups. In most cases, researchers measure data on an interval or a ratio scale of measurement. In our example, the number of high-fat foods chosen is a ratio scale measure. In these situations, when data are interval or ratio scale, the appropriate test statistic for comparing differences among two or more independent samples is the one-way between-subjects analysis of variance (ANOVA). The term one-way indicates the number of factors in a design. In this example, we have one factor or independent variable (type of workout). This test statistic follows a common form:

The one-way between-subjects ANOVA is a statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. This test is used when different participants are observed at each level of a factor and the variance in a given population is unknown.

𝐹=Variability between groups

      Variability attributed to error

An ANOVA is computed by dividing the variability in a dependent measure attributed to the manipulation or groups, by the variability attributed to error or individual differences. When the variance attributed to error is the same as the variance attributed to differences between groups, the value of F is 1.0, and we conclude that the manipulation did not cause differences between groups. The larger the variance between groups relative to the variance attributed to error, the larger the value of the test statistic, and the more likely we are to conclude that the manipulation, not individual differences, is causing an effect or a mean difference between groups.

The one-way between-subjects ANOVA informs us only that the means for at least one pair of groups are different—it does not tell us which pairs of groups differ. For situations in which we have more than two groups in an experiment, we compute post hoc tests or “after the fact” tests to determine which pairs of groups are different. Post hoc tests are used to evaluate all possible pairwise comparisons, or differences in the group means between all possible pairings of two groups. In the exercise and food cravings experiment, we would use the one-way between-subjects ANOVA to determine if the manipulation (easy, moderate, or intense workout groups) caused the mean number of high-fat foods that participants craved to be different or to vary between groups. We would then compute post hoc tests to determine which pairs of group means were different. To illustrate further, we will work through this research example using SPSS. The data for this example, as well as steps for analyzing these data, are described in Figure 10.11.

A post hoc test is a statistical procedure computed following a significant ANOVA to determine which pair or pairs of group means significantly differ. These tests are needed with more than two groups because multiple comparisons must be made.

A pairwise comparison is a statistical comparison for the difference between two group means. A post hoc test evaluates all possible pairwise comparisons for an ANOVA with any number of groups.

Reducing error variance increases power—or likelihood of observing an effect, assuming it exists.

10.9 SPSS in Focus One-Way Between-Subjects ANOVA

We will use SPSS to compute the one-way between-subjects ANOVA for the data given in Figure 10.11 in Step 1. For these data, we will test the hypothesis that patrons at a gym will crave more high-fat foods after an intense aerobic workout, compared with an easy or moderate aerobic workout. There are two commands that we could use to analyze these data; we will use the One-Way ANOVA command.

1. Click on the Variable View tab and enter Groups in the Name column. In the Values column, click on the small gray box with three dots. To label the groups, in the dialog box, enter 1 in the Value cell and Easy in the Label cell, then click Add. Then enter 2 in the Value cell and Moderate in the Label cell, and then click Add. Then enter 3 in the Value cell and Intense in the Label cell, and then click Add. Then click OK.
2. Still in the Variable View, enter Foods in the Name column in the second row. Reduce the value to 0 in the Decimals column for each row. Go to the Measure column to select Nominal from the dropdown menu for Groups (because this is a categorical variable), then select Scale from the dropdown menu for Foods.
3. Click on the Data View tab. In the first column (labeled Groups), enter 1 five times, 2 five times, and 3 five times, which are the codes we entered in Step 2 for each group. In the Foods column, enter the data for each group as shown in Figure 10.12a.
4. Go to the menu bar and click Analyze, then Compare Means, and One-Way ANOVA to bring up the dialog box shown in Figure 10.12b.
5. Using the appropriate arrows, move Groups into the Factor: box. Move Foods into the Dependent List: box.
6. Click the Post Hoc option to bring up the new dialog box shown in Figure 10.12c. Select Tukey, which is a commonly used post hoc test. Click Continue.
7. Select OK, or select Paste and click the Run command.

The output table, shown in Table 10.5, gives the results for the one-way between-subjects ANOVA. The numerator of the test statistic is the variance between groups, 46.667, and the denominator is the variance attributed to individual differences or error, 2.167. When you divide those two values, you obtain the value of the test statistic, 21.538. The Sig. column gives the p value, which in our example shows that the likelihood that anything other than the exercise manipulation caused differences between groups is p < .001. We decide that the group manipulation caused the differences when p < .05; hence, we decide that the manipulation caused group differences. However, remember that this result does not tell us which groups are different; it tells us only that at least one pair of group means differ significantly.

To determine which groups are different, we conducted post hoc tests, shown in Table 10.6. On the left, you see Easy, Moderate, and Intense labels for the rows. You read the table as comparisons across the rows. The first comparison on the first line in the table is Easy and Moderate. If there is an asterisk next to the value given in the Mean Difference column, then those two groups significantly differ (note that the p value for each comparison is also given in the Sig. column for each comparison). The next comparison is Easy and Intense in the top left boxed portion of the table. For all comparisons, the results show that people choose significantly more high-fat foods following an intense workout, compared with a moderate and an easy workout.

Also given in Table 10.5, the one-way between-subjects ANOVA is associated with two sets of degrees of freedom (df): one for the variance between groups and one for the variance attributed to error. Using APA (2020) guidelines, we report the results of an ANOVA by stating the value of the test statistic, both degrees of freedom, and the p value for the F test, and indicate the results of the post hoc test if one was computed as shown:

A one-way between-subjects ANOVA showed that the number of high-fat foods chosen significantly varied by the type of workout participants completed, F(2, 12) = 21.538, p < .001. Participants chose significantly more high-fat foods following a moderate or intense workout compared to an easy workout (Tukey’s honestly significant difference, p ≤ .003).

Description

Figure 10.11 ⦁ The Steps for Analyzing Differences Between More Than Two Group Means