Description
In the natural world, the environment or situation you find yourself in can be dynamic. You need look no further than a college classroom. Suppose, for example, that you take a college exam in which the average student scored a 50%. Why were the exam grades so low? Was the professor ineffective in their teaching? Did the students study for the exam? Was the exam itself not fair? Was the material being studied too difficult or at too high a level? In this example, the answer can be difficult to identify because the environment is constrained by preexisting factors—the time, date, content, professor, and students enrolled in the course were not assigned by a researcher but instead were determined by the school and students. Accounting for these preexisting factors is important to determine why the exam grades were low.
In other situations, we can have difficulty obtaining large samples of participants. If a company were a small business, then it would have few employees, or if a behavioral disorder were rare, then it would afflict few people. In these cases, we probably could not obtain a large sample, so it would be advantageous to observe the behavior of only one or a few individuals. For example, we could observe an employee after a merger as new policy changes successively go into effect, or we could observe a patient’s health across multiple phases of treatment. In each case, we follow one participant and observe their behavior over time.
In this chapter, we introduce quasi-experimental designs used in science to make observations in settings that are constrained by preexisting factors. We also introduce many methods used to assess the behavior of a single participant or subject using single-case experimental designs, typically used when a large sample cannot be obtained.
QUASI-EXPERIMENTAL DESIGNS
UASI-EXPERIMENTAL DESIGNS
Suppose we hypothesize that high school graduates who attend college will value an education more than those who do not attend college. To test this hypothesis, we could select a sample of high school graduates from the same graduating class and divide them into two groups: those who attended college (Group College) and those who did not attend college (Group No College). We could then have all participants complete a survey in which higher scores on the survey indicate a higher value placed on obtaining an education. If the hypothesis is correct and we set up this study correctly, then participants in Group College should show higher scores on the survey than participants in Group No College.
Notice in this example that participants controlled which group they were assigned to—they either attended college or did not. Hence, in this example, the factor of interest (whether or not students attended college) was a quasi-independent variable. When a factor in a study is not manipulated (i.e., quasi-independent), this typically means that the study is a type of quasi-experimental research design. In this chapter, we separate the content into two major sections: quasi-experimental designs and single-case experimental designs. We begin this chapter with an introduction to the type of research design illustrated here: the quasi-experimental research design.
9.1 AN OVERVIEW OF QUASI-EXPERIMENTAL DESIGNS
In this major section, we introduce a common type of research design called the quasi-experimental research design. The quasi-experimental research design, also defined in Chapter 6, is structured similar to an experiment, except that this design does one or both of the following:
It includes a quasi-independent variable (also defined in Chapter 6).
It lacks an appropriate comparison/control group.
A quasi-experimental research design is the use of methods and procedures to make observations in a study that is structured similar to an experiment, but the conditions and experiences of participants lack some control because the study lacks random assignment, includes a preexisting factor (i.e., a variable that is not manipulated), or does not include a comparison/control group.
A quasi-independent variable is a preexisting variable that is often a characteristic inherent to an individual, which differentiates the groups or conditions being compared in a research study. Because the levels of the variable are preexisting, it is not possible to randomly assign participants to groups.
In the example used to introduce this section, the preexisting factor was college attendance (yes, no). The researchers did not manipulate or randomly assign participants to groups. Instead, participants were assigned to Group College or Group No College based on whether they attended college prior to the study. In other words, the participants, not the researcher, controlled which group they were assigned to. In this way, the study described to introduce this section was a quasi-experiment—the study was structured like an experiment in that differences in how students value college were compared between groups, but it lacked a manipulation (of the groups: whether students attended or did not attend college) and randomization (of assigning participants to each group).
Hence, a quasi-experiment is not an experiment because, as illustrated in Figure 9.1, the design does not meet all three requirements for demonstrating cause. In the college attendance study, for example, additional unique characteristics of participants, other than whether or not they attended college, could also be different between groups and therefore could also be causing differences between groups. For example, levels of motivation and academic ability may also be different between people who attend and do not attend college. When other possible causes cannot be ruled out, the design does not demonstrate cause.
Description
Figure 9.1 ⦁ A Simplified Distinction Between Experiments, Quasi-Experiments, and No experiments
The line represents the requirements for demonstrating cause: randomization, manipulation, and comparison/control. A quasi-experiment lacks at least one of these requirements and so fails to demonstrate cause.
In this major section, we introduce four categories of quasi-experimental research designs used in the behavioral sciences:
- One-group designs (posttest only and pretest-posttest)
- Nonequivalent control group designs (posttest only and pretest-posttest)
- Time series designs (basic, interrupted, and control)
- Developmental designs (longitudinal, cross-sectional, and cohort-sequential)
Quasi-experiments include a quasi-independent variable and/or lack a control group.
9.2 QUASI-EXPERIMENTAL DESIGN: ONE-GROUP DESIGNS
In some situations, researchers ask questions that require the observation of a single group. When only one group is observed, the study lacks a comparison group and so does not demonstrate cause; that is, the study is a quasi-experiment. Two types of one-group quasi-experiments are the following:
- One-group posttest-only design
- One-group pretest-posttest design
- One-Group Posttest-Only Design
The type of quasi-experiment most susceptible to threats to internal validity is the one-group posttest-only design, which is also called the one-shot case study (Campbell & Stanley, 1966). Using the one-group posttest-only design, a researcher measures a dependent variable for one group of participants following a treatment. For example, as illustrated in Figure 9.2, after a professor gives a lecture (the treatment), they may record students’ grades on an exam out of 100 possible points (the dependent variable) to test their learning.
The major limitation of this design is that it lacks a comparison or control group. Consider, for example, the exam scores following the lecture. If exam scores are high following the lecture, can we conclude that the lecture is effective? How can we know for sure if scores would have been high even without the lecture? We cannot know this because we have nothing to compare this outcome to; we have no control group. Hence, the design is susceptible to many threats to internal validity, such as history effects (unanticipated events that can co-occur with the exam) and maturation effects (natural changes in learning). In all, these limitations make the one-group posttest-only design a poor research design.
A one-group posttest-only design is a quasi-experimental research design in which a dependent variable is measured for one group of participants following a treatment.
One-group designs lack a control group.
One-Group Pretest-Posttest Design
One way to minimize problems related to having no control or comparison group is to measure the same dependent variable in one group of participants before (pretest) and after (posttest) a treatment. Using this type of research design, called a one-group pretest-posttest design, we measure scores before and again following a treatment, then compare the difference between pretest and posttest scores. The advantage is that we can compare scores after a treatment to scores on the same measure in the same participants prior to the treatment. The disadvantage is that the one-group design does not include a no-treatment control group and therefore is still prone to many threats to internal validity, including those associated with observing the same participants over time (e.g., testing effects and regression toward the mean).
A one-group pretest-posttest design is a quasi-experimental research design in which the same dependent variable is measured in one group of participants before (pretest) and after (posttest) a treatment is administered.
Figure 9.2 ⦁ The One-Group Posttest-Only Quasi-Experimental Design
To illustrate the one-group pretest-post-posttest design, we will look at the research example illustrated in Figure 9.3. Kimport and Hartzell (2015) measured state anxiety—a type of undesired current stress that is temporary or changes as experiences or conditions change—among psychiatric inpatients from two general adult units at a private psychiatric hospital before and after a structured clay therapy in which patients could use and mold clay for up to 10 minutes. Their results showed that state anxiety was significantly reduced from before to after the therapy. A limitation of this design is that participants were not randomly assigned to groups. This means that any other factors related to state anxiety could also explain the findings. Factors include changes in the conditions or experiences of patients other than the therapy, such as patient interactions with the researchers, how long they actually used the clay during the therapy, and distractions during the therapy (e.g., noises, decorations in the setting). These factors were largely beyond the control of the researchers and therefore could have also influenced the results. In addition, because the study lacked a control group with patients who had no therapy at all, the design was susceptible to many threats to internal validity, as stated previously. Indeed, Kimport and Hartzell (2015) directly acknowledged that “control groups in future research may prevent additional confounds from occurring including the novelty effect, which implies that the treatment may have been effective simply because it was new for the participants” (p. 188). Thus, it is possible any type of new therapy intervention could have been effective.
Description
Figure 9.3 ⦁ The One-Group Pretest-Posttest Quasi-Experimental Design
Source: Based on a design used by Kimport and Hartzell (2015).
9.3 QUASI-EXPERIMENTAL DESIGN: NONEQUIVALENT CONTROL GROUP DESIGNS
In some cases, researchers can use nonequivalent control groups, when it is not possible to randomly assign participants to groups. A nonequivalent control group is a type of control group that is matched upon certain preexisting characteristics similar to those observed in a treatment group, but to which participants are not randomly assigned. For example, suppose a professor gives a new lecture method to your research methods class and gives a traditional method in another research methods class, then compares grades on the topic lectured. The classes are matched on certain characteristics: Both classes are on the same topic (research methods), offered at the same school, and taught by the same professor. However, the class taught using the traditional method is a nonequivalent control group because students in that class chose to enroll in the class, so they were not randomly assigned to that class. Any preexisting differences between students who tend to enroll for one class over another, called selection differences, could therefore explain any differences observed between the two classes. Two types of nonequivalent control group quasi-experiments are the following:
A nonequivalent control group is a control group that is matched upon certain preexisting characteristics similar to those observed in a treatment group but to which participants are not randomly assigned. In a quasi-experiment, a dependent variable measured in a treatment group is compared to that in the nonequivalent control group.
Selection differences are any differences, which are not controlled by the researcher, between individuals who are selected from preexisting groups or groups to which the researcher does not randomly assign participants.
Nonequivalent control group posttest-only design
Nonequivalent control group pretest-posttest design
Nonequivalent Control Group Posttest-Only Design
Using the nonequivalent control group posttest-only design, a researcher measures a dependent variable following a treatment in one group and compares that measure to a nonequivalent control group that does not receive the treatment. The nonequivalent control group will have characteristics similar to the treatment group, but participants will not be randomly assigned to this group, typically because it is not possible to do so. For example, as illustrated in Figure 9.4, suppose a professor gives a new teaching method in their research methods class and gives a traditional method in another research methods class, then tests all students on the material taught. In this example, the nonequivalent control group was selected because it matched characteristics in the treatment group (e.g., all students were taking a research methods class). Students, however, enrolled themselves in each class; random assignment was not used, so the comparison is a nonequivalent control group.
A nonequivalent control group posttest-only design is a quasi-experimental research design in which a dependent variable is measured following a treatment in one group and in a nonequivalent control group that does not receive the treatment.
Description
Figure 9.4 ⦁ The Nonequivalent Control Group Posttest-Only Quasi-Experimental Design
A key limitation of this research design is that it is particularly susceptible to the threat of selection differences. In the example illustrated in Figure 9.4, because students enrolled in their college classes, they, not the researcher, controlled which class they enrolled in. Therefore, any preexisting differences between students who choose one section of a class over another, such as how busy the students’ daily schedules are or how motivated they are to attend earlier or later classes, may actually be causing differences in grades between classes. For this reason, the nonequivalent control group posttest-only design demonstrates only that a treatment is associated with differences between groups, not that a treatment caused differences between groups, if any were observed.
Nonequivalent control group designs include a “matched” or nonequivalent control group.
Nonequivalent Control Group Pretest-Posttest Design
A nonequivalent control group pretest-posttest design is a quasi-experimental research design in which a dependent variable is measured in one group of participants before (pretest) and after (posttest) a treatment and that same dependent variable is also measured at pretest and posttest in another nonequivalent control group that does not receive the treatment.
One way to minimize problems related to not having a comparison group is to measure a dependent variable in one group of participants observed before (pretest) and after (posttest) a treatment and measure that same dependent variable at pretest and posttest in another nonequivalent control group that does not receive the treatment. This type of design is called the nonequivalent control group pretest-posttest design. The advantage of this design is that we can compare scores before and after a treatment in a group that receives the treatment and in a nonequivalent control group that does not receive the treatment. While the nonequivalent control group will have characteristics similar to the treatment group, participants are not randomly assigned to this group, typically because it is not possible to do so. Hence, selection differences still can possibly explain observations made using this research design.
To illustrate the nonequivalent control group pretest-posttest design, we will look at the research example in Figure 9.5. Heinicke, Zuckerman, and Cravalho (2017) evaluated the effectiveness of online Readiness Assessment Tests (RATs)—quizzes or tests given prior to class to inform the instructor of where students are struggling the most, from which they can adapt course lectures—on overall class exam grades. Heinicke et al. (2017) hypothesized that implementing RATs into coursework would improve overall grades and class performance in general. To test this hypothesis, college students enrolled in one of two sections of a Psychology of Exceptional Children course were recruited to participate. In one section, the RATs were a required part of the course (the treatment group; Section B); in the other section, the course was structured the same except that RATs were not part of the coursework (the nonequivalent control group; Section A). Knowledge of course material was assessed both prior to and at the end of the course. As shown in Figure 9.6, while knowledge of course material was not different prior to the course, students who were in the class with RATs incorporated (the treatment group) showed overall higher grades on the final assessment by the end of the course compared with students in the nonequivalent control group who did not have RATs incorporated into the course.
A key limitation of this research design is that it is particularly susceptible to the threat of selection differences. In the example illustrated in Figure 9.5, because students enroll in college classes, they, not the researcher, control what classes they will be in. Any preexisting differences between students who choose one class over another, then, could also be causing differences between classes. For example, Heinicke et al. (2017) acknowledged that because students were not randomly assigned to classes, the differences in overall class performance between those who did versus did not have RATs incorporated into their course could also be due to other “potential extraneous variables, such as the timing of the class (e.g., Section B met at 10:00 a.m., whereas Section A met at 8:00 a.m.)” (p. 137). Hence, the nonequivalent control group pretest-posttest design, like the posttest-only design, demonstrates only that a treatment is associated with differences between groups, not that a treatment caused differences between groups, if any were observed.
Figure 9.5 ⦁ The Nonequivalent Control Group Pretest-Posttest Quasi-Experimental Design
Source: Based on a design used by Heinicke et al. (2017).
Description
Figure 9.6 ⦁ The Overall Grades on a Final Assessment Between Groups That Did Versus Did Not Have RATs Incorporated into the Course
Source: Data are adapted from those reported by Heinicke et al. (2017).
RAT = Readiness Assessment Tests.
9.4 QUASI-EXPERIMENTAL DESIGN: TIME SERIES DESIGNS
In some situations, researchers observe one or two preexisting groups at many points in time before and after a treatment, and not just at one time, using designs called the time series quasi-experimental designs. Using these types of designs, we compare the pattern of change over time from before to following a treatment. Three types of time series quasi-experimental designs are as follows:
- Basic time series design
- Interrupted time series design
- Control time series design
Basic Time Series Design
When researchers manipulate the treatment, they use a basic time series design to make a series of observations over time before and after a treatment. The advantage of measuring a dependent variable at multiple times before and after a treatment is that it eliminates the problem associated with only having a snapshot of behavior. To illustrate, suppose we test a treatment for improving alertness during the day. To use the basic time series design, we record alertness at multiple times before and after we give participants the treatment, as illustrated in Figure 9.7. Notice in the figure that a pretest (at 12 p.m.) and posttest (at 4 p.m.) measure can be misleading because the pattern observed before and after the treatment recurred without the treatment at the same time the day before and the day after the treatment was given. The basic time series design allows us to uniquely see this pattern by making a series of observations over time.
A basic time series design is a quasi-experimental research design in which a dependent variable is measured at many different points in time in one group before and after a treatment that is manipulated by the researcher is administered.
Figure 9.7 ⦁ The Time Series Quasi-Experimental Design
A time series design is used to compare the pattern of behavior before and after the treatment. In this example, the pattern that occurs before and after the treatment recurs at the same time of day, even without the treatment.
Using the basic time series design, the researcher manipulates or controls when the treatment will occur. The advantage of this design is that we can identify if the pattern of change in a dependent variable before and after the treatment occurs only during that period and not during other periods when the treatment is not administered. The disadvantage of this design is that only one group is observed, so we cannot compare the results in the treatment group to a group that never received the treatment.
Time series designs include many observations made before and after a treatment.
In a basic time, series design, we manipulate the treatment; in an interrupted time series design, the treatment is naturally occurring.
Interrupted Time Series Design
In some situations, researchers will measure a dependent variable multiple times before and after a naturally occurring treatment or event. Examples of a naturally occurring treatment or event include a scheduled medical procedure, a wedding, a natural disaster, a change in public policy, a new law, and a political scandal. These events occur beyond the control of the researcher, so the researcher loses control over the timing of the manipulation. In these situations, when multiple measurements are taken before and after a naturally occurring treatment, researchers use the interrupted time series design.
An interrupted time series design is a quasi-experimental research design in which a dependent variable is measured at many different points in time in one group before and after a treatment that naturally occurred.
As an example of the interrupted time series design, Fuller, Sahlqvist, Cummins, and Ogilvie (2012) measured the impact of two London Underground (the “Tube”) strikes by the train drivers on the usage of bicycle travel using a public bicycle share program that provided bicycles (“Boris bikes”) at docking stations around London for a small fee. For this study, the researchers recorded the number of trips per day on the Boris bikes. In Figure 9.8, the solid vertical lines show the dates for each 24-hour strike. Note that in their study, each time there was a strike, the number of trips on the Boris bikes spiked, showing evidence that the Tube strikes by train drivers was related to an increase in usage of the Boris bikes.
An advantage of the interrupted time series design is that we can identify if the pattern of change in a dependent variable change from before to following a naturally occurring treatment or event. The disadvantage of this design, like that for the basic time series design, is that only one group is observed, so we cannot compare the results in the treatment group to a group that never received or was never affected by a treatment. To address this disadvantage, we can include a matched or nonequivalent control group, as described in the next section.
Description
Figure 9.8 ⦁ Total Number of Trips per Day on the Boris Bikes
On the day of each strike, there was a sudden increase in total number of trips. Data are reproduced with permission from those reported by Fuller et al. (2012).
Control Time Series Design
A basic or interrupted time series design that includes a matched or nonequivalent control group is called a control time series design. As an example of a control time series design, Hacker et al. (2017) used such a design to test the effects of the implementation of a behavioral health child screening mandate in Massachusetts on the rates of behavioral health screenings. To compare their time series data, they also included rates of behavioral health screenings during the same time period in California, where such a policy was not implemented. California, then, was a nonequivalent control group that was matched because “it has a [similar] large diverse and stable Medicaid population [with] no competing mandate” (Hacker et al., 2017, p. 26). As shown in Figure 9.9, the implementation of the mandate was associated with an increase in behavioral health screenings in Massachusetts; no increase was observed in California during this same period.
A control time series design is a basic or interrupted time series quasi-experimental research design that also includes a nonequivalent control group that is observed during the same period as a treatment group but does not receive the treatment.
Description
Figure 9.9 ⦁ Rate of Behavioral Health Screening in Massachusetts and California Before and After the 2008 Behavioral Health Child Screening Mandate in Massachusetts
Source: Reprinted with permission from Psychiatric Services, (Copyright © 2017). American Psychiatric Association. All Rights Reserved. The behavioral health child screening mandate was associated with an increase in behavioral health screenings in Massachusetts; no change seen in California (the matched control).
As a caution, while the addition of the matched control group strengthens the design, keep in mind that the residents in each state are preexisting groups in that residents chose to live in those locations (or were born in those locations); the researcher did not randomly assign them to live in those locations. It is therefore possible, like that for all other designs that use a nonequivalent control group, that selection differences (such as differences in access residents have to care and even the costs of care for residents in each state) could have caused the differences observed in behavioral health screening rates between the states. For this reason, we conclude that the mandate was associated with an increase in behavioral health screenings, not that the mandate caused the increase.
Table 9.1 summarizes each quasi-experimental research design described in this chapter. In the next section, we introduce a special case of quasi-experiments used in developmental research.
Table 9.1 ⦁ The Quasi-Experimental Research Designs
Type of Quasi-Experimental Design
Description
Key Limitation
One-group posttest only——Observe one group after (posttest) a treatment.
No control group for comparison
One-group pretest-posttest——Observe one group before (pretest) and after (posttest) a treatment.
No control group for comparison
Nonequivalent control group posttest only
Observe treatment and nonequivalent control groups after (posttest) a treatment.
No random assignment between groups
Nonequivalent control group pretest-posttest
Observe treatment and nonequivalent control groups before (pretest) and after (posttest) a treatment.
No random assignment between groups
Basic time series design
Make many observations over time before and after a treatment manipulated by the researcher.
No control group for comparison
Interrupted time series design
Make many observations over time before and after a naturally occurring treatment.
No control group for comparison
Control series design
A time series design with a matched or nonequivalent control group.
No random assignment between groups
Learning Check 1 ✓
1. The quasi-experimental research design is structured similar to an experiment, except ____________ [complete the sentence].
2. State the type of quasi-experimental research design described in each of the following examples:
A researcher records the time (in seconds) it takes a group of participants to complete a computer-based task following an online “how-to” course.
A researcher records the rate of traffic accidents on a section of highway each month for 2 years before and 2 years after the speed limit on that section of highway is reduced.
A researcher records employee satisfaction before and after a training seminar, then compares satisfaction scores for employees at a local branch to the scores for those at the main branch who did not receive the seminar.
Answers:
- the research design includes a quasi-independent variable and/or lacks an appropriate or equivalent control group; 2. A. One-group posttest-only design, B. Interrupted time series design, C. Nonequivalent control group pretest-posttest design.
9.5 QUASI-EXPERIMENTAL DESIGN: DEVELOPMENTAL DESIGNS
An important area of research is used to study changes that occur across the life span. This type of research aims to understand how people or species change as they develop or age. The unique aspect of this area of research is that age, which is the factor being studied, is a quasi-independent variable. Age is a preexisting factor in that the researcher cannot manipulate the age of a participant. Because this design does not include a manipulation, it is also commonly categorized as a nonexperimental design. However, in this chapter, we describe this under the quasi-experimental category because, as you will see, each design is analogous to a quasi-experimental design already introduced in this chapter. Regardless of the category that developmental designs fit best with, it is most important to note that while developmental designs can demonstrate that variables differ by age, they do not demonstrate what causes variables to differ by age—more controlled procedures are needed, such as in an experiment.
The study of developmental changes across the life span is a special case, in that the focus of the field is on a factor that is inherent to the participants (their age). Therefore, researchers have developed research designs specifically adapted to study changes across the life span. Three types of developmental research designs are the following:
- Longitudinal design
- Cross-sectional design
- Cohort-sequential design
Longitudinal Design
Using a research design called the longitudinal design, we can observe changes across the life span by observing the same participants over time as they age. Using this design, researchers observe the same participants and measure the same dependent variable at different points in time or at different ages. The longitudinal design is similar to the one-group pretest-posttest quasi-experimental research design in that one group of participants is observed over time. In a strictly longitudinal design, however, changes at different ages are tested, but no treatment is administered.
A longitudinal design is a developmental research design used to study changes across the life span by observing the same participants at different points in time and measuring the same dependent variable at each time.
To illustrate the longitudinal design, consider the research example illustrated in Figure 9.10. Vrangalova (2015) tested the hypothesis that casual sex among college students is related to their well-being. To test this hypothesis, the researchers had a sample of 528 undergraduate students complete an online survey at the beginning (Time 1) and again at the end (Time 2) of 1 academic year. In support of their hypothesis, students reporting having engaged in “hookups for anonymous reasons” (p. 945) between Time 1 and Time 2, had lower self-esteem, and had higher depression and anxiety scores compared with those who did not report engaging in this activity. This study highlights a key advantage of the longitudinal design in that changes in participant behavior can be recorded over extended periods (e.g., Hawkley, Thisted, & Cacioppo, 2009), even 1 year or more.
The disadvantage of the longitudinal design is that it is prone to many threats to internal validity associated with observing participants over time. For example, many participants may drop out of the study over time (attrition). One possibility is that those who are most motivated to complete the study will remain at the end, so it could be motivation to complete the study, not age, that is associated with any changes observed. In addition, participants could learn how to take the assessments (testing effect) or settle down during the study so that assessments at Time 2 actually reflect their true score (regression toward the mean) on the measures recorded. Finally, the longitudinal design can require substantial resources, money, recruitment efforts, and time to complete, particularly for studies that last years or even decades.
Importantly, participant characteristics, referred to as individual differences, can further be used to explain any differences or changes observed in a longitudinal study. For this reason, many researchers who use this design will record additional measures at Time 1/Time 2 so that they can control for these factors prior to evaluating differences over time. For example, Vrangalova (2015) recorded a variety of participant characteristics, such as demographic background, personality traits, and prior casual and romantic sex (prior to the start of the study), to ensure that such factors could be controlled for (i.e., identified or eliminated as possible reasons or explanations for the results), prior to evaluating the differences described in their study. Measuring participant characteristics, then, is a practical way to control for factors that you anticipate may influence differences over time in a longitudinal study.
Description
Figure 9.10 ⦁ The Longitudinal Design
Based on a design used by Vrangalova (2015). The structure of the longitudinal design is to observe the same participants across time.
Age is the quasi-independent variable using a developmental research design.
Cross-Sectional Design
An alternative developmental design that does not require observing the same participants over time is the cross-sectional design. Using this design, the researcher observes a cross-section of participants who are grouped based on their age. The cross-sectional design is similar to a nonequivalent control group quasi-experimental design in that the different age groups act as nonequivalent control groups. Each age group is called a cohort, which is any group of individuals who share common statistical traits or characteristics, or experiences within a defined period. For example, a cohort could be a group of people who were born in the same year, served in the same war, or attended the same school. For developmental research, cohorts in a cross-sectional analysis are related in terms of when participants were born.
A cross-sectional design is a developmental research design in which participants are grouped by their age and participant characteristics are measured in each age group.A cohort is a group of individuals who share common statistical traits or characteristics, or experiences within a defined period.
To illustrate the cross-sectional design, we will look at the research example illustrated in Figure 9.11. Phillips (2008) selected a sample of 99 community college students and 320 middle school and high school students. Each group represented a different age group or cohort. The researcher measured the identity style of students in each cohort (community college vs. middle school and high school) using the Identity Style Inventory Revised for a Sixth-Grade Reading Level (ISI-6G; White, Wampler, & Winn, 1998). Results showed that the identity style of a student is different for precollege and college-aged cohorts.
The advantage of a cross-sectional design is that participants are observed one time in each cohort. Observing participants one time eliminates many threats to internal validity associated with observing participants over time. Factors such as attrition, testing effects, and regression toward the mean are typically not a concern when participants are observed only one time.
Description
gure 9.11 ⦁ The Cross-Sectional Design
Based on a design used by Phillips (2008). Notice that participants are grouped based on their age using the cross-sectional design.
However, a disadvantage of the cross-sectional design is the possibility of cohort effects (or generation effects), which occur when preexisting differences between members of a cohort can explain an observed result. For example, suppose we use a cross-sectional design to measure how often 20-year-olds, 40-year-olds, and 80-year-olds send text messages. In this example, we are likely to find that texting decreases with age. However, there is also a cohort effect due to the generational gap in advances of technology. An 80-year-old participant was raised when cell phones, and therefore texting, did not yet exist. This cohort effect of differences in experience or familiarity with texting across the life span can alternatively explain why texting appears to decrease with age, without appealing to age as the primary explanation. For this reason, researchers must be cautious to consider any possible cohort effects prior to the conduct of a cross-sectional study.
Table 9.2 summarizes the two developmental research designs described here. These two research designs, the longitudinal design and the cross-sectional design, can also be used together, as is described next.
A cohort effect, or a generation effect, is a threat to internal validity in which differences in the characteristics of participants in different cohorts or age groups confound or alternatively explain an observed result.
Cohort-Sequential Design
To combine the advantages of longitudinal and cross-sectional developmental research designs, we can use a cohort-sequential design. Using the cohort-sequential design, two or more cohorts are observed from or at different points in time (cross-sectional design), and over time (longitudinal design). Figure 9.12 illustrates this design when three cohorts are observed, with each cohort also observed over time. Note that this design requires only that the longitudinal observations overlap across the cohorts. With only two cohorts observed, it is also common for some of the same participants to be represented in each cohort, as described in the following research example, physical activity among adolescent girls. In their study, physical
A cohort-sequential design is a developmental research design that combines longitudinal and cross-sectional techniques by observing different cohorts of participants over time at overlapping times.
Figure 9.12 ⦁ The Cohort-Sequential Design
In this example of a cohort-sequential design, three cohorts of participants born as part of Generation X (oldest cohort), Millennials, or Generation Z (youngest cohort) are observed on some measure over time. The shaded boxes indicate when each group was observed. In this example, each cohort was observed twice, and the times of longitudinal observations overlapped.
Table 9.2 ⦁ Potential Limitations of the Longitudinal and Cross-Sectional Research Designs
Potential Limitations Developmental Research Design Cross-Sectional
Longitudinal
Threats to internal validity
History and maturationYes, because participants are observed more than one time, and the design lacks a control group.
Possibly, because the control groups (by age) are nonequivalent.
Regression and testing effects?
Yes, because participants are observed more than one time.
No, because participants are observed only one time.
Heterogeneous attrition?
Yes, because participants are observed more than one time.
Possibly, but not likely because participants are observed only one time.
Cohort effects?
No, because participants from the same cohort are observed over time.
Yes, because participants are grouped based on their age, which is a cohort.
Additional potential limitations
Time-consuming?
Yes, studies can range from months to years in length.
No, a cross-section of the life span is observed at one time.
Costly/expensive?
Yes, keeping track of participants costs time, recruitment, and money.
Possibly, but this design is typically less costly/expensive than a longitudinal study.
As an example of how the cohort-sequential design can be applied when the same participants are represented in each cohort, Pate et al. (2009) measured age-related changes in physical activity among adolescent girls. In their study, physical activity was measured in sixth-grade girls, and physical activity was again measured 2 years later when the girls were in eighth grade. Part of their sample was longitudinal in that the same girls from sixth grade were sampled again when they were in eighth grade. Also, by chance, some girls were sampled only one time because some sixth-grade girls did not participate in eighth grade and some eighth-grade girls included in the study did not participate when they were in sixth grade. The advantage of using this cohort-sequential design is that researchers can do the following:
Account for threats to internal validity associated with observing participants over time because part of the sample is a cross-section of age groups.
Account for cohort effects because part of the sample includes the same participants observed over time in each age group or cohort.
9.6 Ethics in Focus
Development and Aging
Ethical concerns related to age are often focused on those who are very young and those who are very old. For younger participants, researchers must obtain consent from a parent, caregiver, or legal guardian to study minors, who are children under the age of 18 years. On the other extreme, older individuals require special permissions particularly when they are deemed no longer functionally or legally capable. Additional concerns also arise for the ethical treatment of clinical populations, such as those suffering trauma or disease at any stage of development. In all, you should follow three rules to ensure that such groups or cohorts are treated in an ethical manner:
Obtain assent when necessary. In other words, ensure that informed consent is obtained from the participant only after all possible risks and benefits have been clearly identified.
Obtain permission from a parent, caregiver, legal guardian, or another legally capable individual, such as a medical professional, when a participant is a minor or when a participant is functionally or legally incapable of providing consent.
Clearly show that the benefits of a study outweigh the costs. For any group that is studied, that group (younger, older, or incapable) should specifically benefit from participating in the research with minimal costs.
Learning Check 2 ✓
State the developmental research design that is described by each of the following phrases:
Observing participants over time
Observing groups at one time only
Prone to testing effects
Prone to cohort effects
A __________ is a group of individuals who share common statistical or demographic characteristics.
Answers:
- A. Longitudinal, B. Cross-sectional, C. Longitudinal, D. Cross-sectional; 2. cohort.
SINGLE-CASE EXPERIMENTAL DESIGNS
In this section, we begin by identifying a new research design to test the following research hypothesis: Giving encouragement to students who are at risk of dropping out of school will keep them on task in the classroom. To answer this hypothesis, we could measure the time (in minutes) that an at-risk student stays on task. We could observe the student for a few days with no encouragement. Then we could observe the student for a few days with encouragement given as they work on the task. We could then again observe the student for a few more days with no encouragement. If the hypothesis is correct and we set up this study correctly, then we should expect to find that the time (in minutes) spent on task was high when the encouragement was given but low during the observation periods before and after when no encouragement was given. The unique feature of this design is that only one participant was observed.
In this final section, we introduce the research design that was illustrated here: the single-case experimental design.
9.7 AN OVERVIEW OF SINGLE-CASE DESIGNS
In some cases, often in areas of applied psychology, medicine, and education, researchers want to observe and analyze the behavior of a single participant using a research design called the single-case experimental design. A single-case design is unique in that a single participant serves as their own control; multiple participants can also be observed as long as each individual serves as their own control (Antia, Guardino, & Cannon, 2017; Kazdin, 2016). In addition, the dependent variable measured in a single-case design is analyzed for each individual participant and is not averaged across groups or across participants. By contrast, all other experimental research designs, introduced in Chapters 10 through 12, are grouped designs.
A single-case experimental design is an experimental research design in which a participant serves as their own control and the dependent variable measured is analyzed for each individual participant and is not averaged across groups or across participants.
For a single-case design to be an experimental design, it must meet the following three key elements of control required to draw cause-and-effect conclusions:
Randomization (random assignment). Using single-case designs, each participant can be randomly assigned to experience many phases or treatments controlled by the researcher.
Manipulation (of variables that operate in an experiment). The researcher must manipulate the phases or treatments that are experienced by each participant such that the factor or independent variable is not preexisting.
Comparison/control group. Each participant acts as their own control or comparison. For the single-case designs described here, comparisons can be made across multiple baseline phases (reversal design), participants (multiple-baseline design), or treatments (changing-criterion design).
An advantage of analyzing the data one participant at a time is that it allows for the critical analysis of each individual measure, whereas averaging scores across groups can give a spurious appearance of orderly change. To illustrate this advantage, suppose that a researcher measures the body weight in grams of four rat subjects before and after an injection of a drug believed to cause weight loss. The hypothetical data, provided in Table 9.3, show that rat subjects as a group lost 25 grams on average. However, Rat C actually gained weight following the injection. An analysis of each individual rat could be used to explain this outlier; a grouped design would often disregard this outlier as “error” so long as weight loss was large enough on average.
The single-case design, which is also called the single-subject, single-participant, or small n design, is most often used in applied areas of psychology, medicine, and education.
Table 9.3 ⦁ The Value of an Individual Analysis
Subject
Baseline weight
Weight following drug treatment
Weight loss
Rat A
320
305
15
Rat B
310
280
30
Rat C
290
295
−5
Rat D
360
300
60
8 SINGLE-CASE BASELINE-PHASE DESIGNS
Single-case designs are typically structured by alternating baseline and treatment phases over many trials or observations. In this major section, we introduce three types of single-case experimental research designs:
Reversal design
Multiple-baseline design
Changing-criterion design
Reversal Design
One type of single-case design, called the reversal design (or ABA design), involves observing a single participant prior to (A), during (B), and following (A) a treatment or manipulation. The reversal design is structured into phases, represented alphabetically with an A or a B. Each phase consists of many observations or trials. The researcher begins with a baseline phase (A), in which no treatment is given, then applies a treatment in a second phase (B), and again returns to a baseline phase (A) in which the treatment is removed. This type of research design can be represented as follows:
A reversal design, or ABA design, is a single-case experimental design in which a single participant is observed before (A), during (B), and after (A) a treatment or manipulation.
- A phase is a series of trials or observations made in one condition.
- The baseline phase (A) is a phase in which a treatment or manipulation is absent.
- A (baseline phase) → B (treatment phase) → A (baseline phase)
If the treatment in Phase B causes a change in the dependent variable, then the dependent variable should change from baseline to treatment, then return to baseline levels when the treatment is removed. For example, we opened this section with the hypothesis that giving encouragement to students who are at risk of dropping out of school will keep them on task in the classroom. To test this hypothesis, we measured the time in minutes that an “at-risk” student spent on task in a class with no encouragement (baseline, A) for a few trials, then with encouragement (treatment, B) for a few trials, and again with no encouragement (baseline, A) for a few more trials. If the encouragement (the treatment) was successful, then the time (in minutes) spent on task would be higher when the encouragement was given but lower during the observation periods before and after when no encouragement was given. The second baseline phase minimizes the possibility of threats to internal validity. Adding another B and A phase would further minimize the possibility of threats to internal validity because the pattern of change would be repeated using multiple treatment phases.
A visual inspection of the data, and not inferential statistics, is used to analyze the data when only a single participant is observed. To analyze the data in this way, we look for two types of patterns that indicate that a treatment caused an observed change, as illustrated in Figure 9.13:
A change in level is displayed graphically, as shown in Figure 9.13 (top graph), when the levels of the dependent variable in the baseline phases are obviously less than or greater than the levels of the dependent variable in the treatment phase.
A change in trend is displayed graphically, as shown in Figure 9.13 (bottom graph), when the direction or pattern of change in the baseline phases is different from the pattern of change in the treatment phase. In the typical case, a dependent variable gradually increases or decreases in the treatment phase but is stable or does not change in the baseline phases.
The reversal design is typically conducted in applied areas of research to investigate possible solutions that can benefit individuals or society. For this reason, one advantage of the design is that it can be used to apply treatments that are beneficial to participants. Often this means that researchers will be asked by ethics committees to end their study with a treatment phase (B), which was the phase that was beneficial to the participant. For this reason, many reversal designs are at least four phases, or ABAB, so as not to return to baseline to end an experiment.
A limitation of the reversal design is that the change in a dependent variable in a treatment phase must return to baseline levels when the treatment is removed. However, in many areas of research, such as studies on learning, a return to baseline is not possible. When a participant is taught a new skill, for example, it is often not possible to undo what the participant learned as fully expected, the behavior will not return to baseline. In these situations, when it is not possible for changes in a dependent variable to return to baseline, a reversal design cannot be used.
Figure 9.13 ⦁ Two Ways to Identify if a Treatment Caused Changes in a Dependent Variable
A change in level (top graph) and a change in trend (bottom graph) make it possible to infer that some treatment is causing an effect or a change in behavior.
Source: Republished with permission of John Wiley and Sons Inc, from Enhancing capacity to make sexuality-related decisions in people with an intellectual disability. Dukes, E. & McGuire, B. E., Journal of Intellectual Disability Research, 53 (8), 2009; permission conveyed through Copyright Clearance Center, Inc.
Multiple-Baseline Design
For situations in which it is not possible for changes in a dependent variable to return to baseline levels following a treatment phase, researchers can use the multiple-baseline design. The multiple-baseline design is a single-case design in which the treatment is successively administered over time to different participants, for different behaviors, or in different settings. This design allows researchers to systematically observe changes caused by a treatment without the need of a second baseline phase and can be represented as follows:
A multiple-baseline design is a single-case experimental design in which a treatment is successively administered over time to different participants, for different behaviors, or in different settings.
By representing the multiple-baseline design in this way, a case refers to a unique time, behavior, participant, or setting. Baseline periods are extended in some cases prior to giving a treatment. If the treatment causes an effect following a baseline phase for each case, then the change in level or pattern should begin only when the baseline phase ends, which is different for each case. If this occurs, then we can be confident that the treatment is causing the observed change. This design minimizes the likelihood that something other than the treatment is causing the observed changes if the changes in a dependent variable begin only after the baseline phase ends for each case.
To illustrate the multiple-baseline design, we will look at the research example illustrated in Figure 9.14. Dukes and McGuire (2009) used a multiple-baseline design to measure the effectiveness of a sex education intervention, which they administered to multiple participants with a moderate intellectual disability. The researchers recorded participant knowledge of sexual functioning using the Sexual Consent and Education Assessment (SCEA K-Scale; Kennedy, 1993), on which higher scores indicate greater ability to make decisions about sex. Each participant was given a baseline phase for a different number of weeks. Scores on the SCEA K-Scale were low in this baseline phase. As shown in Figure 9.14 for three participants, only after the baseline period ended and the intervention was administered did scores on the scale increase. Scores also remained high for 4 weeks after the program ended. Hence, the results showed a change in level from baseline to intervention for each participant.
Each participant in the sex education study received the intervention (or the treatment) in successive weeks: Tina (Week 11), Josh (Week 12), and Debbie (Week 13). Because the treatment was administered at different times, and changes in the dependent variable only occurred once the treatment was administered, the pattern showed that the treatment, and not other factors related to observing participants over time, caused the observed changes in SCEA K-Scale scores.
Description
Figure 9.14 ⦁ Results from a Multiple-Baseline Design for Three Participants Receiving a Sex Education Intervention
Source: Republished with permission of John Wiley and Sons Inc, from Enhancing capacity to make sexuality-related decisions in people with an intellectual disability. Dukes, E. & McGuire, B. E., Journal of Intellectual Disability Research, 53 (8), 2009; permission conveyed through Copyright Clearance Center, Inc.
The advantage of a multiple-baseline design is that it can be used when we expect a treatment will not return to baseline, such as when we study learning on some measure, as illustrated in Figure 9.14 for our example. The limitation of a multiple-baseline design is that the design is used when only a single type of treatment is administered. This same limitation applies to the reversal design. For situations when we want to administer successive treatments, then, we require a different type of single-case experimental design.
A and B indicate the phases in a reversal design.
The length of the baseline phase is varied using a multiple-baseline design.
Changing-Criterion Design
For research situations in which we want to change a criterion or treatment after the participant meets an initial criterion or responds to one particular treatment, we can use a changing-criterion design. Using the changing-criterion design, we begin with a baseline phase, which is followed by many successive treatment phases to determine if participants can reach different levels or criteria in each treatment phase. The criterion can be changed as often as necessary or until some final criterion is met. For a three-treatment study, the changing-criterion design can be represented as follows:
A changing-criterion design is a single-case experimental design in which a baseline phase is followed by successive treatment phases in which some criterion or target level of behavior is changed from one treatment phase to the next. The participant must meet the criterion of one treatment phase before the next treatment phase is administered.
To illustrate the changing-criterion design, we will look at the research example illustrated in Figure 9.15. Gentry and Luiselli (2008) used the changing-criterion design to increase the number of bites that Sam, a fictitious name for the 4-year-old boy being observed, would take of a nonpreferred food (i.e., a food he did not like) during a supper meal. In a baseline phase, Sam ate the food with no manipulation. Then a series of manipulations followed. Sam was instructed to spin an arrow that would fall on a number indicating the number of bites of a nonpreferred food that Sam would need to consume during supper to gain a reward, which in this study was his favorite play activity. The initial criterion was a spinner with a 1 and a 2 on it. This criterion was increased over time, until the options on the spinner were 5 and 6 (bites) to meet the criterion to gain a reward. As shown in Figure 9.15, each time the criterion, or the number of bites required to gain a reward, was increased, Sam’s eating behavior correspondingly increased.
Two advantages of the changing-criterion design are that it does not require a reversal to baseline of an otherwise effective treatment and that it enables experimental analysis of a gradually improving behavior. A limitation of the design is that the target behavior must already be in the participant’s repertoire. For example, the number of bites of food is well within the abilities of a healthy child. In addition, researchers should be cautious to not increase or decrease the criterion too soon or by too much, which may impede the natural learning rate of the participant being observed.
Each successive treatment phase in a changing-criterion design is associated with a change in criterion.
Description
Figure 9.15 ⦁ A Changing-Criterion Design to Increase the Number of Bites of Nonpreferred Food for a Single Child (Sam)
At baseline, Sam ate no bites, and then Sam spun an arrow that displayed different criteria for a reward. He began with 1–2 bites, then 2–3 bites, 3–5 bites, 4–6 bites, and finally 5–6 bites in order to receive the reward. The changing criterion is highlighted in each treatment phase. Notice that as the criterion was increased, so did Sam increase the number of bites he took of nonpreferred food. Data based on those presented by Gentry and Luiselli (2008).
Learning Check 3 ✓
Why is the single-case design regarded as an experimental research design?
Identify whether each of the following is an example of a reversal design, a multiple-baseline design, or a changing-criterion design:
A researcher gives a child successively greater levels of positive reinforcement after an initial baseline phase to reduce how often the child bites their nails. The successive treatments are administered until the child has reached a level where they are no longer biting their nails.
A researcher records the duration of time a participant stays on task in a dance recital 4 days before, 4 days during, and 4 days after a behavioral intervention strategy is implemented.
A researcher records the quality of artistic strokes made by three participants. Each participant was given a treatment phase after 3, 4, or 5 days of a baseline phase; no baseline phase was given after the treatment was administered.
For a single-case experimental study, why would a researcher use a multiple-baseline design instead of a reversal design?
Answers:
- Because it meets the three key elements of control required to demonstrate cause and effect: randomization, manipulation, and comparison; 2. A. Changing-criterion design, B. Reversal design, C. Multiple-baseline design; 3. A multiple-baseline design would be used when it is not possible for changes in a dependent variable to return to baseline.
9 VALIDITY, STABILITY, MAGNITUDE, AND GENERALITY
The analysis of single-case experimental research designs is based largely on a visual inspection of the data in a graph and is not based on statistical analyses that require data to be grouped across multiple participants or groups. The specific visual features in a graph that indicate the validity of an observation are described in this section.
Internal Validity, Stability, and Magnitude
Recall from Chapter 6 that internal validity is the extent to which we can demonstrate that a manipulation or treatment causes a change in a dependent measure. Importantly, the extent to which we establish experimental control of all other possible causes is directly related to the internal validity of a research study. The greater the control we establish, the higher the internal validity.
A single-case design requires a visual analysis of the graphical data of a single participant. The level of control and therefore the internal validity of a single-case design can be determined when the following two features are observed in a graph using this type of analysis:
- The stability in the pattern of change across phases
- The stability in the pattern of change across phases
The magnitude or size of the change across phases
Stability is the consistency in the pattern of change in a dependent measure in each phase of a design. The more stable or consistent changes in a dependent measure are in each phase, the higher the internal validity of a research design.
In a visual inspection of a graph, the stability of a measure is indicated by the consistency in the pattern of change in each phase. The stability of a dependent measure is illustrated in Figure 9.16. Data in a given phase can show a stable level (as in Figure 9.16a), can show a stable trend (as in Figure 9.16b), or can be unstable (as in Figure 9.16c). The stability of a measure in each phase is important because when a measure is unstable, changes are occurring in a dependent variable even when the researcher is not manipulating the behavior. When a dependent measure is stable, we can be confident that any changes in level or trend were caused by the manipulation, because changes only occurred between each phase and were otherwise stable or consistent within each phase. Therefore, the more stable a measure, the greater the control and the higher the internal validity in an experiment.
Description
Figure 9.16 ⦁ A Stable Level (a), a Stable Trend (b), and an Unstable Response (c)
Graphs (a) and (b) show a response that indicates high internal validity, whereas graph (c) indicates low internal validity.
Another level of control can be demonstrated by the magnitude of change, which is the size of the change in a dependent measure observed between phases. When a measure is stable within each phase, we look at the magnitude of changes between phases. For a treatment to be causing changes in a dependent measure, we should observe immediate changes as soon as the treatment phase is administered. We can observe an immediate change in level (as shown in Figure 9.17a), or we can observe an immediate change in trend (as shown in Figure 9.17b). The greater the magnitude of changes between phases, the greater the control and the higher the internal validity in a single-case experiment.
Magnitude is the size of the change in a dependent measure observed between phases of a design. The larger the magnitude of changes in a dependent measure between phases, the higher the internal validity of a research design.
Internal validity is related to the stability and magnitude of change across phases in a single-case design.
Description
Figure 9.17 ⦁ Internal Validity and Control
The graphs identify an immediate change in level (top row, a) or a change in trend (bottom row, b) that would indicate a high level of control and high internal validity.
External Validity and Generality
Recall from Chapter 6 that external validity is the extent to which observations generalize beyond the constraints of a study. A single-case design is typically associated with low population validity, which is a subcategory of external validity. In other words, it is not possible to know whether the results in the sample would also be observed in the population from which the sample was selected because single-case experimental designs are associated with very small sample sizes. However, the results in a single-case design can have high external validity in terms of generalizing across behaviors, across subjects or participants, and across settings. The following is an example of each way to generalize results to establish the external validity of a single-case experiment:
As an example of generalizing across behaviors, a psychotherapist may examine the extent to which causes of spousal abuse generalize to or also similarly cause child abuse. In this example, the therapist generalizes across behaviors, from spousal abuse (Behavior 1) to child abuse (Behavior 2).
As an example of generalizing across subjects or participants, an animal researcher may examine the generality of foraging behavior across multiple rat subjects, or a clinical researcher may examine the effectiveness of a behavioral therapy to improve symptoms of depression across multiple participants. In each case, the researcher is generalizing across multiple subjects or participants.
As an example of generalizing across settings, a child psychologist may want to determine the extent to which characteristics of child play behavior during recess generalize to characteristics of play behavior during class time. In this example, the researcher generalizes across settings, from child play behavior during recess (Setting 1) to child play behavior during class time (Setting 2).
External validity is related to the generality of findings in a single-case design.
9.10 Ethics in Focus The Ethics of Innovation
Many single-case experiments look at early treatments for behavioral disorders or simply bad habits such as smoking or nail biting. When these types of behaviors are studied using a single-case design, the treatment is typically hypothesized to have benefits, such as reducing symptoms of the behavioral disorder or reducing the frequency of bad habits. Researchers will end an experiment with the treatment phase that was most beneficial, so as to maximize the benefits that participants receive. In a reversal design, this means that researchers end the study in a B phase (e.g., ABAB). A multiple-baseline design and a changing-criterion design already end in a treatment phase. Adding a treatment phase or otherwise adapting a single-case design is quite manageable for researchers because they observe only one or a few subjects or participants in a single-case experiment. Observing such a small sample size allows researchers the flexibility to make changes, such as when they add or omit treatments to maximize benefits to participants.
The flexibility of a single-case design also allows for greater “investigative play” (Hayes, 1981, p. 193) or greater freedom to ask innovative or new questions about treatments with unknown causes or with unknown costs or benefits. Single-case designs allow for the conduct of such innovative research to rigorously evaluate potential, yet untested, treatments with small samples; this allows researchers to test the treatment without exposing such a treatment to large groups of participants, particularly when the potential costs of implementing such a treatment are largely unknown or untested. In this way, single-case designs can be used as an initial research design for testing some of the most innovative research in the behavioral sciences.
Learning Check 4 ✓
- Perform a visual inspection of the following data. Does the graph illustrate a study with high internal validity? Explain.
Description
- A researcher uses a single-case design to record the number of minutes spent studying in a baseline phase and a calming music treatment phase with a student who studied in a library and the same student who studied in a college dormitory room. Based on this description, can the researcher generalize across behaviors, across participants, or across settings?
- Single-case designs allow for greater freedom to ask innovative or new questions about treatments with unknown causes or with unknown costs or benefits. Why can a single-case design be an ethically appropriate research design to test the effectiveness of such treatments?
Answers:
- Yes, because the data at baseline are stable, and there is a change in trend from baseline to treatment; 2. Generalize across settings; 3. Because single-case designs are used with small samples, thereby testing the treatment without exposing such a treatment to large groups of participants.
Chapter Summary
LO 1 Define and identify a quasi-experiment and a quasi-independent variable.
- A quasi-experimental research design is structured similar to an experiment, except that this design lacks random assignment, includes a preexisting factor (i.e., a variable that is not manipulated), or does not include a comparison/control group.
- A quasi-independent variable is a preexisting variable that is often a characteristic inherent to an individual, which differentiates the groups or conditions being compared in a research study. Because the levels of the variable are preexisting, it is not possible to randomly assign participants to groups.
LO 2 Identify and describe two one-group quasi-experimental research designs: the posttest-only and pretest-posttest designs.
- The one-group posttest-only design is a quasi-experimental research design in which a dependent variable is measured for one group of participants following a treatment.
- The one-group pretest-posttest design is a quasi-experimental research design in which the same dependent variable is measured in one group of participants before and after a treatment is administered.
LO 3 Identify and describe two nonequivalent control group quasi-experimental research designs: the posttest-only and pretest-posttest designs.
- A nonequivalent control group is a control group that is matched upon certain preexisting characteristics similar to those observed in a treatment group, but to which participants are not randomly assigned. When a nonequivalent control group is used, selection differences can potentially explain an observed difference between an experimental and a nonequivalent control group.
- The nonequivalent control group posttest-only design is a quasi-experimental research design in which a dependent variable is measured following a treatment in one group and is compared with a nonequivalent control group that does not receive the treatment.
- The nonequivalent control group pretest-posttest design is a quasi-experimental research design in which a dependent variable is measured in one group of participants before (pretest) and after (posttest) a treatment, and that same dependent variable is also measured at pretest and posttest in a nonequivalent control group that does not receive the treatment.
LO 4 Identify and describe three time series quasi-experimental research designs: basic, interrupted, and control designs.
- The basic time series design is a quasi-experimental research design in which a dependent variable is measured at many different points in time in one group before and after a treatment that is manipulated by the researcher is administered.
- The interrupted time series design is a quasi-experimental research design in which a dependent variable is measured at many different points in time in one group before and after a treatment that naturally occurs.
- A control time series design is a basic or interrupted time series quasi-experimental research design that also includes a nonequivalent control group that is observed during the same period as a treatment group but does not receive the treatment.
LO 5 Identify and describe three developmental quasi-experimental research designs: longitudinal, cross-sectional, and cohort-sequential designs.
- A longitudinal design is a developmental research design used to study changes across the life span by observing the same participants over time and measuring the same dependent variable each time.
- A cross-sectional design is a developmental research design in which participants are grouped by their age and participant characteristics are measured in each age group. Each age group is a cohort, so this design is prone to cohort effects, which occur when unique characteristics in each cohort can potentially explain an observed difference between groups.
- A cohort-sequential design is a developmental research design that combines longitudinal and cross-sectional techniques by observing different cohorts of participants over time at overlapping times.
LO 6 Define the single-case experimental design.
The single-case experimental design is an experimental research design in which a participant serves as their own control and the dependent variable measured is analyzed for each individual participant and is not averaged across groups or across participants. This design meets the three requirements to demonstrate cause and effect: randomization, manipulation, and comparison/control.
LO 7 Identify and describe three types of single-case research designs: the reversal, multiple-baseline, and changing-criterion designs.
- The reversal design is a single-case experimental design in which a single participant is observed before (A), during (B), and after (A) a treatment or manipulation.
- The multiple-baseline design is a single-case experimental design in which a treatment is successively administered over time to different participants, for different behaviors, or in different settings.
- The changing-criterion design is a single-case experimental design in which a baseline phase is followed by successive treatment phases in which some criterion or target level of behavior is changed from one treatment phase to the next. The participant must meet the criterion of one treatment phase before the next treatment phase is administered.
LO 8 Identify in a graph the stability and magnitude of a dependent measure and explain how each is related to the internal validity of a single-case design.
- The stability of a measure is the consistency in the pattern of change in a dependent measure in each phase of a design. The more stable or consistent changes in a dependent measure are in each phase, the higher the internal validity of a research design.
- The magnitude of change in a measure is the size of the change in a dependent measure observed between phases of a design. A measure can have a change in level or a change in trend. The larger the magnitude of change, the greater the internal validity of a research design.
LO 9 Identify three ways that researchers can strengthen the external validity of a result using a single-case design.
A single-case design is typically associated with low population validity (a subcategory of external validity). However, three ways that researchers can strengthen the external validity of a result using a single-case design is to generalize across behaviors, across subjects or participants, and across settings.
REVIEW QUESTIONS
- A quasi-experimental research design is structured similar to an experiment, with what two exceptions?
- State whether each of the following factors is an example of an independent variable or a quasi-independent variable. Only state “quasi-independent variable” for participant variables that cannot be manipulated.
- The age of participants
- Time allotted for taking an exam
- A participant’s work experience
- Time of day a study is conducted
- A participant’s state of residence
- Amount of sugar added to a drink
- How does a one-group pretest-posttest design improve on the posttest-only quasi-experimental design? What is the major limitation of all one-group designs?
- What is a nonequivalent control group, and why does this type of group make it difficult to determine cause and effect using a nonequivalent control group quasi-experimental design?
- What is the key difference between the basic and interrupted time series quasi-experimental research designs?
- Name the developmental research design described in each of the following examples:
- A researcher measures job satisfaction in a sample of employees on their first day of work and again 1 year later.
b. A researcher records the number of nightmares per week reported in a sample of 2-year-old, 4-year-old, and 8-year-old foster children.
7. (A) Cohort effects are a threat to what type of validity? (B) Which developmental research design is most susceptible to effects?
8. Why is the single-case design regarded as an experimental research design?
9. A reversal design is used to test the hypothesis that low lighting in a room reduces how quickly students read. As shown in Graph 1 for one student, a student reads passages of similar length in a room with normal lighting (baseline), then in the same room with dim lighting (treatment), and then again with normal lighting. Do the results shown in the figure support the hypothesis? Explain.
10. What is the most likely reason that a researcher uses a multiple-baseline design instead of a reversal design?
11. Define the changing-criterion design and explain when the design is used.
12. Are the baseline data shown in Graph 2 stable? Do the baseline data in the figure indicate high or low internal validity?
13. A researcher examines the generality of a behavioral treatment for overeating by testing the same treatment to treat overworking. In this example, is the researcher generalizing across behaviors, across participants, or across settings?
14. A researcher examines if the effectiveness of a new learning system used in a classroom is also effective when used in a home (for homeschooled children). In this example, is the researcher generalizing across behaviors, across participants, or across settings?
ACTIVITIES
Use an online database, such as PsycINFO, to search scientific research articles for any topic you are interested in. Perform two searches. In the first search, enter a search term related to your topic of interest, and enter the term longitudinal to find research that used this design in your area of interest. Select and print one article. In the second search, again enter a search term related to your topic of interest, and this time enter the term cross-sectional to find research that used this design in your area of interest. Again, select and print one article. Once your searches are complete, complete the following assignment:
Write a summary of each article, and explain how each research design differed.
Describe at least two potential threats to internal validity in each study.
Include the full reference information for both articles at the end of the assignment.
A researcher proposes that having a pet will improve health.
Write a research plan to test this hypothesis using a single-case experimental design.
What is the predicted outcome or pattern, if the hypothesis that having a pet will improve health were correct?
Identify the extent to which your results demonstrate high or low internal validity.
Graph the expected results.
