Pretest/Posttest Comparison

 

Purpose
Pretest-posttest comparisons, when used in a true experimental design, allow relatively straightforward assessment of a pedagogical or technological intervention by detecting differences in learning outcomes between two points in time – before and after it. This assessment strategy is very common in educational research since its implementation is relatively non-intrusive and its analysis does not normally require more advanced statistical procedures.

Description
 Pretest-posttest comparisons consist of the following simple steps: decided what learning outcomes are of interest, find or create the measures to capture them, randomly assign students to groups, administer the pretest, administer the pedagogical/technological intervention to the experimental group and control or comparison treatment to the other group(s), administer the posttest, and then analyze.

A wide variety of pretest-posttest comparison designs are available; however, the ones described below are all true experimental designs in which students are randomly assigned to groups and identical measures are used to assess the learning outcomes of each group. Why limit the discussion to only those designs which are "truly experimental"? Without random assignment, there is no assurance that the groups are comparable and that the observed differences in learning are the result of the intervention. True experimental designs, then, provide the most reliable information on the effectiveness of a given intervention.

General Requirements
 Again, randomization of participants is necessary to control for all unmeasured, nuisance variables.

Limitations
 It's extremely important that the observations are independent; therefore, the researcher must make sure that the control or comparison groups are not inadvertently exposed to the intervention used in the experimental condition and that participants in the study do not communicate with one another about their experiences. It's also very important that the groups do not differ in ways that are not controlled (e.g., when the technology used in one group has more breakdowns or connection problems than the technology used in another).

In some educational settings, randomly assigning students to groups is not possible or practical. In such cases, most researchers opt for the next best quasi-experimental design. See Cook and Campbell (1979)1 for further information on such strategies.

  

Variations

Two-Group Design
Figure 1. Two Group Control Group Design
Figure 2: Solomon Four-Group Design
Two-Group Control Group Design, which insures all forms of internal validity. In this design, students are randomly assigned to one of two groups (the experimental group or the control group), given a pretest, given the treatment (corresponding to the condition to which they were assigned), and then given the posttest (see Figure 1 to the right).

Solomon Four-Group Design, which includes two additional control groups. It's check on nuisance variables can, in some circumstances, be well worth the requirement of twice as many participants. (see Figure 2 to the right).

 

 

 

 

Counterbalanced Measures Design, which eliminates ordering effects – confounding effects that arise when exposure to the pretest effects subsequent performance on the posttest. For example, a researcher might construct two isomorphic tests:Test A and Test B. At pretest, one half of the participants are randomly assigned to take Test A and other half of the participants are randomly assigned to take Test B. At posttest, who took which test is reversed: participants who received Test A as a pretest now receive Test B, and those who received Test B as a pretest now receive Test A. Although individual data is not interpretable, aggregated group data is. If the researcher suspects that taking the pretest may effect how well one does on the posttest and has a limited number of participants, he or she should consider counterbalancing the measures.

Delayed Posttest Design, which enable the instructor to assess the more long-term or prolonged effects of their course (i.e., what effect persist weeks or even months after the course is completed). Delayed posttests are handy when you think that the learning gains your course fosters may only arise in the long run. Changes in fundamental reasoning or beliefs are good candidates as such.

Matched Subjects Design, in which pairs of students are matched on important characteristics (e.g., pretest scores or SAT scores); each student from the pair is then assigned to one of the two treatment conditions.

Example Research Studies

Additional Resources