Changes in student decisions with Convince Me: Using evidence and making tradeoffs
Siegel (2000)

what was studied
This study assessed the "benefits added" of supplementing an issue-oriented advance biology curricula for high school with a decision-making software. The researcher wanted to find out whether students' development of evidence-based argumentative reasoning skills could be enhanced by the addition of an instructional tool called Convince Me. Convince Me is a software that helps students map the relationships between hypotheses, alternative hypotheses, beliefs and evidence about a given topic. Using this tool, students enter information, make links between related statements, and then check their work to see how well their evidential reasoning "fits" the model reasoning built into the program. Based on the Theory of Explanatory Coherence⁴⁹, the Convince Me system rates the plausibility of a students' claims based on their arguments' (a) simplicity, (b) amount of evidence explained, and (c) the number competing hypotheses. Students rate how plausible they think their statements are and then compare their ratings with the system to see how well their judgments correlate with the computer model.

how effectiveness was measured
In order to assess the impact of adding the software to the curriculum, two classes were compared: an experimental class in which Convince Me was used in addition to the regular curriculum, and a control class in which students only did the regular course work. Students completed a pretest, a posttest, and a delayed posttest. All assessments consisted of open-ended questions requiring short essays or sentences and were embedded in the students' regular classroom activities (e.g., during lab work). A criterion referenced, 4-point scale was used to score students' essays in terms of their use of evidence in general (Do they support claims with relevant evidence?) and use of evidence to make tradeoffs (Do students see drawbacks as well as benefits in choice and support these tradeoffs with evidence?)

what the findings were
Students in both courses showed improvement over time in their ability to use evidence and make trade-offs as expected, but the students who used the Convince Me software in addition to the regular coursework showed even greater improvement. Interestingly, the greatest gain for the Convince Me group was from posttest to delayed posttest, indicating that students may have benefitted from integrating what they learned from the software with ongoing instruction. Understanding the concepts behind the software and ability to use evidence and make tradeoffs were related; students who understood the principles behind the tool performed better than students who did not.