A method for evaluating the effectiveness of educational software

A method for executing a detailed evaluation of educational software is described. Several issues are discussed, including the use of an experimental setting versus a field study, and the design of assessment instruments suited to evaluation of educational software. The instruments include a test of remedial skills, a 103-item test of conceptual understanding, and a system for recording students’ use of the software. The method was used to evaluate ConStatS, a program for teaching conceptual understanding of probability and statistics. Preliminary results of the evaluation are presented.