Computer-assisted versus paper-and-pencil administration of questionnaires

In a field experiment using 95 employees, the results of computer-assisted versus paper-and-pencil administration of questionnaires during computer training were compared. Unlike past research, the computer mode of administration was made as similar as possible, in format, to the paper mode. No differences in means or reliabilities were found between any variables across the two conditions. However, when the correlations were examined, different patterns of relationships were found across the two conditions. Specifically, subjective or affective measures (software efficacy, playfulness, and training reaction) were more highly related when the measures were gathered by computer as opposed to paper-and-pencil. The relationships between subjective measures and more objective or factual measures (experience, performance, training transfer) were not affected. Thus, it appears that, while differences in mode of administration may not be reflected in differences in means, there are differences in nomological validity.