Varieties of Inconsistency Across Test Occasions: Effects of Computerized Test Administration and Repeated Testing

This article investigates different types of instability in subjects' responses to the Minnesota Multiphasic Personality Inventory (MMPI). Indices were constructed to measure the effects of the automated testing format and repeated testing on 150 undergraduate subjects who took a computer-administered and pencil-and-paper MMPI 1 week apart. One set of indices measures systematic shifting, attributable to format or time alone. Two families of six indices each are computed measuring unsystematic changes in responding, tendencies to shift in particular directions between true, false, and cannot say responses across all 566 items. Unsystematic changes were assessed both between formats and across times, although these factors are confounded. Subjects tend to change their responses to cannot say from the pencil-and-paper to the computer administration and to shift from false to true on the second administration. Systematic shifting due to the test format is related to the more general, unsystematic tendency to shift between true and false responses. The number of cannot say responses in the computerized testing situation, although greater in magnitude, is correlated with the use of cannot say in the pencil-and-paper condition. Systematic shifting attributable to time is distinct from the other types of inconsistency. Subjects are more inconsistent across occasions when the test format changes than would be expected from repeated testing alone, although the consequences of this inconsistency for clinical interpretation are not yet clear. Inconsistent responding is related to subjects' personality characteristics, but not to age or prior experience with computers.