| Abstract: | Schneier (1977) proposed that rater cognitive complexity interacts with format to affect the psychometric soundness of ratings. He also speculated that cognitive complexity may be enhanced by having raters participate in training and/or scale construction programs. This study was designed to test these latter hypotheses, while also providing a partial replication of Schneier's original research. Ninety-six undergraduate students were assigned to four groups. One participated in the construction of a set of BARS and was trained in their use, a second participated in BARS construction only, a third received training only, and the fourth served as a control group. All subjects were administered a measure of cognitive complexity before and after the experimental treatments. Also, all subjects evaluated five simulated ratees using the BARS. A training × participation (2 × 2) ANOCOV indicated no change in cognitive complexity scores as a result of the experimental treatments. A training × participation × (nominalized) cognitive complexity (2 × 2 × 2) MANOVA on mean ratings for the five stimuli found no effects on leniency error. A similar MANOVA on variances of ratings provided little evidence for effects on halo error. These results, taken in conjunction with those of other investigators who have failed to replicate Schneier's findings, suggest that the cognitive reinterpretation should be accepted with caution at best. |
