On the history of rating scales

Rating scales constitute one of the most widely employed techniques in research on personality and individual differences. The historical background of rating scales is therefore a matter of considerable interest. Though Galton has generally been given credit for originating rating scale methodology, several applications of rating scales prior to Galton can be identified, and the seminal idea of rating scales can be traced back to Galen.     

Computer-administered interviews and rating scales

To evaluate the value of computer-administered interviews and rating scales, the following topics are reviewed in the present article: (a) strengths and weaknesses of structured and unstructured assessment instruments, (b) advantages and disadvantages of computer administration, and (c) the validity and utility of computer-administered interviews and rating scales. Computer-administered evaluations are more comprehensive and reliable and less biased than evaluations routinely conducted in clinical practice. Also, the use of continuous monitoring systems, which increasingly entail the use of computer administration, has been related to improved treatment outcome. However, the use of computer-administered interviews and rating scales will sometimes lead to false positive diagnoses, and for this reason, it is recommended that computer assessment be combined with clinical judgment.     

Multiple anchoring of category rating scales

A psychophysical experiment compared the effects of two different kinds of anchoring upon category ratings of the sizes of squares: (1) single anchoring in which the same square was presented on every anchoring trial, and (2) multiple anchoring in which squares of different sizes were presented on anchoring trials. Subjects did not rate the anchors, only those squares presented on alternate trials as the series stimuli. The major finding was that the two kinds of anchoring have similar effects. As with the single anchor, the multiple anchor establishes a new endpoint for the scale of judgment. The previously demonstrated relationship of increasing and then decreasing contrast as a function of the remoteness of the single anchor (Sarris, 1967, 1976) was found also for multiple anchoring.     

Adaptive rating scales in sequential decisions

The benefits gained in decision research through the use of computer-controlled procedures, as opposed to the use of traditional pencil-and-paper techniques, can exceed the expected gains of efficiency and standardization. Computers extend the scope of a sequential decision making paradigm by enhancing (1) the strength of experimental design, (2) reliability of measurement, (3) procedural flexibility, and (4) single-subject data analysis. Data are reported that indicate the necessity of using computers to obtain reliable measurements and valid results for this type of decision research.     

Iterative techniques for unbiased rating scales

Three iterative techniques for neutralizing the effects of stimulus bias in category rating experiments were examined with a wide variety of stimulus variables. Under all conditions examined, the iterative techniques quickly led to a stable category estimation. This result was obtained for stimulus variables with strong measurement properties, e.g. length and weight; for stimulus variables with only ordinal properties, e.g. emery papers; and for stimulus variables with only nominal properties, where an ordered set is obtained only in the course of the category scaling, e.g. female profiles.     

Effects of instructions and rating scales on item selection for the BSRI scales

Walkup and Abbott (1978) stated that Edwards and Ashworth's (1977) failure to replicate Bem's (1974) selection of items for the Masculinity and Femininity Scales of the Bern Sex Role Inventory (BSRI) may be attributed to differences in the instructions and anchored rating scales used in the two studies. The present study tested the hypothesis that presence of various interaction effects involving instructions and rating scales would influence the acceptability of items for the BSRI Masculinity and Femininity Scales. Results based on the evaluation of individual items by Bem's item selection criteria in each of the four experimental conditions obtained by systematically manipulating two instructions (Bem's and Edwards' instructions) and two rating scales (Bem's and Edwards' rating scales) and also those based on the analysis of variance of item mean desirability ratings from the four experimental conditions supported the hypothesis.     

Natural anchoring at the neutral point of category rating scales

Contextual theories of judgment assume that the rating of any stimulus depends on its relationship to a context of similar stimuli. For example, any specific act of behavior would be rated less favorably when considered in the context of good behaviors then when considered in the context of bad ones. However, two experiments suggest an absolute anchoring of the natural neutral point, an exception to this generalization. In Experiment 1, morally indifferent acts remained “neutral,” regardless of context; and in Experiment 2 the break-even or zero-point in a simulated game of chance was always rated “neutral.” In both experiments, contextual manipulations had powerful effects upon ratings of other stimuli, though never shifting them across the neutral point. Furthermore, both experiments suggest that neutral-point anchoring also affects the use of the most extreme categories in a manner that is unique to this phenomena. In Experiment 2 for example, the introduction of extremely positive “wins” not only made other wins seem less favorable while leaving ratings of the zero-point unchanged, but also made the most negative losses seem more favorable. It is as if the introduction of an extremely positive “win” also introduces the possibility of an extremely negative “loss.” Taken together these findings contradict the basic adaptation-level premise that the entire scale is determined by the neutral point; they are also inconsistent with the assumption of range theories that the scale is anchored by the endpoints of the range of stimuli actually experienced.     

Factor analysis of MMPI-2 basic scales among college students

The MMPI-2 was administered to 288 college students (178 women and 110 men) between the ages of 18 and 34 years. Scores on the basic 13 (three validity and ten clinical) scales were correlated and subjected to principal component analyses separately for men and women. In each case, four factors were retained and subjected to varimax rotation. Coefficients of congruence were calculated for all possible pairs of factors across two samples of college students and the normative adults, separately for men and women, and across genders within each sample. Results indicated that the factorial structure of the MMPI-2 basic scales seemed to possess considerable uniformity across samples and genders, but not without some notable exceptions.     

Review of clinical validation of ADHD behavior rating scales

The purpose of this review is to assess the range of overall accuracies for Attention Deficit/Hyperactivity Disorder (ADHD) behavior rating scales evaluated in clinical validation studies. Studies were characterized according to the evidence standards of the American Academy of Neurology (AAN). Studies were excluded due to major design problems such as overfitting by discriminant analysis. The 13 included evaluations of rating scales revealed overall accuracy in the range of 59%-79% with a pooled mean of 69% (+/-7%, standard deviation) and a pooled sample size of 2228 subjects from nine studies. While some of the excluded studies demonstrated higher overall accuracies (>79%), these studies were observed to have factors in experimen tal design and statistics that are known to unduly inflate accuracy. We recommend further research following the full AAN standards, namely well-designed, blinded, pro spective studies of rating scales applied to clinically representative samples evaluated with a clinical standard.     

Format effects in two teacher rating scales of hyperactivity

The object of this study was to investigate the effect of differences in format on the precision of teacher ratings and thus on the reliability and validity of two teacher rating scales of children's hyperactive behavior. Teachers (N=242) rated a sample of children in their classrooms using rating scales assessing similar attributes with different formats. For a sub-sample the rating scales were readministered after 2 weeks. The results indicated that improvement can be made in the precision of teacher ratings that may be reflected in improved reliability and validity.This research was supported by grants from the Spencer Foundation and the National Institute of Mental Health (MH 29495). A somewhat different version of this paper was presented at the annual convention of the American Educational Research Association, Toronto, Canada, March 1978. The author thanks Nadine Lambert and Leonard Marascuilo for their assistance in the preparation of this paper.     

Factor analysis of the McCarthy scales for groups of white and Mexican-American children

Factor analyses of the McCarthy Scales of Children's Abilities (McCarthy Scales) were performed on separate groups of Anglo (N = 186) and Mexican-American (N = 126) children to determine the comparability of underlying structures and to examine the construct validity of the McCarthy Scales for each ethnic group. The subjects, ranging in age from 5.6 to 7.6 years, were randomly selected from school enrollment rosters and stratified by age, sex, and ethnicity. A principal component analysis produced three factors for the Anglo as well as the Mexican-American groups. Examination of the coefficients of congruence indicated a close similarity among the matching factors. Additionally, the factors which emerged for the Anglos and Mexican-Americans corresponded to the scales constituting the McCarthy Scales. The results of the study provide evidence for the construct validity of the McCarthy Scales for children of two ethnic groups.     

Individually tailored behavioral rating scales: a possible alternative

Individually tailored behavioral ratings were introduced as a possible alternative to time sampling or counting each occurrence of the behavior. Interrater reliability was found to be good for the more specific behavior categories and reliability between ratings obtained from direct observation and those based on unit records was acceptable. The usefulness of individually tailored behavioral rating scales in demonstrating behavior change is demonstrated through the presentation of a representative case.