Computer-administered interviews and rating scales

To evaluate the value of computer-administered interviews and rating scales, the following topics are reviewed in the present article: (a) strengths and weaknesses of structured and unstructured assessment instruments, (b) advantages and disadvantages of computer administration, and (c) the validity and utility of computer-administered interviews and rating scales. Computer-administered evaluations are more comprehensive and reliable and less biased than evaluations routinely conducted in clinical practice. Also, the use of continuous monitoring systems, which increasingly entail the use of computer administration, has been related to improved treatment outcome. However, the use of computer-administered interviews and rating scales will sometimes lead to false positive diagnoses, and for this reason, it is recommended that computer assessment be combined with clinical judgment.     

Multiple anchoring of category rating scales

A psychophysical experiment compared the effects of two different kinds of anchoring upon category ratings of the sizes of squares: (1) single anchoring in which the same square was presented on every anchoring trial, and (2) multiple anchoring in which squares of different sizes were presented on anchoring trials. Subjects did not rate the anchors, only those squares presented on alternate trials as the series stimuli. The major finding was that the two kinds of anchoring have similar effects. As with the single anchor, the multiple anchor establishes a new endpoint for the scale of judgment. The previously demonstrated relationship of increasing and then decreasing contrast as a function of the remoteness of the single anchor (Sarris, 1967, 1976) was found also for multiple anchoring.     

Adaptive rating scales in sequential decisions

The benefits gained in decision research through the use of computer-controlled procedures, as opposed to the use of traditional pencil-and-paper techniques, can exceed the expected gains of efficiency and standardization. Computers extend the scope of a sequential decision making paradigm by enhancing (1) the strength of experimental design, (2) reliability of measurement, (3) procedural flexibility, and (4) single-subject data analysis. Data are reported that indicate the necessity of using computers to obtain reliable measurements and valid results for this type of decision research.     

Iterative techniques for unbiased rating scales

Three iterative techniques for neutralizing the effects of stimulus bias in category rating experiments were examined with a wide variety of stimulus variables. Under all conditions examined, the iterative techniques quickly led to a stable category estimation. This result was obtained for stimulus variables with strong measurement properties, e.g. length and weight; for stimulus variables with only ordinal properties, e.g. emery papers; and for stimulus variables with only nominal properties, where an ordered set is obtained only in the course of the category scaling, e.g. female profiles.     

Effects of instructions and rating scales on item selection for the BSRI scales

Walkup and Abbott (1978) stated that Edwards and Ashworth's (1977) failure to replicate Bem's (1974) selection of items for the Masculinity and Femininity Scales of the Bern Sex Role Inventory (BSRI) may be attributed to differences in the instructions and anchored rating scales used in the two studies. The present study tested the hypothesis that presence of various interaction effects involving instructions and rating scales would influence the acceptability of items for the BSRI Masculinity and Femininity Scales. Results based on the evaluation of individual items by Bem's item selection criteria in each of the four experimental conditions obtained by systematically manipulating two instructions (Bem's and Edwards' instructions) and two rating scales (Bem's and Edwards' rating scales) and also those based on the analysis of variance of item mean desirability ratings from the four experimental conditions supported the hypothesis.     

Factor analysis of the Iowa family interaction rating scales

Observational coding systems are uniquely suited for investigating interactional processes in couples and families, but their validity in diverse populations is unknown. We addressed this issue by applying factor analysis to interactional data collected from couples in low-income neighborhoods and coded with the widely used Iowa Family Interaction Rating Scales (IFIRS). Our sample of 414 low-income, ethnically diverse newlywed couples each provided 24-min samples of problem-solving and social support behavior. Interrater reliabilities were strong, and the resultant factors--reflecting positive, negative, and effective communication--were very similar to those obtained with White middle-class samples. Additionally, couples were more negative, less positive, and less effective in problem-solving conversations than in socially supportive conversations, further supporting the validity of the IFIRS in this population. We conclude by discussing the strengths and shortcomings of the IFIRS when used in a low-income, ethnically diverse population.     

Natural anchoring at the neutral point of category rating scales

Contextual theories of judgment assume that the rating of any stimulus depends on its relationship to a context of similar stimuli. For example, any specific act of behavior would be rated less favorably when considered in the context of good behaviors then when considered in the context of bad ones. However, two experiments suggest an absolute anchoring of the natural neutral point, an exception to this generalization. In Experiment 1, morally indifferent acts remained “neutral,” regardless of context; and in Experiment 2 the break-even or zero-point in a simulated game of chance was always rated “neutral.” In both experiments, contextual manipulations had powerful effects upon ratings of other stimuli, though never shifting them across the neutral point. Furthermore, both experiments suggest that neutral-point anchoring also affects the use of the most extreme categories in a manner that is unique to this phenomena. In Experiment 2 for example, the introduction of extremely positive “wins” not only made other wins seem less favorable while leaving ratings of the zero-point unchanged, but also made the most negative losses seem more favorable. It is as if the introduction of an extremely positive “win” also introduces the possibility of an extremely negative “loss.” Taken together these findings contradict the basic adaptation-level premise that the entire scale is determined by the neutral point; they are also inconsistent with the assumption of range theories that the scale is anchored by the endpoints of the range of stimuli actually experienced.     

Review of clinical validation of ADHD behavior rating scales

The purpose of this review is to assess the range of overall accuracies for Attention Deficit/Hyperactivity Disorder (ADHD) behavior rating scales evaluated in clinical validation studies. Studies were characterized according to the evidence standards of the American Academy of Neurology (AAN). Studies were excluded due to major design problems such as overfitting by discriminant analysis. The 13 included evaluations of rating scales revealed overall accuracy in the range of 59%-79% with a pooled mean of 69% (+/-7%, standard deviation) and a pooled sample size of 2228 subjects from nine studies. While some of the excluded studies demonstrated higher overall accuracies (>79%), these studies were observed to have factors in experimen tal design and statistics that are known to unduly inflate accuracy. We recommend further research following the full AAN standards, namely well-designed, blinded, pro spective studies of rating scales applied to clinically representative samples evaluated with a clinical standard.     

Format effects in two teacher rating scales of hyperactivity

The object of this study was to investigate the effect of differences in format on the precision of teacher ratings and thus on the reliability and validity of two teacher rating scales of children's hyperactive behavior. Teachers (N=242) rated a sample of children in their classrooms using rating scales assessing similar attributes with different formats. For a sub-sample the rating scales were readministered after 2 weeks. The results indicated that improvement can be made in the precision of teacher ratings that may be reflected in improved reliability and validity.This research was supported by grants from the Spencer Foundation and the National Institute of Mental Health (MH 29495). A somewhat different version of this paper was presented at the annual convention of the American Educational Research Association, Toronto, Canada, March 1978. The author thanks Nadine Lambert and Leonard Marascuilo for their assistance in the preparation of this paper.     

Individually tailored behavioral rating scales: a possible alternative

Individually tailored behavioral ratings were introduced as a possible alternative to time sampling or counting each occurrence of the behavior. Interrater reliability was found to be good for the more specific behavior categories and reliability between ratings obtained from direct observation and those based on unit records was acceptable. The usefulness of individually tailored behavioral rating scales in demonstrating behavior change is demonstrated through the presentation of a representative case.     

Behavior rating scales: need for refining normative data

The purpose of this study was to generate normative data by grade and sex to accompany behavior rating scales. Teachers rate 483 boys and girls in Grades 1 through 4. The findings suggest rating scales be re-examined since norms by grade level and sex may be desirable attributes.     

The effects of biasing information on behavioral observations and rating scales

The effects of biasing information on behavioral observations and rating scales were studied. Fortyone undergraduate students trained in making reliable behavioral observations were given differential expectations concerning the activity level of a target child. They then viewed videotape recordings of that child and tallied frequency counts of six behavioral categories simultaneously. In addition, subjects completed postexperimental rating scales composed of specific, identifiable behaviors in regard to the target child. Results indicated that, for the most part, neither the behavioral observations nor the rating scales were significantly affected by the biasing information. It is suggested that rating scales constructed of items as discrete and readily identifiable as those of behavioral observation measures may prove resistant to biasing effects.This study is based in part on a thesis submitted by the second author to Case Western Reserve University in partial fulfillment of the requirements for the master of arts degree in psychology. The authors express their appreciation to the Claremont Unified School District, Claremont, California, for their assistance in the production of the videotape and to the Instructional Support Center at Case Western Reserve University for their co-operation in providing research space. Special thanks are given to Thomas Hyde for his advice and assistance throughout the study.