Development and validation of the Conduct Disorder Rating Scale

Several rating scales for measuring externalizing problems in children are inconsistent with widely used diagnostic criteria for conduct disorder (CD). The Conduct Disorder Rating Scale (CDRS) was developed to provide valid measures of CD in children age 5 to 12 years. In Study 1, the CDRS was evaluated in a community sample of 1,554 children. Parents and teachers showed modest agreement in their ratings and children's age and gender differences were consistent with epidemiological trends. In Study 2, the CDRS was evaluated in a clinical sample of 80 children who were assessed for a summer treatment program. The parent version of the CDRS showed 78.8% accuracy in identifying CD against a diagnostic interview, and both parent and teacher versions of the CDRS were significantly associated with observations of antisocial behavior. Although further research with this new tool is warranted, results indicated that the CDRS is well suited to observational and clinical studies of CD.     

Quantifying the effects of the differential outcomes procedure in humans: A systematic review and a meta‐analysis

We present a systematic review and a meta‐analysis comparing the differential outcomes procedure to a nondifferential outcomes procedure among clinical and nonclinical populations. Sixty distinct experiments were included in the systematic review, 43 of which were included in the meta‐analysis. We calculated pooled effect sizes for accuracy (overall accuracy, test accuracy, transfer accuracy) and acquisition outcomes (latency, errors, and trials to mastery). The meta‐analysis revealed significant medium‐to‐large effect sizes for all three accuracy measures (pooled effect size range, 0.57 to 1.30). We found relatively greater effect sizes among clinical populations (effect size = 1.04). The single‐subject experimental literature included in the systematic review was consistent with the findings from the group studies, demonstrating improvements in accuracy and speed of learning for the majority of participants. Moderator and subgroup analyses suggest that discrimination difficulty may induce relatively larger differential outcomes effects. The results indicate that the differential outcomes procedure can be a valuable addition to reinforcement‐based interventions.     

On the methodology of body image assessment: the use of figural rating scales to evaluate body dissatisfaction and the ideal body standards of women

We examined the use of figural rating scales to measure body dissatisfaction and the ideal body standards of women. We also attended to the potential influence of demand characteristics in body assessment research using these measures, a methodological concern largely ignored in this field. We used both within- and between-subjects designs to assess whether women hold different body standards for themselves compared to their same sex peers and whether their own ideal body size (IBS) is different from what they think men prefer. Regardless of the method of data collection (within or between design), we failed to find any differences on separate items of body image ideals. The findings do provide evidence of robust and large effects for the current–ideal discrepancy as measured by figural rating scales, but argue against including multiple questions about ideal body preferences on these types of scales. Concerns about the influence of demand characteristics were not supported by the data.     

Evaluation of the BSRI masculine and feminine items using desirability and stereotype ratings

This study empirically tested Bem's (1974) assumption that the BSRI Masculinity and Femininity scales measure sex-typed standards of desirable behavior for men and women in American society. The adequacy of items in the two scales was evaluated by Bem's (1974) criteria using two types of desirability and stereotype ratings. Results obtained in all the experimental conditions except one involving Bem's desirability instructions and Bem's rating scale did not support the tested assumption. Implications of these results for revising the BSRI scales were discussed.     

结构化面试中评分量表对评分准确性的影响

采用2(组内变量:量尺大小(25分和9分))×2(组间变量:评分方法(相对和绝对))的混合实验设计探讨评分量表对115名大学生新手评委评分准确性的影响。对于评分准确性,采用Cronbach1955年提出的四个指标,Elevation(EL)、Differential elevation(DE)、Stereotype accuracy(SA)、Differential Accuracy(DA)。结果发现,评分方法只在SA上主效应显著,量尺大小在只在DA上主效应边缘显著,评分方法和量尺大小在DE、SA和DA三个指标上均有交互作用。总体上看,在结构化面试评分中,对于评分准确性,相对评分量表优于绝对评分量表,小量尺量表优于大量尺量表。     

Using Reliability Generalization Methods to Explore Measurement Error: An Illustration Using the MMPI–2 PSY–5 Scales

This study empirically tested Bem's (1974) assumption that the BSRI Masculinity and Femininity scales measure sex-typed standards of desirable behavior for men and women in American society. The adequacy of items in the two scales was evaluated by Bem's (1974) criteria using two types of desirability and stereotype ratings. Results obtained in all the experimental conditions except one involving Bem's desirability instructions and Bem's rating scale did not support the tested assumption. Implications of these results for revising the BSRI scales were discussed.     

Agreement of parent and teacher rating scales with comprehensive clinical assessments of attention deficit disorder with hyperactivity

Forty-one clinic-referred children were given the DSM-III diagnosis of attention deficit disorder with hyperactivity (ADD/H) based on structured diagnostic interviews of the child and mother and on two teacher and two parent rating scales. The degree of agreement between diagnostic classification of ADD/H based solely on criterion scores on each rating scale and classification based on the comprehensive clinical assessments that included use of the same rating scaleswas assessed by comparing the ratings of the 41 children with ADD/H to those of 47 clinic-referred children given other diagnoses or no diagnosis. The accuracy of classification was highly similar across scales. Increasing the cutoff score increased the acing true positives. Using the point of intersection of these two curves to choose the optimal cutoff scores resulted in 70–75% accurate classification and 25–30% misclassification of children given clinical diagnoses of ADD/H. This rate of misclassification suggests that rating scales alone should not be used to classify ADD/H for most clinical and research purposes, especially because the method of comparison used in this study favored finding a high degree of correspondence.     

The Beavers, McMaster and Circumplex clinical rating scales: a study of their sensitivity, specificity and discriminant validity

To assess the sensitivity and specificity of clinical rating scales from the Beavers, McMaster and Circumplex models of family functioning videotapes of sixty families engaging in a standardized family task interview were rated using the three rating scales. The sixty families included twenty containing a child with an emotional disorder, twenty containing a child with a mixed disorder of emotions and conduct, and twenty in which none of the children presented with clinically significant difficulties. The three rating scales accurately classified 85-90% of normal controls, 70-90% of cases containing a child with a mixed disorder of emotions and conduct, and 55-65% of families containing a child with an emotional disorder. On the rating scales, the Beavers and McMaster models showed particularly high levels of sensitivity in detecting clinical cases, whereas the Circumplex rating scale was particularly good at classifying non-clinical cases accurately.     

Performance Appraisal Process Research in the 1980s: What Has It Contributed to Appraisals in Use?

In the early 1980s, Landy and Farr (1980) and Feldman (1981) redirected performance appraisal research from issues related to the development of psychometrically sound rating scales to those involving the cognitive processes of raters. Since that time, several reviews have attempted to translate principles from social cognition and cognitive psychology to the specific conditions of formal appraisal systems in work-oriented organizations. In addition, a number of empirical studies have been conducted on this topic. This article reviews empirical research during the 1980s that focused on performance appraisal processes, particularly the research that has focused upon rating accuracy. The review is structured around a three-stage process model of gathering, storing, and retrieving information about social stimuli for the purposes of rating performance. Factors affecting this process are clustered into four categories: appraisal settings, ratees, raters, and the nature of the scales used for the appraisal. Once reviewed, the research is evaluated in terms of its contributions to improving the quality of appraisal systems as they are used in organizations.     

I feel so sorry! Tapping the joint influence of empathy and personal distress on helping behavior

Observing a person in need usually provokes a compound and dynamic emotional experience made up of empathy and personal distress which, in turn, may influence helping behavior. As the exclusive use of rating scales to measure these two emotions does not permit the analysis of their concurrent evolution, we added the analogical emotional scale (AES) in order to measure how these two emotions evolve throughout the emotional experience, from its onset to its conclusion. Therefore, in two studies, the concurrence of empathy and personal distress was induced, both rating scales and AES were used, and participants were given an unexpected opportunity to help. Two effects were found. First, the helping behavior was lower when personal distress prevailed over empathy at the end of the experience (Studies 1 and 2). Second, this “end” effect was coherent with the nature of the different motives evoked by personal distress and empathy—directed to increasing either one’s own welfare (egoistic) or the victim’s welfare (altruism) (Study 2). These results support the usefulness of combining the rating scales and the AES for gaining a better understanding of the nature and behavioral consequences of complex, compound and dynamic emotional experiences.     

The clinical representativeness of couple therapy outcome research

The clinical representativeness of outcome studies is defined as the generalizability of recruitment processes, assessment/diagnostic procedures, treatment protocols, and therapeutic results from research settings to naturalistic treatment settings. The main goal of the present study was to examine the clinical representativeness of couple therapy in outcome studies. The data set was formed by 50 published clinical trials, including 34 couple therapy outcome studies for marital distress (CTMD) and 16 couple therapy outcome studies for comorbid relational and mental disorders (CTMD + C). The present findings showed that, overall, the clinical representativeness of couple therapy outcome studies is only fair (i.e., the mean global score is slightly lower than the midpoint of the rating scale used to assess representativeness). CTMD + C studies fared better than CTMD studies on many dimensions of clinical relevance. Studies in which pretherapy training was less intensive (for CTMD studies only), treatment was less structured, and therapists were more experienced showed larger effect sizes than those in which such was not the case.     

Accuracy of MMPI-A scales ACK,MAC-R,and PRO in detecting comorbid substance abuse among psychiatric inpatients

This study investigated the accuracy of various indices involving the MMPI-A scales, ACK, MAC-R, and PRO in diagnosing substance abuse in a sample of 79 psychiatric inpatients. In the full sample, 89.9% of the cases were accurately classified by at least one of the three scales. The overall accuracy of classification was similar among males, females, Caucasians, and African Americans, although there was a tendency for more false positive misclassifications among males. Profile code type moderated the accuracy of classification with greatest accuracy for code types including Scales 1, 2, 3, 5, or 0 and least accuracy for code types including Scales 4, 6, or 9. ACK, MAC-R, and PRO were better at screening out cases of substance abuse than in accurately identifying those adolescents who were using substances.     

A Systematic Review and Psychometric Evaluation of Adaptive Behavior Scales and Recommendations for Practice

Adaptive behavior scales are vital in assessing children and adolescents who experience a range of disabling conditions in school settings. This article presents the results of an evaluation of the design characteristics, norming, scale characteristics, reliability and validity evidence, and bias identification studies supporting 14 norm-referenced, informant-based interviews and rating scales designed to measure adaptive behaviors. To derive these results, the manuals for each of these scales were reviewed using a standardized coding procedure, and information about each scale was double-coded by reviewers. Findings reveal that several evidence-based adaptive behavior scales are available to school psychologists. Concluding recommendations address selection and use of adaptive behavior scales as part of a comprehensive assessment, using the optimal methods of administration of adaptive behavior scales, and interpreting resultant scores that have demonstrated the highest levels of reliability and the largest body of validity evidence.     

The clinical utility of behavior rating scales: Comments on the diagnostic assessment of ADHD

Responding to Reid and Maag's critique of behavior rating scales in assessing ADHD, published in a previous issue of the Journal of School Psychology, we provided specific recommendations for improving the accuracy of diagnostic decisions using rating scales and affirmed the importance of these procedures in the assessment of ADHD. Strategies for managing the error intrinsic to the use of diagnostic measures, specifically behavior rating scales, were outlined. A multiple stage procedure, involving one or more levels of screening prior to conducting a diagnostic assessment, was recommended as a strategy to increase diagnostic accuracy in a time- and cost-efficient manner.     

EFFECTS OF TRAINING AND RATING SCALES ON RATING ERRORS

Ninety business students were randomly assigned to one of three conditions where they used behavioral observation scales (BOS), behavioral expectation scales (BES), or trait scales in observing people on videotape. Half the individuals received four hours of training to minimize rating errors. Rating errors were reduced significantly regardless of the rating scale that was used. However, behavioral criteria were more resistant to rating errors than trait scales. There was no significant difference between BOS and BES on this dimension. With regard to practicality, BOS were evaluated as significantly better than BES and trait scales. BES and trait scales did not differ significantly on this measure.     

A CRITICAL ANALYSIS OF STUDIES COMPARING BEHAVIORALLY ANCHORED RATING SCALES (BARS) AND OTHER RATING FORMATS1

All published studies comparing behaviorally anchored rating scales (BARS) with other rating formats were reviewed. Results indicate that BARS and other formats appear to differ relatively little (if at all) with regard to psychometric characteristics (leniency, halo, interrater agreement, ratee discriminability, validity and “accuracy”). Other variables, such as degree of rater participation in scale construction and rater preferences for different rating scale formats, also exhibited relatively little differences between BARS and other formats. A close examination of the studies, however, reveals numerous methodological problems confounding interpretations of results, making conclusions about the superiority or inferiority of BARS quite tenuous. Suggestions are made for more appropriate research directions and questions which seem more appropriate for rating scale investigations.     

Psychological reality of cross-media artistic styles

The sensitivity of artistically naive people to cross-media styles (baroque, neoclassic, and romantic) and to period styles (works composed by artists born during the same epoch) in four media (painting, poetry, music, and architecture) was assessed. In two studies, adult subjects tended spontaneously to sort stimuli according to both cross-media styles and period styles. In a third study, nursery school children were shown to be able to sort pictures of paintings and architectural facades on the basis of cross-media styles. Other experiments using rating scales again demonstrated that artistically naive adults are sensitive to both cross-media styles and period styles even when they are not implicitly urged to disregard medium. These and other studies using rating scales suggested that the bases for discrimination of both cross-media styles and period styles are the dimensions of realistic versus unrealistic and of overall arousal potential.     

Evidence for the Effectiveness of an Alternative Multisource Performance Rating Methodology

Despite persistent concerns as to the quality of performance information obtained from multisource performance ratings (MSPRs), little research has sought ways to improve the psychometric properties of MSPRs. Borrowing from past methodologies designed to improve performance ratings, we present a new method of presenting items in MSPRs, frame‐of‐reference scales (FORS), and test the efficacy of this method in a field and lab study. The field study used confirmatory factor analysis to compare the FORS to traditional rating scales and revealed that FORS are associated with increased variance due to dimensions, decreased overlap among dimensions, and decreased error. The laboratory study compared rating accuracy associated with FORS relative to frame‐of‐reference training (FORT) and a control group and demonstrated that FORS are associated with higher levels of accuracy than the control group and similar levels of accuracy as FORT. Implications for the design and implementation of FORS are discussed.     

A study of temperament and personality in anorexia and bulimia nervosa

Although temperament and personality traits could influence the development and course of eating disorders, only a few studies examined the similarities and differences in personality between anorexia and bulimia nervosa. We compared 72 patients with DSM-IV eating disorders and 30 healthy controls. Dimensions of personality and personality disorders were evaluated with the Eysenck's EPQ, Cloninger's TCI, and the SCID-II questionnaires. The rates of impulsivity and clinical features were evaluated using specific rating scales. A comorbid personality disorder was found in 61.8% of patients with eating disorder. Avoidant personality disorder appeared was relatively common in anorexia nervosa restricting type; borderline personality disorder was most frequent in bulimia nervosa and the binge eating-purging type of anorexia nervosa. From a dimensional perspective, anorexic patients presented high scores in the dimension of persistence. Higher harm avoidance and impulsivity was found in bulimic patients. The overall eating disorders group presented high scores in neuroticism and low scores in self-directedness. Eating disorder patients have heterogeneous features of temperament and personality traits. Cluster C personality disorders seem more common in anorexia nervosa restricting type and impulsive personality features are associated with bulimic symptoms. Impulsivity seems to be a key aspect of temperament of bulimic patients, whereas anorexic symptoms are linked to persistent temperament traits.