A change of mind: The Conners Abbreviated Rating Scales Reconsidered

A cutoff score of 15 on the Conners 10-item Abbreviated Teacher Rating Scale (ATRS), based on a study from this laboratory, has been widely used by investigators for diagnosis of the syndrome known now as Attention Deficit Disorder with or without Hyperactivity. A replication of the original research employing a larger norm sample indicates that the suggested score of 15 is too low. Comparing the norm sample with hyperactive subjects selected by cutoff of 15 on the ATRS showed that the groups differed greatly on hyperactivity but not on inattention. The abbreviated form of the Conners scale does not effectively select children with attention deficits. Numerous problems with both the 39-item teacher scale and the abbreviated form suggest strongly that they be abandoned as research tools.This research was supported in part by research grant No. MH18909 from the National Institute of Mental Health.     

认知负荷主观评价量表比较

运用双任务实验范式,比较了三种认知负荷主观评价量表的灵敏度与效度。结果发现：在本研究任务条件下,次任务反应时的稳定性、抗干扰性较好,可以作为认知负荷主观评价的标尺;WP量表与PAAS量表的敏感性均较好,其中WP量表的敏感性、诊断性高于后者,TLX量表的敏感性较弱;WP量表与PAAS量表的效度较好,且好于TLX量表。综合各项指标,在中低难度任务下,WP量表是目前认知负荷较为理想的测量工具。     

Effects of Instructions and Rating Scales on Item Selection for the BSRI Scales

Walkup and Abbott (1978) stated that Edwards and Ashworth's (1977) failure to replicate Bem's (1974) selection of items for the Masculinity and Femininity Scales of the Bern Sex Role Inventory (BSRI) may be attributed to differences in the instructions and anchored rating scales used in the two studies. The present study tested the hypothesis that presence of various interaction effects involving instructions and rating scales would influence the acceptability of items for the BSRI Masculinity and Femininity Scales. Results based on the evaluation of individual items by Bem's item selection criteria in each of the four experimental conditions obtained by systematically manipulating two instructions (Bem's and Edwards' instructions) and two rating scales (Bem's and Edwards' rating scales) and also those based on the analysis of variance of item mean desirability ratings from the four experimental conditions supported the hypothesis.     

External Validity of the New Devereux Adolescent Behavior Rating Scales

The discriminant and concurrent validity of the five new scales for the Devereux Adolescent Behavior Rating Scale (DAB) was explored using a heterogeneous sample of psychiatric and substance abuse patients. Consistent with predictions, the substance abuse patients scored higher on the Acting Out Behaviors (AOB) and Heterosexual Interests (HI) scales, and psychiatric patients scored higher on the Psychotic Behaviors scale. Gender differences also were found, including boys being rated higher on Acting Out Behaviors, and girls higher on Heterosexual Interests. The new DAB scales demonstrated sufficient concurrent validity using a thorough record review and a parent rating scale (the Child Behavior Checklist [CBCL]). The Neurotic/Dependent Behaviors scale (NDB) showed a consistent relationship with substance abuse and several other measures of more externalizing behaviors, in addition to the predicted relationships with anxious, tense, and dependent behaviors. The Withdrawn/Timid Behaviors scale (WTB) proved to be a purer measure of internalizing behaviors in both sexes.     

Normative data on Revised Conners Parent and Teacher Rating Scales

Normative data are presented for 570 children on newly revised versions of the Conners Parent and Teacher Rating Scales. Symptom ratings were factor analyzed and structures compared favorably with those obtained using earlier versions of the questionnaires. Interrater correlations (mother-father, parent-teacher) were also reported. Age and sex effects were found to be significant determinants of children 's scores, while social class effects were nonsignificant.This study was supported by a grant from Abbott Laboratories.     

A Multidimensional and Multilevel Extension of a Random-Effect Approach to Subjective Judgment in Rating Scales

In responding to rating scale items, respondents may hold different perspectives on the given categories. The random-effect rating scale model (RERSM), developed to account for variations in the category thresholds across respondents, is unidimensional and unilevel. It becomes statistically inefficient when multiple unidimensional tests have to be analyzed and inapplicable when data have a multilevel structure (e.g., respondents nested within organizations, students nested within schools). To resolve these problems, this study develops a multidimensional and multilevel version of the RERSM. The parameters can be estimated with existing computer software. Thus, there is no need to develop estimation procedures or corresponding computer programs. Simulation studies were conducted to evaluate the parameter recovery of the multidimensional RERSM, the multilevel RERSM, and the multidimensional and multilevel RERSM using WinBUGS. The results showed that the parameter recovery was generally satisfactory. An empirical example of the application of the multidimensional and multilevel RERSM to 2006 Program for International Student Assessment inventories about attitudes toward learning sciences is provided.     

Observing home-visiting quality across time: A longitudinal reliability study of the Home Visit Rating Scales

Empirical examination of home-visiting quality over time is lacking in the research literature. To this end, this study examined the most recent edition of a widely used observational measure of home-visiting quality, the Home Visit Rating Scales (HOVRS), over four repeated assessments spanning an average of 6 months. Psychometric quality for the HOVRS Total and Home Visitor Practice Scales, including item ratings and calculated scores, across time was demonstrated. Stability indicators for the Family Engagement Scales varied, with superior reliability often indicated for individual ratings over the calculated score. The exploration of interrelationships among family demographics and change in HOVRS scores over time were largely insignificant. Interestingly, a decline in the Home Visitor Practice Scale significantly related to increased symptoms of maternal depression, possibly suggesting that home-visitors’ focus shifted from children's needs to those of parent s when mental health concerns were observed. Although this study is an examination of one program model and poses limited generalizability, it sets the stage for advancing the HOVRS and longitudinal measurement of home-visiting quality.     

Assessing Children's Homework Performance: Development of Multi-Dimensional, Multi-Informant Rating Scales

Efforts to develop interventions to improve homework performance have been impeded by limitations in the measurement of homework performance. This study was conducted to develop rating scales for assessing homework performance among students in elementary and middle school. Items on the scales were intended to assess student strengths as well as deficits in homework performance. The sample included 163 students attending two school districts in the Northeast. Parents completed the 36-item Homework Performance Questionnaire — Parent Scale (HPQ-PS). Teachers completed the 22-item teacher scale (HPQ-TS) for each student for whom the HPQ-PS had been completed. A common factor analysis with principal axis extraction and promax rotation was used to analyze the findings. The results of the factor analysis of the HPQ-PS revealed three salient and meaningful factors: student task orientation/efficiency, student competence, and teacher support. The factor analysis of the HPQ-TS uncovered two salient and substantive factors: student responsibility and student competence. The findings of this study suggest that the HPQ is a promising set of measures for assessing student homework functioning and contextual factors that may influence performance. Directions for future research are presented.     

Capturing age-group differences and developmental change with the BASC Parent Rating Scales

Estimation of age-group differences and intra-individual change across distinct developmental periods is often challenged by the use of age-appropriate (but non-parallel) measures. We present a short version of the Behavior Assessment System (Reynolds & Kamphaus, 1998) Parent Rating Scales for Children (PRS-C) and Adolescents (PRS-A), which uses only their common-items to derive estimates of the initial constructs optimized for developmental studies. Measurement invariance of a three-factor model (Externalizing, Internalizing, Adaptive Skills) was tested across age-groups (161 mothers using PRS-C; 200 mothers using PRS-A) and over time (115 mothers using PRS-C at baseline and PRS-A five years later) with the original versus short PRS. Results indicated that the short PRS holds a sufficient level of invariance for a robust estimation of age-group differences and intra-individual change, as compared to the original PRS, which held only weak invariance leading to flawed developmental inferences. The importance of test–content parallelism for developmental studies is discussed.     

Construct Validity of ADHD/ODD Rating Scales: Recommendations for the Evaluation of Forthcoming DSM-V ADHD/ODD Scales

Exploratory structural equation modeling (SEM) was applied to a multiple indicator (26 individual symptom ratings) by multitrait (ADHD-IN, ADHD-HI and ODD factors) by multiple source (mothers, fathers and teachers) model to test the invariance, convergent and discriminant validity of the Child and Adolescent Disruptive Behavior Inventory with 872 Thai adolescents and the ADHD Rating Scale-IV and ODD scale of the Disruptive Behavior Inventory with 1,749 Spanish children. Most of the individual ADHD/ODD symptoms showed convergent and discriminant validity with the loadings and thresholds being invariant over mothers, fathers and teachers in both samples (the three latent factor means were higher for parents than teachers). The ADHD-IN, ADHD-HI and ODD latent factors demonstrated convergent and discriminant validity between mothers and fathers within the two samples. Convergent and discriminant validity between parents and teachers for the three factors was either absent (Thai sample) or only partial (Spanish sample). The application of exploratory SEM to a multiple indicator by multitrait by multisource model should prove useful for the evaluation of the construct validity of the forthcoming DSM-V ADHD/ODD rating scales.     

Assessing Culturally Different Students for Attention Deficit Hyperactivity Disorder Using Behavior Rating Scales

Behavior rating scales are commonly used in the assessment of attention deficit-hyperactivity disorder (ADHD). However, there is little information available concerning the extent to which scales are valid with culturally different students. This study explored the use of the ADHD-IV Rating Scale School Version with male Caucasian (CA) and African American (AA) students from ages 5 to 18 years. Teachers rated AA students higher on all symptoms across all age groups. LISREL analysis indicated that scale does not perform identically across groups. This was supported by the results of multidimensional scaling with suggested that there is a different relation between items across groups. Implications for research and practice are discussed.     

Diagnostic Classification Models for a Mixture of Ordered and Non-ordered Response Options in Rating Scales

When developing ordinal rating scales, we may include potentially unordered response options such as “Neither Agree nor Disagree,” “Neutral,” “Don’t Know,” “No Opinion,” or “Hard to Say.” To handle responses to a mixture of ordered and unordered options, Huggins-Manley et al. (2018) proposed a class of semi-ordered models under the unidimensional item response theory framework. This study extends the concept of semi-ordered models into the area of diagnostic classification models. Specifically, we propose a flexible framework of semi-ordered DCMs that accommodates most earlier DCMs and allows for analyzing the relationship between those potentially unordered responses and the measured traits. Results from an operational study and two simulation studies show that the proposed framework can incorporate both ordered and non-ordered responses into the estimation of the latent traits and thus provide useful information about both the items and the respondents.     

Investigating With IRT and MDS Approaches Translation and Adaptation of Rating Scales for Spanish-Speaking Populations

The goal of this study is to investigate how features of a rating scale developed for English-speaking populations interact with Spanish-speaking respondents' response styles and functional categories of judgment. A sample of 400 Spanish-speaking students took a translated scale and a scaling task developed to measure response sets and functional categories of judgment, respectively. Three response set models—extreme response, central tendency, and acquiescence—under two conditions—base and revised with respondents' functional categories—were studied with item response theory and multidimensional scaling methods. Revising the number of scale categories with the number of salient functional categories statistically improved fit of the base models. Multidimensional scaling results showed scale content features interacting with response styles and functional categories. Translation of rating scales requires adapting scale features to characteristics of target languages, such as salient response styles and respondents' functional categories of judgment.