The communication of friendly and hostile attitudes by verbal and non-verbal signals

Two experiments are reported here in which Ss were asked to rate videotapes of a performer reading friendly, neutral and hostile messages in a friendly, neutral or hostile non-verbal style. These messages and non-verbal styles had previously been presented independently to a separate group of Ss jor rating, in order to obtain an estimate of their individual strengths in terms of six rating scales, and thus permit a matching of verbal (messages) and non-verbal (styles) cues in the experiment where both types of cues were presented in combination. The results of both experiments indicate that non-verbal cues had a greater effect on ratings made on 7-point scales, such as hostile-friendly, than verbal cues. The magnitude of this greater effect of non-verbal cues, however, was dependent on the relative strength of non-verbal as opposed to verbal cues. In the first experiment, both types of cues were approximately equal in strength when ruted alone; here non-verbal cues accounted for 12.5 times us much variance us verbal cues, and produced 5.7 times as much shift on the ratitig scales. In the second experiment the verbal cues were much stronger than the non-verbal cues when rated alone. Here the relative effect of non-verbal cues in the second experiment was diminished; the ratio of non-verbal : verbal variance was now 1.67:1. When verbal and non-verbal signals were inconsistent, the performance was rated as insincere, unstable and confusing - which was not found in earlier experiments on the superior-inferior dimension.     

等级评定量表的最佳反应项数目:来自眼动研究的证据

在心理学研究中,等级评定量表是研究数据的常用测量工具。然而,研究者们使用的等级评定量表的版式并不一致,尤其表现在使用的反应项的数目不同。关于最佳反应项数目的研究只局限于从被试的最终反应结果来进行分析,而没有从被试对量表的反应过程来进行分析。根据前人相关文献,设计6种反应项的量表,并通过眼动实验来测量被试的信息处理过程。通过对反应时、净默认反应方式、极端反应方式和选项比例变化的综合比较,我们得出最佳的量表反应项数目是5。     

The convergent validity of MMPI and Rorschach scales: an extension using profile scores to define response and character styles on both methods and a reexamination of simple Rorschach response frequency

Past research indicated the convergence of Rorschach and MMPI scales may be a function of (a) simple Rorschach response frequency (R) or (b) complex response-character styles on both methods. In this study, new criteria were developed for defining the second assumption using F and K from the MMPI and R and Lambda from the Rorschach. Although substantially different from the factor criteria used previously (KS = .45 and .30), the new criteria still produced the expected pattern of correlations among MMPI and Rorschach scales. Averaged across 17 constructs, the new criteria produced strong validity coefficients for patients with similar styles (M composite r = .50), though they were less effective for patients with discordant styles (M composite r = -.27). It was also demonstrated that R by itself does not moderate convergent validity. Rather, statistical modeling with two sets of 300 random samples (a) demonstrated the prior findings related to R were the result of sampling error and (b) supported the general hypothesis that Rorschach and MMPI scales correlate to the extent response-character styles correlate. Implications are considered.     

Evidence that ideal and attractive figures represent different constructs: A replication and extension of Fingeret, Gleaves, and Pearson (2004)

Figure rating scales are commonly used to measure body dissatisfaction through discrepancy indexes (e.g., difference between current and ideal body) and recent methodological work by Fingeret, Gleaves, and Pearson suggests that distinctions between referent bodies (e.g., ideal or most attractive) are unnecessary. The purpose of the current study was to replicate these findings using both male and female figure rating scales and an opposite-sex comparison group. We examined the relationship between three referent figures (Ideal, Most Attractive to the Opposite Sex, and Most Attractive) and attractiveness attributions in 365 undergraduate men (n = 164) and women (n = 201). Results suggested that referent figures were associated with different attractiveness attributions for both male and female figures, but physical characteristics differed only in male figures. The impact of referent bodies on discrepancy indexes was also explored. Considerations for future research using figure rating scales are discussed.     

Short report: The impact of measurement conditions on the validity of self-assessment in a selection setting

Two experimental studies investigated the impact of format and instructions on the validity of SAs, using participants applying for junior management positions in a public sector organization. In Study 1 (607 participants) self-assessments (SAs) were valid when applicants rated overall competencies or the same competencies broken down into discrete elements, but not when providing behavioural evidence prior to rating competencies. The effect of social comparison instructions was also explored but found to have no impact on validity. Study 2 used a subset of 193 candidates from the previous study who attended the next stage of the selection process. Consistent with predictions it was found that unbalanced, positively toned scales were more valid than conventional scales, and highest validities were achieved when used with instructions requesting a realistic SA of strengths and weaknesses. Interestingly, gender differences were found in both studies. These findings are discussed in terms of their implications for both research and practice.     

Measuring Response Styles Across the Big Five: A Multiscale Extension of an Approach Using Multinomial Processing Trees

This study shows how to address the problem of trait-unrelated response styles (RS) in rating scales using multidimensional item response theory. The aim is to test and correct data for RS in order to provide fair assessments of personality. Expanding on an approach presented by Böckenholt (2012), observed rating data are decomposed into multiple response processes based on a multinomial processing tree. The data come from a questionnaire consisting of 50 items of the International Personality Item Pool measuring the Big Five dimensions administered to 2,026 U.S. students with a 5-point rating scale. It is shown that this approach can be used to test if RS exist in the data and that RS can be differentiated from trait-related responses. Although the extreme RS appear to be unidimensional after exclusion of only 1 item, a unidimensional measure for the midpoint RS is obtained only after exclusion of 10 items. Both RS measurements show high cross-scale correlations and item response theory-based (marginal) reliabilities. Cultural differences could be found in giving extreme responses. Moreover, it is shown how to score rating data to correct for RS after being proved to exist in the data.     

Extreme Response Style and the Measurement of Intra-Individual Variability in Affect

Extreme response style (ERS) has the potential to bias the measurement of intra-individual variability in psychological constructs. This paper explores such bias through a multilevel extension of a latent trait model for modeling response styles applied to repeated measures rating scale data. Modeling responses to multi-item scales of positive and negative affect collected from smokers at clinic visits following a smoking cessation attempt revealed considerable ERS bias in the intra-individual sum score variances. In addition, simulation studies suggest the magnitude and direction of bias due to ERS is heavily dependent on the mean affect level, supporting a model-based approach to the study and control of ERS effects. Application of the proposed model-based adjustment is found to improve intra-individual variability as a predictor of smoking cessation.     

Review of clinical validation of ADHD behavior rating scales

The purpose of this review is to assess the range of overall accuracies for Attention Deficit/Hyperactivity Disorder (ADHD) behavior rating scales evaluated in clinical validation studies. Studies were characterized according to the evidence standards of the American Academy of Neurology (AAN). Studies were excluded due to major design problems such as overfitting by discriminant analysis. The 13 included evaluations of rating scales revealed overall accuracy in the range of 59%-79% with a pooled mean of 69% (+/-7%, standard deviation) and a pooled sample size of 2228 subjects from nine studies. While some of the excluded studies demonstrated higher overall accuracies (>79%), these studies were observed to have factors in experimen tal design and statistics that are known to unduly inflate accuracy. We recommend further research following the full AAN standards, namely well-designed, blinded, pro spective studies of rating scales applied to clinically representative samples evaluated with a clinical standard.     

Concordance of teacher-rated and performance-based measures of executive functioning in preschoolers

The most common method used to evaluate child behavior and functioning is rating scales completed by parents and/or teachers. Given that executive functioning (EF) plays a fundamental role in the developing child’s cognitive, behavioral, and social-emotional development, it would be ideal if ratings of EF and performance-based EF measures assess the same construct. However, most studies report a small to negligible association between performance-based measures and ratings of EF. There are few studies investigating this association for preschoolers, and most only include parent ratings. Teachers may be more reliable reporters of EF behaviors due to the higher demand for EF skills in the preschool setting than at home and because teachers may have a better sense of what behaviors are normative. In this study, we reviewed the associations between three EF rating scales completed by teachers on 243 preschool children. Results showed small to moderate correlations with EF measures of inhibition and cognitive flexibility/switching for all three scales, with the strongest associations observed between Child Behavior Rating Scale (CBRS) Behavioral Regulation subscale and child EF measures. Exploratory multivariate path analyses showed that, after controlling for age, sex, and socioeconomic status (SES), Behavioral Regulation significantly predicted performance-based measures of EF and accounted for incrementally more variance in the models. We conclude that in ideal situations, it is best to measure EF using both rating scales and performance-based measures of EF. The CBRS seems to be a sensitive measure of EF in preschoolers and may be a helpful brief screening tool for use with teachers.     

“A joke a day keeps the doctor away?” Meta‐analytical evidence of differential associations of habitual humor styles with mental health

Humor and mental health are interconnected as is evidenced by a large number of studies. However, associations are only small and inconsistent as the operationalization of humor poses a methodological challenge. The Humor Styles Questionnaire (HSQ ) differentiates four humor styles that might be beneficial or harmful to mental health. The aim of the present study was to meta‐analytically aggregate studies using the HSQ to assess the associations of different humor styles with four areas of mental health (self‐esteem, life satisfaction, optimism, depression). An extensive electronic database literature search identified 37 studies that reported correlations between the HSQ scales and the four areas of mental health in 45 independent samples (total N = 12,734). In total, 16 meta‐analyses were conducted. Moderating effects of participant age, sex, and geographic region were examined via subgroup analyses and meta‐regression. Humor styles differed in terms of their associations with mental health. Health‐promoting humor styles were overall positively correlated with mental health (small‐to‐medium effect sizes). Self‐defeating humor was overall negatively correlated with mental health. Aggressive humor was overall unrelated with mental health. Moderator analyses suggested geographic differences (Eastern vs. Western samples) and sex differences for some of these associations. Fostering specific humor styles may be beneficial for mental health. In addition, observing the habitual use of humor styles might help therapists to develop a better understanding of their clients. Differences in the utilization and the correlates of humor styles in Eastern and Western societies, and sex differences, need to be addressed in future research.     

Not worth the extra cost? Diluting the differentiation ability of highly rated products by altering the meaning of rating scale levels

Over the last decade, the use of rating scales has grown in popularity in various fields, including customer online reviews and energy labels. Rating scales convey important information on attributes of products or services that consumers evaluate in their purchase decisions. By applying multidimensional scaling, this paper reveals that the meaning of the levels of a rating scale can be altered by manipulating the labeling of the rating scale levels. The study reveals that consumers perceive product attributes as being more similar if the labels share similar or identical linguistic or visual characteristics. In addition, two choice‐based conjoint studies examine whether the way consumers make their choices among products can be influenced by changing the labeling of rating scale levels. The results show that a manipulation of the meaning of rating scale levels diminishes both the importance of the rating scale information and consumers' willingness to pay a premium for a rating upgrade. Copyright © 2013 John Wiley & Sons, Ltd.     

Effects of instructions and rating scales on item selection for the BSRI scales

Walkup and Abbott (1978) stated that Edwards and Ashworth's (1977) failure to replicate Bem's (1974) selection of items for the Masculinity and Femininity Scales of the Bern Sex Role Inventory (BSRI) may be attributed to differences in the instructions and anchored rating scales used in the two studies. The present study tested the hypothesis that presence of various interaction effects involving instructions and rating scales would influence the acceptability of items for the BSRI Masculinity and Femininity Scales. Results based on the evaluation of individual items by Bem's item selection criteria in each of the four experimental conditions obtained by systematically manipulating two instructions (Bem's and Edwards' instructions) and two rating scales (Bem's and Edwards' rating scales) and also those based on the analysis of variance of item mean desirability ratings from the four experimental conditions supported the hypothesis.     

Developing equivalent Chinese and English scale‐point labels for rating scales used in survey research

Rating scales are the most frequently‐used response tool in surveys, and the scale levels are commonly described with words [verbal scale‐point labels (VSPL)]. In this study, Chinese VSPL were identified which employ five‐point scales that are psychometrically equivalent to English VSPL. In several bilingual studies, a total of 61 Chinese and 44 English items addressing intensity, frequency, and agreement rating modalities were tested. For each VSPL, three aspects were measured: position between minimum and maximum, familiarity, and appeal. The correspondence between pertinent Chinese and English words was also assessed. Based on these findings, we recommend specific VSPL that are best‐suited for achieving equivalence between Chinese and English in rating scales.     

Performance Appraisal Process Research in the 1980s: What Has It Contributed to Appraisals in Use?

In the early 1980s, Landy and Farr (1980) and Feldman (1981) redirected performance appraisal research from issues related to the development of psychometrically sound rating scales to those involving the cognitive processes of raters. Since that time, several reviews have attempted to translate principles from social cognition and cognitive psychology to the specific conditions of formal appraisal systems in work-oriented organizations. In addition, a number of empirical studies have been conducted on this topic. This article reviews empirical research during the 1980s that focused on performance appraisal processes, particularly the research that has focused upon rating accuracy. The review is structured around a three-stage process model of gathering, storing, and retrieving information about social stimuli for the purposes of rating performance. Factors affecting this process are clustered into four categories: appraisal settings, ratees, raters, and the nature of the scales used for the appraisal. Once reviewed, the research is evaluated in terms of its contributions to improving the quality of appraisal systems as they are used in organizations.     

A multitrait-multisource confirmatory factor analytic approach to the construct validity of ADHD rating scales

Confirmatory factor analysis was used to model a multitrait-multisource design to evaluate the construct validity of attention-deficit/hyperactivity disorder (ADHD) rating scales. The 2 trait factors were the ADHD inattention and hyperactivity/impulsivity dimensions. The 2 source factors were parents and teachers. In Study 1, parents and teachers rated 1,475 Australian elementary school children on the ADHD symptoms. In Study 2, parents and teachers rated 285 Brazilian elementary school children on the ADHD symptoms. Similar results occurred in both studies with most of the ADHD symptoms containing more source than trait variance, thus providing weak evidence for the convergent and discriminant validity of the symptoms as measured by rating scales. The study outlines the implications of such strong source effects for understanding ADHD.     

Effects of Instructions and Rating Scales on Item Selection for the BSRI Scales

Walkup and Abbott (1978) stated that Edwards and Ashworth's (1977) failure to replicate Bem's (1974) selection of items for the Masculinity and Femininity Scales of the Bern Sex Role Inventory (BSRI) may be attributed to differences in the instructions and anchored rating scales used in the two studies. The present study tested the hypothesis that presence of various interaction effects involving instructions and rating scales would influence the acceptability of items for the BSRI Masculinity and Femininity Scales. Results based on the evaluation of individual items by Bem's item selection criteria in each of the four experimental conditions obtained by systematically manipulating two instructions (Bem's and Edwards' instructions) and two rating scales (Bem's and Edwards' rating scales) and also those based on the analysis of variance of item mean desirability ratings from the four experimental conditions supported the hypothesis.     

Repression and effective coping styles

This study investigated the relationship between repressive coping style as defined by Weinberger (1990)—low reported anxiety and high reported defensiveness—and several individual difference measures related to general coping styles. Twenty‐nine studies looking at cognitive, behavioural, and individual difference factors associated with repressive coping were tabulated and reviewed. In the current study 116 women and 32 men were administered a questionnaire including measures of personality, ways of coping, self‐monitoring, interpersonal influence, and locus of control to further explore the repressor coping style. Two measures, as opposed to the usual one, were used to categorize repressors: the use of anxiety and social desirability measures of Weinberger, Schwartz and Davidson (1979) and the Gudjonsson (1981) method using Neuroticism and Lie scales from the EPQ(R). Analysis of variance showed that repressors reported using more positive/healthy and less negative/unhealthy coping styles. The results did not change substantially when more extreme scoring groups were used. Factor analysis of the scales revealed six clear factors relating to ways of coping; again, repressors scored highly on the factor denoting positive coping. It was also found that the Lie and Neuroticism scales of the revised EPQ personality questionnaire of Eysenck, Eysenck and Barrett (1985) could be used as a substitute for anxiety and defensiveness to predict repression with considerable success. However results from both one‐ and two‐way analysis of variance indicated that the two different ways of classifying repressors was not totally inter‐changeable as the overlap in results occurred only with highly significant effects. The results are discussed in the light of previous investigations into repression which suggested that repressors report an overly optimistic way of coping in order to avoid negative affect. Copyright © 1999 John Wiley & Sons, Ltd.     

Investigating With IRT and MDS Approaches Translation and Adaptation of Rating Scales for Spanish-Speaking Populations

The goal of this study is to investigate how features of a rating scale developed for English-speaking populations interact with Spanish-speaking respondents' response styles and functional categories of judgment. A sample of 400 Spanish-speaking students took a translated scale and a scaling task developed to measure response sets and functional categories of judgment, respectively. Three response set models—extreme response, central tendency, and acquiescence—under two conditions—base and revised with respondents' functional categories—were studied with item response theory and multidimensional scaling methods. Revising the number of scale categories with the number of salient functional categories statistically improved fit of the base models. Multidimensional scaling results showed scale content features interacting with response styles and functional categories. Translation of rating scales requires adapting scale features to characteristics of target languages, such as salient response styles and respondents' functional categories of judgment.     

Rating scales for hyperactivity: Concurrent validity,reliability, and decisions to label for the Conners and Davids abbreviated scales

Rating scales have become the instrument of choice in labeling and assessing change in behavior of hyperactive children. However, several criticisms have recently have levied against their use. The present investigation examined the concurrent validity, and inter- and intrarater reliability for the Abbreviated Teacer Questionnaire (ATQ, Conners, 1973) and the Rating Scales for Hyperkinesis (Davids, 1971). Sixteen teachers from two special and two regular schools (grades 1-4) rated 211 normal and 49 special children using both scales. High correlations were found suggesting excellent predictability between scales and considerable stability across time and rater. Lower scores on a subsequent rating relative to an initial rating were demonstrated, dependent on time between ratings but independent of (a) teacher expectation of treatment gains, (b) bias produced by rating selected children, and (c) whether children were hyperactive or normal. Use of initial and infrequent rating scores versus subsequent, closely spaced ratings was related to the rater's objective (e.g., diagnosis, treatment, or assessment).