THE DEVELOPMENT OF BEHAVIORAL OBSERVATION SCALES FOR APPRAISING THE PERFORMANCE OF FOREMEN

Behavioral observation scales (BOS) were developed for first line foremen. BOS are similar to behavioral expectation scales (BES) in that both are based on a job analysis procedure known as the critical incident technique. However, the BOS differ from BES in that, in developing BOS, (a) a group of individuals is observed and rated on a five-point scale as to the frequency with which they engage in the behavior described by each incident/statement, (b) a total score for each individual is determined by summing the observer's responses for each behavioral item, and (c) an item analysis (or factor analysis, depending upon the sample size) is conducted to select the most discriminating items. Those items with the highest correlations with the total score on a scale are retained to form one behavioral criterion or scale (BOS).     

ISSUES AND STRATEGIES FOR REDUCING THE LENGTH OF SELF-REPORT SCALES

Greater understanding of the complex interrelationships among work-relevant constructs has increased the number of constructs on organizational surveys. Good psychometric practice also dictates the use of multiple items per construct. The net result has been longer surveys. Longer surveys take more time to complete, tend to have more missing data, and have higher refusal rates than short surveys. Arguably, then, techniques for reducing the length of scales while maintaining psychometric quality are worthwhile. Little guidance exists on how to reduce the length of a multi-item scale and we argue that the most common technique, maximizing internal consistency, is problematic and should be avoided. We present a set of item "quality indices" to help conceptualize the competing issues that influence item retention decisions. Statistical analysis of an example case using these indices suggested that there are 3 key aspects of item quality to consider when reducing a scale. We describe strategies that can assist scale developers in using these 3 aspects of item quality when making scale reduction decisions.     

A MODEL FOR THE USE OF ATTITUDE SCALES ACROSS CULTURES

To minimize the verbal content of the attitude measurement situation with Papua New Guinea students, a set of pictorial items was selected for use in the place of verbal items. Initially, 150 pictures related to concepts in the Wilson-Patterson Attitude Inventory (WPAI) were described by a group of 50 students. Seventy of these items, those which were described in a consistent way, were presented to 227 students along with a Papua New Guinea version of the WPAI. A final set of 25 pictures was selected on the basis of their discriminative power, internal consistency, and their content validity. Reliability and validity measures and the rotated factor structure of the 25 item pictorial scale were consistent with the WPAI.     

THE PRACTICALITY OF BEHAVIORAL OBSERVATION SCALES, BEHAVIORAL EXPECTATION SCALES, AND TRAIT SCALES

The practicality of three appraisal instruments were measured in terms of user preference, namely, behavioral observation scales (BOS), behavioral expectation scales (BES), and trait scales. A questionnaire containing items pertaining to differentiating good from poor performers, objectivity, providing feedback, suggesting training needs, and ease of use was administered to managers and their subordinates. In all instances, BOS were preferred to BES, and in all but two instances, BOS were viewed as superior to trait scales. Trait scales were felt to be as good if not better than BES. A second questionnaire administered to attorneys indicated that BOS would be easier to defend in the courtroom than either BES or trait scales.     

AN EXTENSION AND FIELD TEST OF THE RETRANSLATION METHOD FOR DEVELOPING RATING SCALES

The retranslation method for rating scale development was used to develop a scale to evaluate an educational television series designed to teach basal reading skills. This extension of the method, outside the context of job performance rating, was evaluated using both developmental and actual field use data. The results during the developmental stage of the method are generally supportive of the use of the technique in this situation; the results from the field test yielded mixed support.     

A COMPARISON OF TWO TECHNIQUES FOR THE MEASUREMENT OF INTERNATIONAL JUDGMENT

On trouve, dans la littérature, de bons arguments en faveur d'une procédure pour l'étude des jugements qui permette aux répondants de fournir leurs propres dimensions de réponses. Cette approche diffère du différenciateur sémantique, entre autres, dans lequel les échelles utilisées sont fournies par le chercheur. Les AA. examinent dans cet article les relations entre les résultats obtenus avec l'une et l'autre des deux méthodes. On demande aux sujets de juger du gouvernement actuel de io pays en termes de ressemblance-dissemblance par paires; les réponses sont soumises à une analyse pluridimensionnelle (multidimensional scaling) afin de mettre en évidence les dimensions qui sous-tendent les jugements. Ces représentations, en termes de dimensions, sont alors comparées aux notes correspondant aux facteurs Evaluation et Puissance du différenciateur sémantique. On constate que pour l'ensemble des sujets la dimension principale mise en évidence par l'analyse pluridimensionnelle correspond nettement au facteur Evaluation et que la dimension qui vient en second rang est en liaison avec le facteur Puissance. Des analyses séparées ont été également faites en fonction de l'orientation politique des répondants : on constate que les jugements que portent sur les gouvernements les sujets de droite sont beaucoup plus teintés d'Evaluation.     

THE UTILITY OF CURRICULUM-BASED MEASUREMENT FOR EVALUATING THE EFFECTS OF METHYLPHENIDATE ON ACADEMIC PERFORMANCE

Two case studies were conducted to investigate the utility of curriculum-based measurement of math and reading for evaluating the effects of methylphenidate on the academic performance of 2 students diagnosed with attention deficit hyperactivity disorder. Following baseline measurement, double-blind placebo-controlled procedures were employed to evaluate each student's response to three levels (5 mg, 10 mg, and 15 mg) of the medication. Results of the first study suggest that the curriculum-based measures were sensitive indicators of the student's response to medication. This finding was replicated in the second study. In the second study, when the student's follow-up dose of medication was based on trial-phase data, follow-up performance was improved compared to baseline performance. These case studies suggest that further research is warranted on the utility of curriculum-based measurements for monitoring and evaluating stimulant medication interventions with children with this disorder.     

CONSTRUCT VALIDATION OF THE COMREY PERSONALITY SCALES

To investigate the construct validity of the Comrey Personality Scales, this personality test and a biographical data inventory inquiring about past and present life circumstances were administered to 209 volunteer U.C.L.A. students. Biographical data variables were correlated with scores on the eight personality scales. Personality scale score estimates by the authors, based on examination of the respondents' biographical data responses, were also correlated with actual scale scores. The personality scales which received the best confirmation were Social Conformity vs Rebelliousness and Orderliness vs. Lack of Compulsion. Support for the construct validity of the other personality scales varied from moderate to negligible.     

SCALES FOR SUBJECTIVE DISTANCE

By means of the method of ratio estimation, scale values were obtained for subjective distance. In three experiments different stimulus ranges of the objective distances were used. It was found: ( 1 ) Subjective distance is a power function of the objective distance. ( 2 ) The exponent of the function varies with the stimulus range. With increasing stimulus range the exponent has a tendency to decrease. It is conceivable that the change of the exponent may be explained by an adaptation of the subjective range to the stimulus range.     

PSYCHOPHYSICAL SCALES OF THE ODOR INTENSITY OF AMYL ACETATE

The paper presents psychophysical scales of amyl acetate in benzyl benzoate sniffed from cotton. Four scales obtained by direct scaling procedures, ratio estimation and magnitude estimation, yield functions of the form R = kSⁿ , with n ranging from 0.39 to 0.57. These data support earlier findings that the intensity of smell is a negatively accelerated function of stimulus intensity. In addition, comparison of magnitude and category scales indicate that the subjective intensity of smell is a prothetic continuum. Finally, further analysis of subjective ratios as a function of stimulus ratios again shows that ( a ) the magnitude scale is a ratio scale and ( b ) the function obtained conforms to the power law.     

THE DISCRIMINANT VALIDITY OF SCALES DEVELOPED BY RETRANSLATION

The retranslation procedure is designed to yield unambiguous sets of behaviors exemplifying conceptually important and distinct performance dimensions. Previous research has shown that dimensions developed with the retranslation procedure possess only low to moderate discriminant validity. This study proposed a multiple-group factor analysis of the scaled behavioral examples for improving the discriminant validity of retranslation dimensions. Multitrait-multimethod analyses of actual ratings indicated that the factor-analytic procedure led to an improvement in discriminant validity. Nonetheless, the amount of discriminant validity possessed by the dimensions was still low. Several suggestions were made for further improving the discriminant validity of dimensions developed with the retranslation procedure.     

ON THE PROBLEM OF CATEGORY SCALES IN PSYCHOPHYSICS

E isler , H. On the problem of category scales in psychophysics. Scand. J. Psychol ., 1962, 3 , 81–87.—In the first Part of the paper it is shown that, if three assumptions are granted, the category scale must be logarithmically related to the magnitude scale: K =α log φ +β . In the second part of the paper, the relation K =α log ( kφ+ q) +β is derived, starting from the assumptions that the category scale is a pure function of discrimination and that discrimination is appropriately described by the linear generalization of Weber's law for prothetic continua. We can reconcile these two formulas by defining the zero-point of the magnitude scale as the point where variability vanishes.     

BRIGHTNESS SCALES FOR MONOCHROMATIC LIGHT

By means of amethod of ratio estimation, scale values were obtained for the subjective brightness of various physical intensities of monochromatic light of various wave lengths. In a second experiment the scale was constructed by a method of magnitude estimation. The brightness functions were studied by plotting the scale values against stimulus intensity for each wave length. The two experiments gave essentially the same results. It was shown: (1) Brightness of monochromatic light is a power function of stimulus intensity. The exponent of the function is approximately one-third for all wave lengths. (2) Properties of the brightness functions can explain certain empirical relations between brightness, hue and saturation.     

心理模糊性的测量

本文初步得到如下三点结果:1.表征了人类心理概念化具有模糊集(Fuzzy Sets)特征。2.经验地评论了L.A.Zadeh的模糊语义定量的某些算子或运算规则。3.利用模糊熵的概念测量了人类的某些推论过程中的智力差异。     

A SIMULTANEOUS DISCRIMINATION PROCEDURE FOR THE MEASUREMENT OF VISION IN NONVERBAL CHILDREN1

Few nonverbal developmentally disabled children ever receive adequate vision assessment because of their limited language skills. The present study details a simultaneous discrimination procedure for measuring subjective visual acuity in such children. A stimulus fading procedure was used to train a discrimination between Snellen Es differing in orientation, and a psychophysical tracking method was used to determine acuity thresholds. The procedure was tested with 11 nonverbal autistic and schizophrenic children and validated with four nonpsychotic children. Eight of the psychotic children were successfully examined in one to three sessions. Two of these children were identified as having significant acuity losses. The validity assessment showed that the experimental procedure resulted in thresholds equal to or slightly lower than those obtained with the Illiterate E chart.     

THE PROBLEM OF MAGNITUDE AND CATEGORY SCALES: INTERPRETATION OF RESULTS

J unge , K. The problem of magnitude and category scales: interpretation of results. Scand. J. Psychol ., 1962, 3 , 215–218.—It is proposed that three main factors are responsible for the relation between magnitude and category scales: (1) ratio judgments on a transformed category scale, (2) interval judgments, and (3) hybrid judgments. The last term refers to the tentative explanation that magnitude judgments are a cross between pure ratio and interval judgments.     

真实自豪与自大自豪倾向量表的修订及适用

对Tracy和Robins编制的真实自豪与自大自豪倾向量表进行中文版修订并初步考察我国大学生自豪感的特点,分别选取202名和484名大学生进行问卷预测和正式施测。验证性因素分析表明修订后量表仍保持英文版的双维结构,两分量表的α系数为0.785和0.743,重测信度为0.978和0.952。修订后量表具有良好的心理测量学指标,可以用于我国大学生的测试。进一步发现我国大学生的真实自豪显著高于自大自豪,大学生自豪感不存在性别差异,自大自豪存在显著的年级差异。     

BEHAVIORAL OBSERVATION SCALES AND THE EVALUATION OF PERFORMANCE APPRAISAL EFFECTIVENESS

An earlier article by the present authors (Bernardin and Kane, 1980) pointed out a serious flaw in the conceptual basis of Behavioral Observation Scales (BOS). The present article explains this flaw in more detail and shows that its solution would make BOS indistinguishable from other methods already in existence. The focus then shifts from the conception of appraisal methods to their evaluation with the presentation of a discussion of the special problems of error capitalization that arise in the use of item analysis in the development of multi-item rating scales. This discussion proceeds to describe the correct approach to removing the effects of error capitalization in the evaluation of the psychometric properties of such rating scales.