胜任特征模型评价的一致性研究

本研究通过比较评价者内部和评价者之间对军事指挥院校学员胜任特征评价的一致性,探讨应用胜任特征模型进行评价时,其评价结果的一致性问题。结果表明：(1)应用胜任特征模型进行评价时,来自同一个评价源的评价者之间和来自不同评价源的评价者之间均具有较好的评价一致性;(2)同事评价者之间对胜任特征评价的一致性高于上级评价者之间评价的一致性;(3)评价者内部一致性越高,并不能改善评价者间的一致性;(4)胜任特征模型在不同的情景中应用时具有较好的稳定性。     

Abstract Conceptual Feature Ratings Predict Gaze Within Written Word Arrays: Evidence From a Visual Wor(l)d Paradigm

The Abstract Conceptual Feature (ACF) framework predicts that word meaning is represented within a high‐dimensional semantic space bounded by weighted contributions of perceptual, affective, and encyclopedic information. The ACF, like latent semantic analysis, is amenable to distance metrics between any two words. We applied predictions of the ACF framework to abstract words using eyetracking via an adaptation of the classical “visual word paradigm” (VWP). Healthy adults (n = 20) selected the lexical item most related to a probe word in a 4‐item written word array comprising the target and three distractors. The relation between the probe and each of the four words was determined using the semantic distance metrics derived from ACF ratings. Eye movement data indicated that the word that was most semantically related to the probe received more and longer fixations relative to distractors. Importantly, in sets where participants did not provide an overt behavioral response, the fixation rates were nonetheless significantly higher for targets than distractors, closely resembling trials where an expected response was given. Furthermore, ACF ratings which are based on individual words predicted eye fixation metrics of probe‐target similarity at least as well as latent semantic analysis ratings which are based on word co‐occurrence. The results provide further validation of Euclidean distance metrics derived from ACF ratings as a measure of one facet of the semantic relatedness of abstract words and suggest that they represent a reasonable approximation of the organization of abstract conceptual space. The data are also compatible with the broad notion that multiple sources of information (not restricted to sensorimotor and emotion information) shape the organization of abstract concepts. While the adapted “VWP” is potentially a more metacognitive task than the classical visual world paradigm, we argue that it offers potential utility for studying abstract word comprehension.     

Age effects on the employability-career success relationship

This study investigated the similarity of the factor structure for self-reported versus supervisor-rated employability for two age groups of workers, and then validated a career success enhancing model of employability across the two age groups. The results confirmed a two-factor model including self-reported and supervisor-rated employability as underlying factors. Moreover, Multi-Group Structural Equation Modeling (SEM) indicated that for the youngsters both self- and supervisor ratings of employability related significantly to objective career success outcomes. However, for the over-forties self-rated employability related positively to promotions throughout the career, while the supervisor ratings related negatively to overall promotions. The findings have important implications for performance appraisal practices aimed at increasing life-long employability and career success.     

What sources contribute to variance in observer ratings? Using generalizability theory to assess construct validity of psychological measures

Cronbach and Meehl (Psychol. Bull. 1955; 52:281–302) stated that the key question to be addressed when assessing construct validity is ‘What sources contribute to variance in test performance?’ We illustrate the utility of generalizability theory (GT) as a conceptual framework that encourages psychological researchers to address this question and as a flexible set of analytic tools that can provide answers to inform both substantive theory and measurement practice. To illustrate these capabilities, we analyze observer ratings of 27 caregiver–child dyads, focusing on the importance of situational (contextual) factors as sources of variance in observer ratings of caregiver–child behaviors. Cross‐situational consistency was relatively low for the categories of behavior analyzed, indicating that dyads vary greatly in their interactional patterns from one situation to the next, so that it is difficult to predict behavioral frequencies in one context from behaviors observed in a different context. Our findings suggest that single‐situation behavioral measures may have limited generalizability, either to behavior in other contexts or as measures of global interaction tendencies. We discuss the implications of these findings for research and measurement design in developmental psychology. Copyright © 2008 John Wiley & Sons, Ltd.     

The prediction of Honesty–Humility-related criteria by the HEXACO and Five-Factor Models of personality

We examined two questions involving the relative validity of the HEXACO and Five-Factor Models of personality structure. First, would the HEXACO model outpredict the Five-Factor Model (FFM) with regard to several diverse criteria that are conceptually relevant to the Honesty–Humility dimension of personality? If so, would the addition of a proxy Honesty–Humility scale—as computed from relevant facets of the FFM Agreeableness domain—allow the FFM to achieve predictive validities matching those of the HEXACO model? Results from self- and observer ratings in three samples (each N > 200) indicated that the HEXACO model showed considerable predictive validity advantages over the FFM. When a measure of Honesty–Humility derived from the FFM was added to the original five domains of that model, the predictive validity reached that of the HEXACO model for some criteria, but remained substantially below for others.     

Maternal ratings of infant regulatory competence from 6 to 12 months: influence of perceived stress, birth-weight, and intervention: a randomized controlled trial

Mothers’ reports of preterm and term infants’ temperament from 6 to 12 months of age were studied, with intervention and stress as predictors. Preterm infants with a birth weight <2000 g were randomized to an intervention (71) or a control (69) group. A control group of healthy term infants (74) was also established. The intervention was a modified version of the “Mother–Infant Transaction Program”, aimed at sensitizing caregivers to the infants’ individual characteristics. Temperament was measured with the Infant Behavior Questionnaire, and stress with the Parenting Stress Index. There were no group differences in temperament at 6 or 12 months or in change during the same period. There was a strong association between stress and negative reactivity in the preterm control group at 12 months. In the preterm intervention group, there were strong negative correlations between stress and regulatory competence at 6 months. The intervention seemed to change the relationship between stress and temperament. The strength of this association indicates that the intervention sensitized mothers to the temperamental regulatory competence of their preterm infants.     

No grammatical gender effect on affective ratings: evidence from Italian and German languages

In this study, we tested the linguistic relativity hypothesis by studying the effect of grammatical gender (feminine vs. masculine) on affective judgments of conceptual representation in Italian and German. In particular, we examined the within- and cross-language grammatical gender effect and its interaction with participants’ demographic characteristics (such as, the raters’ age and sex) on semantic differential scales (affective ratings of valence, arousal and dominance) in Italian and German speakers. We selected the stimuli and the relative affective measures from Italian and German adaptations of the ANEW (Affective Norms for English Words). Bayesian and frequentist analyses yielded evidence for the absence of within- and cross-languages effects of grammatical gender and sex- and age-dependent interactions. These results suggest that grammatical gender does not affect judgments of affective features of semantic representation in Italian and German speakers, since an overt coding of word grammar is not required. Although further research is recommended to refine the impact of the grammatical gender on properties of semantic representation, these results have implications for any strong view of the linguistic relativity hypothesis.     

Political skill as neutralizer of felt accountability—job tension effects on job performance ratings: A longitudinal investigation

This study examined the effects of felt accountability, political skill, and job tension on job performance ratings. Specifically, we hypothesized that felt accountability would lead to higher (lower) job performance ratings when coupled with high (low) levels of political skill, and that these relationships would be mediated by job tension. Data were gathered at multiple times over a one-year period (i.e., baseline performance, attitudinal variables one month later, supervisor reports of subordinate performance six months and one year after baseline performance was measured). Strong support was shown for the total effects model, whereby political skill moderated felt accountability—job performance ratings, felt accountability—job tension, and job tension—job performance ratings relationships. However, more focused analyses demonstrated that political skill most strongly moderated the job tension—job performance ratings linkage. Implications, strengths and limitations, and directions for future research are provided.     

Why convene rater teams: An investigation of the benefits of anticipated discussion,consensus, and rater motivation

This study explores the importance of anticipated group discussion, the consensus decision rule, and rater motivation in determining how well rater teams identify ratee behaviors, i.e., behavioral accuracy. Results, based on 382 raters in 111 teams, suggest that the anticipation of group discussion can improve behavioral accuracy, but it appears that the benefits of discussion-only teams are limited to this anticipation effect. Furthermore, it also appears that rater motivation plays an important role in this type of team. Rater teams required to reach consensus, however, appear to show improved behavioral accuracy, regardless of whether raters can anticipate the consensus discussion and regardless of rater motivation levels. Implications, especially for assessment centers, are discussed.