创造力测评中的评分者效应

创造力测评中的评分者效应(rater effects)是指在创造性测评过程中, 由于评分者参与而对测评结果造成的影响.评分者效应本质上源于评分者内在认知加工的不同, 具体体现在其评分结果的差异.本文首先概述了评分者认知的相关研究, 以及评分者,创作者,社会文化因素对测评的影响.其次在评分结果层面梳理了评分者一致性信度的指标及其局限, 以及测验概化理论和多面Rasch模型在量化,控制该效应中的应用.最后基于当前研究仍存在的问题, 指出了未来可能的研究方向, 包括深化评分者认知研究,整合不同层面评分者效应的研究, 以及拓展创造力测评方法和技术等.     

多源评价的特点与内在机制

多源评价在国外企业中的运用已日益成熟, 但在我国还停留在探索与发展阶段。基于已有的研究发现, 围绕评价过程、评价源及被评价者三方面对多源评价的特点及内在机制进行了探讨与分析。从评价过程看, 其评价目的具有多重性, 评价形式注重匿名性, 且评价结果的合理应用非常重要; 从评价源看, 不同评价源间的评价一致性较低, 且易造成晕轮效应和宽大效应; 从被评价者来看, 个体对多源评价结果的反应, 受到个性特征、反馈信号及自我-他人评价间差距等因素影响。研究也发现, 多源评价所带来的绩效改进结果具有不稳定性。基于此, 如何提高多源评价过程的有效性与准确性, 改善评价者对评价结果的反应, 以及如何对多源评价结果进行有效汇总等是未来值得研究的重要内容。     

Classification objects, ideal observers & generative models

A successful vision system must solve the problem of deriving geometrical information about three-dimensional objects from two-dimensional photometric input. The human visual system solves this problem with remarkable efficiency, and one challenge in vision research is to understand how neural representations of objects are formed and what visual information is used to form these representations. Ideal observer analysis has demonstrated the advantages of studying vision from the perspective of explicit generative models and a specified visual task, which divides the causes of image variations into the separate categories of signal and noise. Classification image techniques estimate the visual information used in a task from the properties of “noise” images that interact most strongly with the task. Both ideal observer analysis and classification image techniques rely on the assumption of a generative model. We show here how the ability of the classification image approach to understand how an observer uses visual information can be improved by matching the type and dimensionality of the model to that of the neural representation or internal template being studied. Because image variation in real world object tasks can arise from both geometrical shape and photometric (illumination or material) changes, a realistic image generation process should model geometry as well as intensity. A simple example is used to demonstrate what we refer to as a “classification object” approach to studying three-dimensional object representations.     

Characterizing perceptual learning with external noise

Performance in perceptual tasks often improves with practice. This effect is known as ‘perceptual learning,’ and it has been the source of a great deal of interest and debate over the course of the last century. Here, we consider the effects of perceptual learning within the context of signal detection theory. According to signal detection theory, the improvements that take place with perceptual learning can be due to increases in internal signal strength or decreases in internal noise. We used a combination of psychophysical techniques (external noise masking and double-pass response consistency) that involve corrupting stimuli with externally added noise to discriminate between the effects of changes in signal and noise as observers learned to identify sets of unfamiliar visual patterns. Although practice reduced thresholds by as much as a factor of 14, internal noise remained virtually fixed throughout training, indicating learning served to predominantly increase the strength of the internal signal. We further examined the specific nature of the changes that took place in signal strength by correlating the externally added noise with observer’s decisions across trials (response classification). This technique allowed us to visualize some of the changes that took place in the linear templates used by the observers as learning occurred, as well as test the predictions of a linear template-matching model. Taken together, the results of our experiments offer important new theoretical constraints on models of perceptual learning.     

Why are moderators of self-other agreement difficult to establish?

Agreement between the self and other rated personality profiles was studied in two samples involving 11,096 speakers of two languages, Dutch and Estonian, who completed two different personality questionnaires, the NEO-PI-3 and HEXACO-PI-R. An outstanding agreement was achieved in the most occasions: in only 4–6% of dyadic pairs was the correlation between two randomly paired profiles higher than the actually observed correlation between true pairs. As in previous studies, we found that age and sex of participants and length of acquaintance had no significant effect on the level of self-other agreement. However, intimate knowledge helped married and unmarried couples in both samples be more accurate in their personality judgments; family members, in turn, had knowledge that made them more accurate than two people who were just acquaintances or friends. We believe that these outcomes can be explained by the contention that the judgment of another’s personality is a relatively simple task, which is accomplishable for most people most of the time. In other words, because judging another person’s personality is an easy task, we are not able to determine “good targets,” “good judges,” or “good traits.” Perhaps it is only “good information” which determines the closeness of the target-judge relationship, and which has a small but reliable impact on the level of self-other agreement. This explains why it is so difficult to find individual differences in the ability to judge another person’s personality.     

Construct validity of global life-satisfaction judgments: A look into the black box of self–informant agreement

Life-satisfaction judgments are ubiquitously used as indicators of wellbeing. The construct validity of these judgments relies heavily on self–informant agreement. However, agreement is necessary, but not sufficient to claim that life-satisfaction judgments are valid. In addition, self–informant agreement should be based on the use of valid information about satisfaction with important life domains. An alternative possibility is that agreement is based on impressions about personality traits. We tested these two hypotheses in a model that predicted self-ratings and informant ratings of life satisfaction from shared perceptions of personality and satisfaction with life domains. In a round-robin study of families, we found that life-domains were the key predictor of self–informant agreement. However, the Depressiveness facet of Neuroticism had a small direct effect. In addition, it was indirectly related to self–informant agreement because it predicts lower satisfaction in important life-domains that were used to form life-satisfaction judgments.     

The evaluation of behavioural problems in the first three years of life: Comparing parents and early childhood educators

The study investigates the degree of agreement between mothers, fathers and educators in the evaluation of early childhood behavioural problems, through the CBCL 1½–5. Data analysis indicates a good level of agreement between mothers and fathers, along with a significant divergence between parents and educators.