表现性评价中评分者信度估计方法的比较研究--从相关法、百分比法到概化理论

随着素质教育的推进．表现性评价受到越来越多的重视。影响表现性评价结果信度的一个重要因素是评分者之间的不一致。文章使用模拟数据,在对比评分者一致性的相关法、一致性百分比法和概化系数等各种估计方法的基础上,提出概化理论在表现性评价中评分者信度问题上的应用是理论和实践发展的有益方向。     

职业心理综合测验的编制

1 .前言　　职业心理综合测验是用于职业指导和职业招聘的综合性心理测验 ,包括职业兴趣测验 ,职业能力倾向测验和职业人格测验。根据人 -职匹配理论 ,个体差异是普遍存在的 ,每一个个体都有自己的个性特征 ,而每一种职业由于其工作性质、环境、条件、方式的不同 ,对工作者的能力、知识、技能、性格、气质等心理素质有不同的要求。进行职业决策 ,如选拔、安置、职业指导时 ,就要根据一个人的个性特征来选择与之相对应的职业种类。实践表明 ,一个人的能力特点和个性特点与他所从事的工作的匹配程度 ,直接影响着工作的成效。虽然能力可在实践…     

情境判断测验的效度研究述评

情境判断测验是一种为作答者呈现工作相关的典型情境以及该情境下可能的行为反应, 要求根据指导语提示进行选择或评价的测验形式。随着其理论和实践的发展, 研究者越来越关注情境判断测验的效度研究, 包括对其构想效度、效标关联效度和递增效度的探讨, 以及指导语类型、情境保真度以及计分方式等因素对其效度的影响。基于这些研究进展, 未来情境判断测验实践领域可能的方向是：(1)开发针对特定构想的情境判断测验; (2)结合具体构想选用相应的指导语; (3)应用作假和培训对效度影响的研究结果指导实践。     

人事测评中的概化理论应用

概化理论是现代心理与教育测量理论之一,可应用在各种人事测评中,如表现性评价、多源评估、心理测验、结构化面试、水平测试、工作分析、评价中心等.与经典测量理论相比,概化理论应用于人事测评,表现出较强的优势,能够同时考察多种因素、确定多个维度权重等,其应用对象主要包括两大类,即企业和机构.概化理论应用于人事测评,存在应用领域、样本数据、测评效度和微观分析等问题.     

《考夫曼成套儿童评价测验》简介

本文扼要介绍了《考夫曼成套儿童评价测验,简称 K-ABC,它是目前国外比较新颖的一套智力测验。该测验的突出特点是以认知心理学中有关信息加工模式的理论为基础而编制的,其中绝大部分内容为非言语测验。国外的实践表明:K-ABC 在临床、教育评价及科学研究等方面已显示出了一定的应用价值。     

空间能力测验及其研究应用

空间能力所表现出的个体差异一直是空间能力研究中的热点问题,在研究中所使用的测验也多种多样。二维空间能力测验主要包括标准心理旋转测验及其多种变式;三维空间能力测验则是运用虚拟现实技术针对动态空间定位与位置学习等空间能力开发出的新型测验。这些测验主要用于探讨空间能力个体差异的影响因素。本文在介绍测验的同时总结了空间能力个体差异研究的结果并提出展望。     

儿童认知发展动态测验

动态测验是一系列有着共同基本假设的多种能力测验的统称,是以维果茨基的最近发展区理论和实验为基础,针对传统静态测验低估弱势儿童能力、缺乏对教育实践的有效指导而提出的测量范式。它采用与智力测验相似的项目,以学习率、迁移能力和认知改变为指标,通过在测验中提示和干预的方法考查个体潜在的认知发展水平。该着重介绍了当代动态测验影响较大的几种方法或技术：Feuerstein的中介学习、Budofr的训练测验、Campione和Brown的逐步提示法、Guthke的学习测验、Carlson和Wiedl的极限探测法,并试图对动态测验进行整体评价。     

在高中历史课堂教学中表现性评价的运用浅议

在新课程的教育实践中,我也深切体会到传统评价体系的不足。因此在高中课堂教学实践中我进行了有益的尝试,积极探索新的学科学习评价方法。新的历史课程标准中特别强调能力的培养“中学历史课程培养的能力包括：搜集史料、提取信息、解决问题、交流成果。”在日常教学中,我经常结合课程教学开展一些研究性学习活动。活动之后学生能力的提高究竟有多大,我应该如何改进今后的教学工作,这就需要一种有效的评价方法。我尝试用“表现性评价”来对学生的学,教师的教做出一个全面的评价。经过几年的实践,我对“表现性评价”在高中历史教学中的应用有了一些心得,下面就这种新的评价手段和我的实践经验谈一些看法。     

项目反应理论的分数分布的预测作用

1引言在心理和教育测量的实践过程中,测试者常常会从已实施过的测验项目里根据需要挑选一组项目重新组成新的测验。如果测试者事先能够通过一定的方法预测到关于这组测验分数的大致信息,这对编制不同要求的测验、检查教学效果以及评价学生能力有很大的帮助。在项目反应理论（IRT）中,预测分数的一般做法是通过测验特性函数来计算的（HambletomandSwaminathan,1985）。但是这样计算的结果只能知道在能力参数θ的某个水平上的分数．从本质上来说这是属于条件概率的点估计。为了了解在能力θ的整个范围里的测验分数的变化趋向,本文利用…     

影响Sternberg双重任务测验评定飞行能力的因素分析

１前言Ｓｔｅｒｎｂｅｒｇ双重任务作为一种典型的心理运动测验,以测量人的手一眼协调能力和双任务下的注意分配能力为目的．受到心理选拔工作的高度重视,在许多著名的心理测验系统中都可以找到该技术的踪影。研究中我们发现,Ｓｔｅｒｎｂｅｒｇ双重任务主要用于评价实际飞行的快速反应能力、分析判断能力、实际飞行操纵能力和飞行能力综合评价。但由于飞行员年龄、文化程度和驾驶飞行机种的差异,其评价效果是有差异的。本研究目的在于,探讨年龄、文化程度和飞行机种对Ｓｔｅｒｎｂｅｒｇ双重任务测验成绩的影响,探讨三项因素对双重任…     

Assessing Religion and Spirituality in Counseling: Some Reflections and Recommendations

In this article, the importance of conducting a religious and spiritual assessment in counseling is considered. Some essential dimensions of religion and spirituality to assess are described. The authors recommend assessment questions that can be asked during clinical interviews or included on written intake questionnaires. They also briefly describe a few standardized religious and spiritual assessment instruments. Finally, they offer suggestions for conducting spiritual assessments in school settings.     

Neuropsychological Assessment of Mental Capacity

The assessment of mental capacity to assist legal determinations of competency is potentially a growth area for neuropsychology, although to date neuropsychologists have published relatively little in this area. In this paper a systematic review of methods used to assess capacity is presented, including coverage of specialized tests and interviews used for this purpose. A neuropsychological model for conducting capacity assessments is proposed. This model involves comprehensive assessment of a wide range of cognitive abilities as well as assessment of specific skills and knowledge related to the type of capacity being assessed. The purpose of proposing this model is to stimulate further discussion and debate about the contribution neuropsychologists might make in this area.     

Using HLM to Explore the Effects of Perceptions of Learning Environments and Assessments on Students’ Test Performance

The study examines the role that perceptions or impressions of learning environments and assessments play in students’ performance on a large-scale standardized test. Hierarchical linear modeling (HLM) was used to test aspects of the Learning Errors and Formative Feedback model to determine how much variation in students’ performance was explained by students’ and school principals’ perceptions of learning environments and assessments. Results from sequential HLM testing indicated that students’ but not principals’ perceptions explained a significant, although modest, amount of the total variation in students’ test performance. These results suggest that when students perceive learning environments to be safe and valuable, and positive assessment activities to be taking place, they tend to perform better on standardized tests than when they perceive learning environments and assessment activities otherwise. These findings provide a rationale for investigating the variables that can help improve students’ perceptions in order to enhance their test performance.     

The Influence of Rater Effects in Training Sets on the Psychometric Quality of Automated Scoring for Writing Assessments

Automated essay scoring engines (AESEs) are becoming increasingly popular as an efficient method for performance assessments in writing, including many language assessments that are used worldwide. Before they can be used operationally, AESEs must be “trained” using machine-learning techniques that incorporate human ratings. However, the quality of the human ratings used to train the AESEs is rarely examined. As a result, the impact of various rater effects (e.g., severity and centrality) on the quality of AESE-assigned scores is not known. In this study, we use data from a large-scale rater-mediated writing assessment to examine the impact of rater effects on the quality of AESE-assigned scores. Overall, the results suggest that if rater effects are present in the ratings used to train an AESE, the AESE scores may replicate these effects. Implications are discussed in terms of research and practice related to automated scoring.     

Using Advice and Assessing Its Quality

People received advice from four sources and used it to produce a judgment. They also assessed the quality of advice by estimating the probability that it would be correct. They were better at assessing than at using advice: combinations of advice based on their assessments were superior to their judgments. Order of assessing and using advice, superficial differences between advisors, and using other methods of advice assessment had no significant effects on this superiority of advice assessment over advice use. However, use but not assessment was improved when some advisors exhibited biases opposite to those that people typically show. It appears that using advice imposes a heavier processing load than assessing its quality and that this load can be lightened by including advisors who exhibit unusual behavior. Their salience may help people working under a heavy processing load make appropriate pairings between advisor weights and advice.     

An Upgrading Procedure for Adaptive Assessment of Knowledge

In knowledge space theory, existing adaptive assessment procedures can only be applied when suitable estimates of their parameters are available. In this paper, an iterative procedure is proposed, which upgrades its parameters with the increasing number of assessments. The first assessments are run using parameter values that favor accuracy over efficiency. Subsequent assessments are run using new parameter values estimated on the incomplete response patterns from previous assessments. Parameter estimation is carried out through a new probabilistic model for missing-at-random data. Two simulation studies show that, with the increasing number of assessments, the performance of the proposed procedure approaches that of gold standards.     

INTERACTIVE VIDEO ASSESSMENT OF CONFLICT RESOLUTION SKILLS

In this research we developed and validated an interactive video assessment of conflict resolution skills. A model of conflict management was used to develop the conflict scenarios and part of the scoring key. Computer assessments of conflict resolution skills and two cognitive abilities were administered to 347 supervisors and job performance ratings were collected from their managers. The conflict skills assessment was found to be significantly related to supervisory ratings of on-the-job performance in managing conflict but to be unrelated to the measures of cognitive ability. In addition, the conflict skills assessment had no adverse impact for women. The implications of these results and directions for future research are discussed.     

Adaptive spatial working memory assessments for aging pet dogs

Assessments for spatial working memory (SWM) in pet dogs that can detect age-related cognitive deficits in a single session may aid in diagnosing canine dementia and may facilitate translational research on Alzheimer’s disease in humans. Adaptive testing procedures are widely used in single-session assessments for humans with diverse cognitive abilities. In this study, we designed and deployed two up-down staircase assessments for SWM in which 26 pet dogs were required to recall the location of a treat hidden behind one of two identical boxes following delays of variable length. In the first experiment, performance tended to decline with age but few dogs completed the test (n?=?10). However, all of the dogs that participated in the second experiment (n?=?24) completed the assessment and provided reliable evidence of learning and retaining the task. Delay length and age significantly predicted performance supporting the validity of this assessment. The relationships between age and performance were described by inverted U-shaped functions as both old and young dogs displayed deficits in weighted cumulative-scores and trial-by-trial performance. Thus, SWM in pet dogs may develop until midlife and decline thereafter. Exploratory analyses of non-mnemonic fixation strategies, sustained engagement, inhibitory control, and potential improvements for future SWM assessments which adopt this paradigm are also discussed.

     

Cross-national applicability of a game-based cognitive assessment

New technology has had a discernable impact on how organizations recruit and select potential employees. Game-based assessment has emerged as a potential technology that can be used to enhance the assessment of individual differences and applicants' views of the selection process. However, studies investigating the psychometric properties and predictive validity of game-based assessments are still lacking. This study investigated the structural equivalence of a game-based assessment of cognitive ability across 228 Australians and 239 South Africans. A smaller sample of 115 South Africans also received work performance ratings to investigate the predictive validity of the cognitive assessment. Results of factor analysis supported a strong general factor of cognitive ability across the entire sample but only partial metric and scalar invariance across the two nations. The general factor of the game-based assessment further revealed promising results in terms of its predictive validity for five broad dimensions of individual work performance.     

At the Threshold of School: Parental Assessments of the Competencies of Their Preschool-Aged Children

The study set out to examine the organization of parental assessments of their children's competencies in terms of the parents' education and gender and the chilďs gender. Parents with university education ( N = 231) and vocational education ( N = 343) were asked to appraise their preschool-aged chilďs competencies in domains pertaining to school subjects and abilities. It was found that parents with university education emphasized cognitive-verbal competencies when assessing their children, and parents with vocational education emphasized practical competencies. Girls' competencies were generally assessed more favorably than were boy's. Yet boys were considered to have higher mathematical skills than were girls, and this assessment was further specified by the parents' educational position. In sum, the organization of parental assessments seemed to have already taken shape by the time the child enters school.