计算机化多维测验中作答时间和作答精度数据的联合分析

随着心理与教育测量研究的发展和科技的进步,计算机化(大规模)测验逐渐受到人们的关注。为探究在计算机化多维测验中如何利用作答时间数据来辅助评估多维潜在能力,以及为我国义务教育阶段教育质量监测提供数据分析方法上的理论支持。本研究以2012年和2015年国际学生能力评估(PISA)计算机化数学测验数据为例,提出了一种可同时利用作答时间和作答精度数据的联合作答与时间的多维Rasch模型。根据新模型对PISA数据的分析结果,表明引入作答时间数据,不仅有助于提高模型参数的估计精度,还有助于数据分析者利用被试的作答时间信息来做进一步的决策和干预(e.g., 对异常作答行为或预备知识的诊断)。     

引入眼动注视点的联合-交叉负载多模态认知诊断建模

多模态数据为实现对认知结构的精准诊断及其他认知特征(如, 认知风格)的全面反馈提供了可能性。为实现对题目作答精度、作答时间(RT)和视觉注视点数(FC)的联合分析, 本文基于联合-交叉负载建模法提出3个多模态认知诊断模型。实证研究及模拟研究结果表明: (1)联合分析比分离分析更适用于多模态数据; (2)新模型可直接利用RT和FC中信息提高潜在能力或潜在属性的估计准确性; (3)新模型的参数估计返真性较好; (4)忽略交叉负载所导致的负面结果比冗余考虑交叉负载所导致的更严重。     

面向“为学习而测评”的纵向认知诊断模型

基于“为学习而测评”的理念,以促进学生学习为目的,客观量化学习现状并提供诊断反馈的测评模式日益受到重视。相比于横断认识诊断测评,纵向认知诊断测评更有利于实现促进学生发展的目标。为使国内学者系统性地了解纵向认知诊断模型,首先,依据建模逻辑将已有纵向认知诊断模型划分为基于潜在转换分析的和基于高阶潜在结构模型的两类,并逐一介绍和说明两类模型的理论基础和应用情景;然后,通过模拟研究为读者呈现如何使用纵向认知诊断模型进行数据分析及如何解读相应的诊断结果。最后,提炼出四个可进一步研究的议题。     

多维对数正态作答时间模型：对潜在加工速度多维性的探究

在心理与教育测量中, 潜在加工速度反映学生运用潜在能力解决问题的效率。为在多维测验中探究潜在加工速度的多维性并实现参数估计, 本研究提出多维对数正态作答时间模型。实证数据分析及模拟研究结果表明：(1)潜在加工速度具有与潜在能力相匹配的多维结构; (2)新模型可精确估计个体水平的多维潜在加工速度及与作答时间有关的题目参数; (3)冗余指定潜在加工速度具有多维性带来的负面影响低于忽略其多维性所带来的。     

学生科学教学知觉对科学素养成绩影响的跨文化比较

教学是通过影响学生思维而达成对学业成绩的影响,知觉是思维的重要组成。本研究以国际学生评价项目PISA2006的数据为基础,考察美国、芬兰、日本和中国香港学生科学教学知觉对科学素养成绩的影响。研究结果发现,在控制学生背景变量的情况下,学生的科学教学知觉对科学素养成绩具有显著的预测作用;4个国家(地区)学生的教学知觉对科学成就的影响具有相似性,关注应用知觉和动手实验知觉显著地正向预测科学成就,而科学探究知觉和课堂交流知觉显著地反向预测科学成就;学生教学知觉对科学兴趣的影响存在一定的文化差异,美国学生的科学探究知觉和课堂交流知觉显著地正向预测科学兴趣,日本学生课堂交流知觉显著地正向预测科学兴趣。     

教育认知诊断测验与认知模型一致性的评估

构建正确的认知模型是成功进行认知诊断的关键之一,如果认知诊断测验不能完整准确地代表这个认知模型,这个测验的效度就存在问题.属性及其层级可以表示一个认知模型.在认知模型正确基础上,给出了一个计量公式以衡量认知诊断测验能够多大程度上代表认知模型;对于不止包含一个知识状态的等价类及其形成原因进行了分析,对Cui等人的属性层级相合性指标(HCI)提出修改建议,以更好地探查数据与专家给出的认知模型的一致性.     

新技术引进决策的信息结构与辅助要求

本研究通过对浙皖两省38家企业与6家引进主管单位的现场调研与测评,就新技术引进决策的信息特征与结构关系进行了系统的分析,结果发现,新技术引进决策的信息包含八个主要因素:(1)职工状态信息;(2)决策者特征信息;(3)决策关系信息;(4)决策参与信息;(5)组织气氛信息;(6)目标与技术信息;(7)配备条件信息;(8)经济分析信息.它们在更高层次上形成组织心理、技术经济、决策者与关系三个信息模块.文章进一步分析了上述决策信息的结构关系及其认知加工特点,并提出新技术引进任务的决策心理辅助要求.     

应对量表(COPE)测评维度结构研究

该文旨在对应对量表(COPE)的测评维度结构进行进一步的鉴别分析和验证研究。研究一对736名大学生的应对量表中文修订本(C-COPE)测评数据进行探索性二阶因素分析;研究二根据已有研究关于应对量表测评维度组构模式的不同结论,以及研究一的结果,提出十个假设模型,采用验证性因素分析测试这些模型与另一大学生样本(N：465)测评数据的拟合度。研究结果支持C-COPE八因子斜交模型。该量表如何进一步修订也在文中予以讨论。     

儿童早期数学认知能力的结构及其特点

从北京市10所幼儿园中选取234名3、4岁的儿童为被试,采用个别测查的方法对儿童早期数学认知能力的结构及其特点进行了考察。经验证性因素分析发现：(1)儿童早期数学认知能力的结构模型是合理的,可接受的,具有较好的构想效度,具体讲,数、计算、测量、空间／几何和模式认知能力五个维度共同解释着儿童早期的数学认知能力;(2)不同年龄儿童早期数学认知能力的结构具有稳定性,但是结构模型并不完全一致,某些项目的解释率有所不同;(3)男、女儿童早期数学认知能力的结构模型具有一致性。     

使用验证性补偿多维IRT模型进行认知诊断评估

随着人们对测验反馈结果精细化的需求逐渐提高, 具有认知诊断功能的测量方法逐渐受到人们的关注。在认知诊断模型(CDMs)闪耀着光芒的同时, 另一类能够在连续量尺上提供精细反馈的多维IRT模型(MIRTMs)似乎受到些许冷落。为探究MIRTMs潜在的认知诊断功能, 本文以补偿模型为视角, 聚焦于分别属于MIRTMs的多维两参数logistic模型(M2PLM)和属于CDMs的线性logistic模型(LLM); 之后为使两者具有可比性, 可对补偿M2PLM引入验证性矩阵(Q矩阵)来界定题目与维度之间的关系, 进而得到验证性的补偿M2PLM (CC-M2PLM), 并通过把潜在特质按切点划分为跨界属性, 以期使CC-M2PLM展现出其本应具有的认知诊断功能; 预研究表明logistic量尺上的0点可作为相对合理的切点; 然后, 通过模拟研究对比探究CC-M2PLM和LLM的认知诊断功能, 结果表明CC-M2PLM可用于分析诊断测验数据, 且认知诊断功能与直接使用LLM的效果相当; 最后, 以两则实证数据为例来说明CC-M2PLM在实际诊断测验分析中的可行性。     

Reading Achievement and Science Proficiency: International Comparisons From the Programme on International Student Assessment

Students need to develop scientific literacy in order to participate fully as citizens, community members, and in the globalized economy. But what is the relationship between scientific literacy and reading literacy? Three international data sets from the Programme on International Student Assessment (PISA) were used to calculate correlations between scientific literacy and reading literacy for 15-year-old students. Mean correlations at the individual student level across countries were .840 for the PISA 2000 data set, .805 for the PISA 2003 data set, and .819 for the PISA 2006 data set. In all three data sets, this correlation varied among countries, and the reading-science relationship was weakest in countries with low country mean reading scores. Three possible interpretations are discussed, favoring the interpretation that knowledge and skills that drive higher reading comprehension also drive higher science achievement.     

Incongruence Between Native and Test Administration Languages: Towards Equal Opportunity in International Literacy Assessment

Previous research has primarily addressed the effects of language on the Program for International Student Assessment (PISA) mathematics and science assessments. More recent research has focused on the effects of language on PISA reading comprehension and literacy assessments on student populations in specific Organization for Economic Cooperation and Development (OECD) and non-OECD countries. Recognizing calls to highlight the impact of language on student PISA reading performance across countries, the purpose of this study was to examine the effect of home languages versus test languages on PISA reading literacy across OECD and non-OECD economies, while considering other factors. The results of Ordinary Least Squares regression showed that about half of the economies demonstrated a positive and significant effect of students' language status on their reading performance. This finding is consistent with observations in the parallel analysis of PISA 2009 data, suggesting that students' performance on reading literacy assessment was higher when they were tested in their home language. Our findings highlight the importance of the role of context, the need for new approaches to test translation, and the potential similarities in language status for youth from OECD and non-OECD countries that have implications for interpreting their PISA reading literacy assessments.     

PISA and the assessment triangle

PISA is a well-known and high profile Program for International Student Assessment with 4 editions since 2000. This study aims to examine the validity of PISA proficiency estimates, working with the framework provided by the assessment triangle. We pay explicit attention to how PISA proceeds as far as the three elements of the assessment triangle are concerned: cognition, observation, and interpretation. Results reveal not only the psychometrically sound proficiency estimates of PISA and the high standards reached, but also that there is room for improvement; for instance, cognitive diagnostic models could contribute both to test design and data analysis.     

The Reading Literacy test for Secondary Education (CompLEC)

A new test to evaluate reading literacy, the Test of Reading Literacy for Secondary Education (CompLEC) is presented. CompLEC is based on the PISA assessment framework and new definitions of reading literacy. The test, easy to apply and score, assesses the level of reading literacy of children between 11 and 14 years of age in several reading situations (i.e., public, educational, personal and occupational) and with different types of texts (i.e., continuous and non-continuous). The scale has been standardized with a sample of 1,854 students from five different Spanish regions. Empirical results show that CompLEC is a homogeneous, reliable and valid instrument.     

Are Learning Strategies Linked to Academic Performance Among Adolescents in Two States in India? A Tobit Regression Analysis

The results of the fourth cycle of the Program for International Student Assessment (PISA) revealed that an unacceptably large number of adolescent students in two states in India—Himachal Pradesh and Tamil Nadu—have failed to acquire basic skills in reading, mathematics, and science (Walker, 2011). Drawing on data from the PISA 2009 database and employing multivariate left-censored tobit regression as a data analytic strategy, the present study, therefore, examined whether or not the learning strategies—memorization, elaboration, and control strategies—of adolescent students in Himachal Pradesh (N = 1,616; Mean age = 15.81 years) and Tamil Nadu (N = 3,210; Mean age = 15.64 years) were linked to their performance on the PISA 2009 reading, mathematics, and science assessments. Tobit regression analyses, after accounting for student demographic characteristics, revealed that the self-reported use of control strategies was significantly positively associated with reading, mathematical, and scientific literacy of adolescents in Himachal Pradesh and Tamil Nadu. While the self-reported use of elaboration strategies was not significantly associated with reading literacy among adolescents in Himachal Pradesh and Tamil Nadu, it was significantly positively associated with mathematical literacy among adolescents in Himachal Pradesh and Tamil Nadu. Moreover, the self-reported use of elaboration strategies was significantly and positively linked to scientific literacy among adolescents in Himachal Pradesh alone. The self-reported use of memorization strategies was significantly negatively associated with reading, mathematical, and scientific literacy in Tamil Nadu, while it was significantly negatively associated with mathematical and scientific literacy alone in Himachal Pradesh. Implications of these findings are discussed.     

学校教育资源对学生数学素养预测效应的跨文化比较

采用国际学生评价项目PISA2006的数据,使用多水平线性模型方法,比较中国香港、日本、芬兰和美国四个国家（地区）的学校教育资源与学生数学素养成绩的关系。结果发现,在控制学生背景变量的情况下,学校教育资源对学生数学素养成绩的影响在四个国家（地区）之间存在一定的文化差异：学校大小、生师比对中国香港学生数学素养成绩有显著的正向预测作用;学校大小、班级大小、学校类型、有硕士研究生学历的教师比例对日本学生数学素养成绩有显著的正向预测作用,用于教学的计算机比例对日本学生数学素养成绩有显著的反向预测作用;学校教育资源对芬兰学生数学素养成绩没有显著的预测作用;学校类型对美国学生数学素养成绩具有显著的反向预测作用。     

Development and Validation of the Scientific Reasoning Scale

Scientific findings and innovations play an important role in a range of decisions faced by nonscientists, yet little is known about the skills that nonscientists need in order to read and evaluate scientific evidence. Drawing on research in public understanding of science, cognitive developmental psychology, and behavioral decision research, we develop an individual difference measure of scientific reasoning skills, defined as the skills needed to evaluate scientific findings in terms of the factors that determine their quality. We present the results of three studies assessing its psychometric validity. Our results indicate that the Scientific Reasoning Scale (SRS) is internally consistent and distinct from extant measures of scientific literacy. Participants with higher SRS scores are more likely to have beliefs consistent with the scientific consensus on potentially contentious issues, above and beyond education, political and religious beliefs, and scores on two widely used measures of scientific literacy. Participants with higher SRS scores also had better performance on a task requiring them to analyze scientific information. Our results suggest that the SRS provides a theoretically informed contribution to decoding lay responses to scientific results and controversies. Copyright © 2015 John Wiley & Sons, Ltd.     

Analyses of Model Fit and Robustness. A New Look at the PISA Scaling Model Underlying Ranking of Countries According to Reading Literacy

This paper addresses methodological issues that concern the scaling model used in the international comparison of student attainment in the Programme for International Student Attainment (PISA), specifically with reference to whether PISA’s ranking of countries is confounded by model misfit and differential item functioning (DIF). To determine this, we reanalyzed the publicly accessible data on reading skills from the 2006 PISA survey. We also examined whether the ranking of countries is robust in relation to the errors of the scaling model. This was done by studying invariance across subscales, and by comparing ranks based on the scaling model and ranks based on models where some of the flaws of PISA’s scaling model are taken into account. Our analyses provide strong evidence of misfit of the PISA scaling model and very strong evidence of DIF. These findings do not support the claims that the country rankings reported by PISA are robust.