项目功能差异在跨文化人格问卷分析中的应用

利用IRT的等级模型调查了中日两组被试关于SHIBA简易人格量表中“环境敏感性”的项目功能差异(DIF)的现状。研究发现：（1）量表中DIF的项目比例大(3/4);（2）DIF与项目内容、阈值有关而与区分度大小关系不大;（3）DIF项目间的日方特征曲线较之中方有较强的整合性。该研究利用DIF研究结果对跨文化的人格比较作了新尝试。最后提出了关于深化DIF研究的新课题     

经济法试题DIF的参数法检测研究

该研究基于项目反应理论的Samejima等级反应模型(GRM),在MULTILOG软件支持下,应用参数检测方法,对某年度全国性资格考试的某科目试卷中经济法部分的21个项目做了DIF检测分析。结果如下:存在性别DIF的项目一个,存在民族DIF的项目四个,存在工作性质DIF的项目一个。其中项目68在民族层面上表现为一致性DIF,项目64既存在民族DIF又存在工作性质DIF。通过对项目统计量、反应曲线的分析和专家的讨论,文章最后还分析了产生这些DIF的几个可能的原因。     

分类数据测量等价性检验方法及其比较：项目阈值(难度)参数的组间差异性检验

测量工具满足等价性是进行多组比较的前提, 测量等价性的检验方法主要有基于CFA的多组比较法和基于IRT的DIF检验两类方法。文章比较了单维测验情境下基于CCFA的DIFFTEST检验方法和基于IRT模型的IRT-LR检验方法, 以及多维测验情境下DIFFTEST和基于MIRT的卡方检验方法的差异。通过模拟研究的方法, 比较了几种方法的检验力和第一类错误, 并考虑了样本总量、样本量的组间均衡性、测验长度、阈值差异大小以及维度间相关程度的影响。研究结果表明：(1)在单维测验下, IRT-LR是比DIFFTEST更为严格的检验方法; 多维测验下, 在测验较长、测验维度之间相关较高时, MIRT-MG比DIFFTEST更容易检验出项目阈值的差异, 而在测验长度较短、维度之间相关较小时, DIFFTEST的检验力反而略高于MIRT-MG方法。(2)随着阈值差值增加, DIFFTEST、IRT-LR和MIRT-MG三种方法的检验力均在增加, 当阈值差异达到中等或较大时, 三种方法都可以有效检验出测验阈值的不等价性。(3)随着样本总量增加, DIFFTEST、IRT-LR和MIRT-MG方法的检验力均在增加; 在总样本量不变, 两组样本均衡情况下三种方法的检验力均高于不均衡的情况。(4)违背等价性题目个数不变时, 测验越长DIFFTEST的检验力会下降, 而IRT-LR和MIRT-MG检验力则上升。(5) DIFFTEST方法的一类错误率平均值接近名义值0.05; 而IRT-LR和MIRT-MG方法的一类错误率平均值远低于0.05。     

大学生社交焦虑的计算机化自适应测验开发:CAT-SA

本文在IRT框架下,结合国内外知名的社交焦虑量表,构建社交焦虑题库及其计算机化自适应测验(CAT-SA)。IRT分析包括:单维性检验、项目模型-资料拟合检验、局部独立性检验、DIF检验,选择符合IRT要求的项目构建社交焦虑题库及其CAT。最后分析了CAT-SA诊断效果及信、效度验证。结果显示CAT-SA具有较好的信、效度;能大大减少测试题量,达到减轻测试负担的目的。总之,本文开发的CAT-SA为实现对社交焦虑的高效、快速和准确测量提供新的测量技术和工具。     

运用均数与协方差结构模型侦查项目功能差异

阐释了运用多组均数与协方差结构(MACS)模型侦查多级反应项目的一致性与非一致性项目功能差异(DIF)的原理与程序, 以道德自我概念量表DIF的侦查进行示例, 并对该方法进行了评价。与项目反应理论比照, MACS采用系统的、迭代的方式利用修正指数来侦查DIF, 并提供多个拟合指数协同评价模型拟合;与标准验证性因素分析相较, MACS不仅能侦查非一致性DIF, 而且能侦查一致性DIF。运用MACS侦查DIF是一种值得推荐的方法。     

变通的题组项目功能差异检验方法在篇章阅读测验中的应用

篇章形式的阅读测验在语文学科考试与语言能力测试中占有越来越重要的地位。篇章阅读测验是一种典型的题组测验, 因此需要采用能够处理题组效应的统计方法进行分析。在进行项目功能差异(DIF)检验时, 也需要采用与之匹配的DIF检验方法。目前能够处理题组效应的DIF检验方法主要包括变通的题组DIF检验方法和基于题组反应模型的DIF检验方法, 基于题组反应模型的DIF检验方法由于实现过程繁琐, 目前只停留在理论探讨阶段。本研究将变通的题组DIF检验方法及其效应值指标引入篇章阅读测验的DIF检验中, 能够解决篇章阅读测验中DIF检验与测量的问题, 效应值指标能够为如何处理有DIF效应的题组项目提供重要依据。本研究首先选用非题组DIF检验方法与变通的题组DIF检验方法对一份试卷进行DIF检验, 两种方法的比较结果体现了进行题组DIF检验的必要性与优越性, 然后选用变通的题组DIF检验方法中有代表性的四种方法对某阅读成就测验进行题组DIF检验。研究结果表明, 在篇章阅读测验中, 能够处理题组效应的DIF检验方法较传统的DIF检验方法具有较大的优越性。     

认知诊断测验中的项目功能差异检测方法比较

检测项目功能差异(DIF)是认知诊断测验中很重要的问题。首先将逻辑斯蒂克回归法(LR)引入认知诊断测验DIF检测, 然后将LR法与MH法和Wald检验法的DIF检验效果进行比较。在比较中同时考察了匹配变量、DIF种类、DIF大小和受测者人数的影响。结果表明：(1) LR法在认知诊断测验DIF检测中, 检验力较高, 一类错误率较低。(2) LR法在检测认知诊断测验的DIF时, 不受认知诊断方法的影响。(3) LR法可以有效区分一致性DIF和非一致性DIF, 并有较高检验力和较低一类错误率。(4)采用知识状态作为匹配变量, 能够得到较理想的检验力和一类错误率。(5) DIF越大, 受测者人数越多, 统计检验力越高, 但一类错误率不受影响。     

当前考试理论研究的进展

近二十年以来,考试理论(Testing Theories)的研究取得了长足进展,这种进展表现在两个方面一方面,在上个世纪六十年代由Lord提出的项目反应理论(Item Response Theory,IRT)得到了很大的扩展,出现了多维度项目反应理论(multi-dimensional IRT)、非参数项目反应理论(Nonparametric IRT)以及认知诊断理论(Cognitively Diagnostic Theory)等;另一方面,项目反应理论在考试实践中得到了广泛的应用,使考试实践产生了革命性的变化,出现了计算机自适应考试(Computerized Adaptive Testing,CAT).     

CTT与IRT方法对人格测验结果处理的比较研究

为了说明使用经典测量理论(CTT)方法和项目反应理论(IRT)方法计算出的人格测验结果的差异,本研究使用IRT和CTT这两种方法分别计算出模拟人格测验和实际人格测验的测验结果,并对此进行比较。研究表明,两种不同的方法得到的测验结果之间平均有0．11个标准差以上的差异。进一步研究发现,在对测验结果进行分析时,IRT方法比CTT方法更为有效。     

项目反应理论中模型-资料拟合检验常用统计量

项目反应理论(IRT)模型依据项目与被试的特征预测被试的作答表现, 是常用的心理测量模型。但IRT的有效运用依赖于所选用IRT模型与实际数据资料相符合的程度(即模型?资料拟合度, goodness of fit)。只有当所采用IRT分析模型与实际数据资料拟合较好时, IRT的优点和功能才能真正发挥出来(Orlando & Thissen, 2000)。而当所采用IRT模型与资料不拟合或选择了错误的模型, 则会导致如参数估计、测验等值及项目功能差异分析等具有较大误差(Kang, Cohen & Sung, 2009), 给实际工作带来不良影响。因此, 在使用IRT分析时, 应首先充分考察及检验所选用模型与实际数据是否相匹配/相拟合(McKinley & Mills, 1985)。IRT领域中常用模型?资料拟合检验统计量可从项目拟合、测验拟合两个角度进行阐述并比较, 这是心理、教育测量领域的重要主题, 也是测验分析过程中较易忽视的环节, 目前还未见此类公开发表的文章。未来的研究可以在各统计量的实证比较研究以及在认知诊断领域的拓展方面有所发展。     

APPROPRIATE METHODS FOR COMMUNICATION POLICY SCIENCE: SOME PRELIMINARY CONSIDERATIONS

This article inquires into the appropriateness of scientific methods now used by a social science of communication for use by an emerging communication policy science. Toward this end, changes in the communication field leading to a policy science are considered, and some of the problems and possibilities in methods of inquiry are outlined. Questions of communication sciences and methods are then set in a multicultural historical perspective. The article concludes with suggestions on how uniquely appropriate methods might be developed for communication policy science.     

The Informative Analysis of Individual Trend Curves

Additive and non-additive models for an individual trend curve are examined, and five methods for fitting these to a set of individuals are described. It is suggested that classical fitting methods are more informative than latent curve methods, and commonly preferable. A limited study of the effect of time-structure is reported, and results on the relationship between a non-additive model and the approximating additive model are given.     

Meta-analytic interval estimation for bivariate correlations

The currently available meta-analytic methods for correlations have restrictive assumptions. The fixed-effects methods assume equal population correlations and exhibit poor performance under correlation heterogeneity. The random-effects methods do not assume correlation homogeneity but are based on an equally unrealistic assumption that the selected studies are a random sample from a well-defined superpopulation of study populations. The random-effects methods can accommodate correlation heterogeneity, but these methods do not perform properly in typical applications where the studies are nonrandomly selected. A new fixed-effects meta-analytic confidence interval for bivariate correlations is proposed that is easy to compute and performs well under correlation heterogeneity and nonrandomly selected studies.     

Analyzing Goal-Striving Processes and a Test of the Generalizability of Perceptual Control Theory

Theories that articulate dynamic processes are relatively rare, but methods for testing the theories are even rarer. This study illustrates two methods for examining goal-striving processes and a tool for collecting dynamic data. The first method tests a hypothesis regarding what variable the participants are attempting to maintain. The second method involves creating multilevel models used to describe the dynamic data generated by study participants, which can be used to test between- and within-subject manipulations or differences. The tool is a research simulation of a manager's role in scheduling subordinates in a hospital wing. Together these methods and the tool are used to test the generalizability of perceptual control theory in explaining striving for cognitive goals. The results confirm the viability of a control theory accounting of goal striving and highlight the potential of the methods and the research tool in future research.     

Self-informant Agreement for Personality and Evaluative Person Descriptors: Comparing Methods for Creating Informant Measures

Little attention typically is paid to the way self-report measures are translated for use in self-informant agreement studies. We studied two possible methods for creating informant measures: (a) the traditional method in which self-report items were translated from the first- to the third-person and (b) an alternative meta-perceptual method in which informants were directed to rate their perception of the targets' self-perception. We hypothesized that the latter method would yield stronger self-informant agreement for evaluative personality dimensions measured by indirect item markers. We studied these methods in a sample of 303 undergraduate friendship dyads. Results revealed mean-level differences between methods, similar self-informant agreement across methods, stronger agreement for Big Five dimensions than for evaluative dimensions, and incremental validity for meta-perceptual informant rating methods. Limited power reduced the interpretability of several sparse acquaintanceship effects. We conclude that traditional informant methods are appropriate for most personality traits, but meta-perceptual methods may be more appropriate when personality questionnaire items reflect indirect indicators of the trait being measured, which is particularly likely for evaluative traits.     

A comparison of interview and self-report methods for the assessment of borderline personality disorder criteria

Interview methods are widely regarded as the standard for the diagnosis of borderline personality disorder (BPD), whereas self-report methods are considered a time-efficient alternative. However, the relative validity of these methods has not been sufficiently tested. The current study used data from the Collaborative Longitudinal Personality disorder Study to compare diagnostic base rates and the relative validity of interview and self-report methods for assessing functional outcome in BPD. Although self-report yielded higher base rates of criteria endorsement, results did not support the common assumption that diagnostic interviews are more valid than self-reports, but instead indicated the combined use of these methods optimally identifies BPD criteria.     

The analysis of classroom talk: Methods and methodologies

This article describes methods for analysing classroom talk, comparing their strengths and weaknesses. Both quantitative and qualitative methods are described and assessed for their strengths and weaknesses, with a discussion of the mixed use of such methods. It is acknowledged that particular methods are often embedded in particular methodologies, which are based on specific theories of social action, research paradigms, and disciplines; and so a comparison is made of two contemporary methodologies, linguistic ethnography, and sociocultural research. The article concludes with some comments on the current state of development of this field of research and on ways that it might usefully progress.     

An overview of applied robust methods

Data in social sciences are typically non-normally distributed and characterized by heavy tails. However, most widely used methods in social sciences are still based on the analyses of sample means and sample covariances. While these conventional methods continue to be used to address new substantive issues, conclusions reached can be inaccurate or misleading. Although there is no ‘best method’ in practice, robust methods that consider the distribution of the data can perform substantially better than the conventional methods. This article gives an overview of robust procedures, emphasizing a few that have been repeatedly shown to work well for models that are widely used in social and behavioural sciences. Real data examples show how to use the robust methods for latent variable models and for moderated mediation analysis when a regression model contains categorical covariates and product terms. Results and logical analyses indicate that robust methods yield more efficient parameter estimates, more reliable model evaluation, more reliable model/data diagnostics, and more trustworthy conclusions when conducting replication studies. R and SAS programs are provided for routine applications of the recommended robust method.     

Validation and error in multiplicative utility functions

An approach to the concept of error in utility assessment is proposed. Four kinds of errors are considered and each kind is related to four separate elicitation methods - all in the context of a general multiplicative multiattribute utility model. The methods are a Keeney-Raiffa (1976) procedure; SMART, for Simple Multi-Attribute Rating Technique (Edwards 1977); SJT, for a Social Judgment Theory based regression model (Hammond et al. 1975); and HOPE, for Holistic Orthogonal Parameter Estimation (Barron and Person 1979).The individual judgments elicited are either holistic - in which the entity to be evaluated is considered as a whole - or decomposed - in which attention is directed to one or two aspects at a time.If a general multiplicative model can be assumed to be an appropriate representation of the decision maker's basic preference structure, error can occur in the direct estimation of the scaling constants and univariate utility functions for decomposition methods (Keeney-Raiffa and SMART), or in the holistic assessments for holistic methods (SJT and HOPE). Individual estimates may be merely subject to random noise or may be substantially incorrect. The utility model may be incorrectly specified; finally, all four methods may be subject to systematic error. The four assessment methods are considered in conjunction with errors of each kind.     

Methods for Practising Ethics in Research and Innovation: A Literature Review,Critical Analysis and Recommendations

This paper provides a systematic literature review, analysis and discussion of methods that are proposed to practise ethics in research and innovation (R&I). Ethical considerations concerning the impacts of R&I are increasingly important, due to the quickening pace of technological innovation and the ubiquitous use of the outcomes of R&I processes in society. For this reason, several methods for practising ethics have been developed in different fields of R&I. The paper first of all presents a systematic search of academic sources that present and discuss such methods. Secondly, it provides a categorisation of these methods according to three main kinds: (1) ex ante methods, dealing with emerging technologies, (2) intra methods, dealing with technology design, and (3) ex post methods, dealing with ethical analysis of existing technologies. Thirdly, it discusses the methods by considering problems in the way they deal with the uncertainty of technological change, ethical technology design, the identification, analysis and resolving of ethical impacts of technologies and stakeholder participation. The results and discussion of our literature review are valuable for gaining an overview of the state of the art and serve as an outline of a future research agenda of methods for practising ethics in R&I.