Distance stability analysis in multidimensional scaling using the jackknife method

Stability or sensitivity analysis is an important topic in data analysis that has received little attention in the application of multidimensional scaling (MDS), for which the only available approaches are given in terms of a coordinate‐based analytical jackknife methodology. Although in MDS the prime interest is in assessing the stability of the points in the configuration, this methodology may be influenced by imprecisions resulting from the inherently necessary Procrustes method. This paper proposes an analytical distance‐based jackknife procedure to study stability and cross‐validation in MDS in terms of the jackknife distances, which is not influenced by the Procrustes method. For each object, the corresponding jackknife estimated points are considered as naturally clustered points, and stability and cross‐validation are analysed in terms of the MDS distances arising from the jackknife procedure, on the basis of a weighted cluster‐MDS algorithm. A jackknife‐relevant configuration is also proposed for cross‐validation in terms of coordinates, in a cluster‐MDS framework.     

Developing new online calibration methods for multidimensional computerized adaptive testing

Multidimensional computerized adaptive testing (MCAT) has received increasing attention over the past few years in educational measurement. Like all other formats of CAT, item replenishment is an essential part of MCAT for its item bank maintenance and management, which governs retiring overexposed or obsolete items over time and replacing them with new ones. Moreover, calibration precision of the new items will directly affect the estimation accuracy of examinees’ ability vectors. In unidimensional CAT (UCAT) and cognitive diagnostic CAT, online calibration techniques have been developed to effectively calibrate new items. However, there has been very little discussion of online calibration in MCAT in the literature. Thus, this paper proposes new online calibration methods for MCAT based upon some popular methods used in UCAT. Three representative methods, Method A, the ‘one EM cycle’ method and the ‘multiple EM cycles’ method, are generalized to MCAT. Three simulation studies were conducted to compare the three new methods by manipulating three factors (test length, item bank design, and level of correlation between coordinate dimensions). The results showed that all the new methods were able to recover the item parameters accurately, and the adaptive online calibration designs showed some improvements compared to the random design under most conditions.     

Non‐ignorable missingness item response theory models for choice effects in examinee‐selected items

Examinee‐selected item (ESI) design, in which examinees are required to respond to a fixed number of items in a given set, always yields incomplete data (i.e., when only the selected items are answered, data are missing for the others) that are likely non‐ignorable in likelihood inference. Standard item response theory (IRT) models become infeasible when ESI data are missing not at random (MNAR). To solve this problem, the authors propose a two‐dimensional IRT model that posits one unidimensional IRT model for observed data and another for nominal selection patterns. The two latent variables are assumed to follow a bivariate normal distribution. In this study, the mirt freeware package was adopted to estimate parameters. The authors conduct an experiment to demonstrate that ESI data are often non‐ignorable and to determine how to apply the new model to the data collected. Two follow‐up simulation studies are conducted to assess the parameter recovery of the new model and the consequences for parameter estimation of ignoring MNAR data. The results of the two simulation studies indicate good parameter recovery of the new model and poor parameter recovery when non‐ignorable missing data were mistakenly treated as ignorable.     

Assessment of Differential Rater Functioning in Latent Classes with New Mixture Facets Models

Multifaceted data are very common in the human sciences. For example, test takers' responses to essay items are marked by raters. If multifaceted data are analyzed with standard facets models, it is assumed there is no interaction between facets. In reality, an interaction between facets can occur, referred to as differential facet functioning. A special case of differential facet functioning is the interaction between ratees and raters, referred to as differential rater functioning (DRF). In existing DRF studies, the group membership of ratees is known, such as gender or ethnicity. However, DRF may occur when the group membership is unknown (latent) and thus has to be estimated from data. To solve this problem, in this study, we developed a new mixture facets model to assess DRF when the group membership is latent and we provided two empirical examples to demonstrate its applications. A series of simulations were also conducted to evaluate the performance of the new model in the DRF assessment in the Bayesian framework. Results supported the use of the mixture facets model because all parameters were recovered fairly well, and the more data there were, the better the parameter recovery.     

A Hierarchical Rater Model for Longitudinal Data

Research studies in psychology and education often seek to detect changes or growth in an outcome over a duration of time. This research provides a solution to those interested in estimating latent traits from psychological measures that rely on human raters. Rater effects potentially degrade the quality of scores in constructed response and performance assessments. We develop an extension of the hierarchical rater model (HRM), which yields estimates of latent traits that have been corrected for individual rater bias and variability, for ratings that come from longitudinal designs. The parameterization, called the longitudinal HRM (L-HRM), includes an autoregressive time series process to permit serial dependence between latent traits at adjacent timepoints, as well as a parameter for overall growth. We evaluate and demonstrate the feasibility and performance of the L-HRM using simulation studies. Parameter recovery results reveal predictable amounts and patterns of bias and error for most parameters across conditions. An application to ratings from a study of character strength demonstrates the model. We discuss limitations and future research directions to improve the L-HRM.     

《中国大学生心理健康量表》的编制

本研究在文献研究、咨询案例分析、专家访谈和讨论、开放式问卷调查的基础上,采用实证法,编制了大学生心理健康量表,制定了全国大学生常模,并对其信度和效度进行了检验。结果表明:该量表具有良好的信度与效度,可以用于我国大学生心理健康水平的的检测。     

Higher-order latent trait models for cognitive diagnosis

Higher-order latent traits are proposed for specifying the joint distribution of binary attributes in models for cognitive diagnosis. This approach results in a parsimonious model for the joint distribution of a high-dimensional attribute vector that is natural in many situations when specific cognitive information is sought but a less informative item response model would be a reasonable alternative. This approach stems from viewing the attributes as the specific knowledge required for examination performance, and modeling these attributes as arising from a broadly-defined latent trait resembling theϑ of item response models. In this way a relatively simple model for the joint distribution of the attributes results, which is based on a plausible model for the relationship between general aptitude and specific knowledge. Markov chain Monte Carlo algorithms for parameter estimation are given for selected response distributions, and simulation results are presented to examine the performance of the algorithm as well as the sensitivity of classification to model misspecification. An analysis of fraction subtraction data is provided as an example. This research was funded by National Institute of Health grant R01 CA81068. We would like to thank William Stout and Sarah Hartz for many useful discussions, three anonymous reviewers for helpful comments and suggestions, and Kikumi Tatsuoka and Curtis Tatsuoka for generously sharing data.     

汉语无关联词因果复句的命题表征项目互换效应

采用句子—图画验证任务探讨了汉语无关联词因果复句的命题表征项目互换效应。结果表明 ,汉语无关联词因果复句的项目表达顺序为“结果→原因”时 ,所形成的命题表征中项目发生互换 ,其结果为“原因→结果”。本研究结果初步表明 ,在没有关联词的语法提示条件下 ,读者在理解汉语无关联词的因果复句的过程中能够根据其语义内容按照“原因→结果”固定方向进行系列的认知加工     

Differential item functioning of the Rosenberg Self‐Esteem Scale in the US and China: Measurement bias matters

Using an item‐response theory‐based approach (i.e. likelihood ratio test with an iterative procedure), we examined the equivalence of the Rosenberg Self‐Esteem Scale (RSES) in a sample of US and Chinese college students. Results from the differential item functioning (DIF) analysis showed that the RSES was not fully equivalent at the item level, as well as at the scale level. The two cultural groups did not use the scale comparably, with the US students showing more extreme responses than the Chinese students. Moreover, we evaluated the practical impact of DIF and found that cultural differences in average self‐esteem scores disappeared after the DIF was taken into account. In the present study, we discuss the implications of our findings for cross‐cultural research and provide suggestions for future studies using the RSES in China.     

基于等级反应模型的规则空间方法

本研究基于Tatsuoka的规则空间方法, 对理想反应模式与异常反应指标进行了扩展, 推导了多级评分项目下规则空间方法的算法公式。在4种属性层级结构(发散型、收敛型、线型与无结构型)×4种“失误”作答概率(2%、5%、10%与15%)测验情境下, 以属性模式判准率、被试属性判准率、敏感性与特异性为指标, 检验了多级评分项目下规则空间方法的分类准确性。结果表明：(1) 基于多级评分项目构建的异常反应指标, 能有效地对被试进行分类与解释, 且0-1评分项目下异常反应指标及其性质都是多级评分下的特例; (2) 随着“失误”作答概率的增加, 4种属性层级结构的分类准确性都会降低; (3) 线型和收敛型的分类准确性明显好于发散型与无结构型; (4) 纯规则点的分布对规则空间方法的分类准确性有显著影响。