TouchStat: A Monte Carlo program for calculating sequential touching probabilities

A growing number of child cognition researchers are using an object-manipulation, sequential-touching paradigm to explore young children’s conceptual abilities. Within this paradigm, it is essential to distinguish children’s intracategory touching sequences from those expected by chance. The sequentialtouching approach is congruent with a permutation testing model of statistical inference and is best modeled by sampled permutations as derived from Monte Carlo procedures. In this article, we introduce a computer program for generating Monte Carlo sequential-touching simulations. TouchStat permits users to determine their own specifications to simulate sequential touching to category exemplars across a wide range of task parameters. We also present Monte Carlo chance probabilities for the standard two-category, four-exemplar task, with sequences up to 30 touches long. Finally, we consider broader applications of the TouchStat program.     

K-means、潜在类别模型和混合Rasch模型的比较

基于模拟研究比较了K-means方法、潜在类别模型和混合Rasch模型在二分外显变量情境下的聚类效果.结果表明：（1）潜在类别数量、变量数量、样本量、样本平衡和变量间相关对K-means方法、潜在类别模型和混合Rasch模型的分类准确性均有影响且因素间的交互作用存在;（2）除了在2个潜在类别的样本不平衡条件下K-means方法表现较差外,在其他条件下与潜在类别模型和混合Rasch模型的表现相当;（3）混合Rasch模型的分类一致性在2个潜在类别的情境下要好于潜在类别模型,但是在4个潜在类别的情境下要差于潜在类别模型.     

Cognitive diagnosis modelling incorporating item response times

To provide more refined diagnostic feedback with collateral information in item response times (RTs), this study proposed joint modelling of attributes and response speed using item responses and RTs simultaneously for cognitive diagnosis. For illustration, an extended deterministic input, noisy ‘and’ gate (DINA) model was proposed for joint modelling of responses and RTs. Model parameter estimation was explored using the Bayesian Markov chain Monte Carlo (MCMC) method. The PISA 2012 computer-based mathematics data were analysed first. These real data estimates were treated as true values in a subsequent simulation study. A follow-up simulation study with ideal testing conditions was conducted as well to further evaluate model parameter recovery. The results indicated that model parameters could be well recovered using the MCMC approach. Further, incorporating RTs into the DINA model would improve attribute and profile correct classification rates and result in more accurate and precise estimation of the model parameters.     

Category width and differentiation in semantic categories

The hypothesis that the judgemental categorizing behaviour measured by Pettigrew's Category-Width scale is linked to semantic categorization is investigated. Individual differences in broad vs. narrow categorizing behaviour measured by the C-W scale are reconceptualized within the framework of Rosch's (1977) theory of human categorization. Two extreme groups on the C-W scale, broad categorizers and narrow categorizers, were selected as subjects. One experiment employed a short-term release-from-proactive-inhibition paradigm. The amount of discrepancy in recall between the preshift and the shift trials was used as an index to perceived semantic distance. The result showed that narrow categorizers perceived the same fuzzy items in a semantic category as less similar to the good exemplars of that category than broad categorizers did. In a further experiment using a semantic-categorization technique, narrow categorizers judged the same fuzzy items as category members less frequently than broad categorizers did, indicating that narrow categorizers possess more differentiated semantic categories than do broad categorizers. These findings are seen as an extension of the differential categorizing behaviour on artificial categories in the C-W scale to semantic categorization for which there is a pre-existing system.     

Multiway contingency tables: Monte Carlo resampling probability values for the chi-squared and likelihood-ratio tests

Monte Carlo resampling methods to obtain probability values for chi-squared and likelihood-ratio test statistics for multiway contingency tables are presented. A resampling algorithm provides random arrangements of cell frequencies in a multiway contingency table, given fixed marginal frequency totals. Probability values are obtained from the proportion of resampled test statistic values equal to or greater than the observed test statistic value.     

Studying Canonical Analysis: A Reply To Thorndike's Comments

Robert M. Thorndike (1976) commented on the results of a Monte Carlo study on the stability of canonical correlations, canonical weights, and canonical variate-variable correlations (Barcikowski and Stevens, 1975). In this paper each of his comments are examined by the authors of the Monte Carlo Study. In addition, a possible solution to the large number of subjects necessary for stable weights and variate-variable correlations using ridge regression procedures is suggested.     

Detecting latent taxa: Monte Carlo comparison of taxometric, mixture model, and clustering procedures

A Monte Carlo evaluation of four procedures for detecting taxonicity was conducted using artificial data sets that were either taxonic or nontaxonic. The data sets were analyzed using two of Meehl's taxometric procedures, MAXCOV and MAMBAC, Ward's method for cluster analysis in concert with the cubic clustering criterion and a latent variable mixture modeling technique. Performance of the taxometric procedures and latent variable mixture modeling were clearly superior to that of cluster analysis in detecting taxonicity. Applied researchers are urged to select from the better procedures and to perform consistency tests.     

A parametric procedure for ultrametric tree estimation from conditional rank order proximity data

The psychometric and classification literatures have illustrated the fact that a wide class of discrete or network models (e.g., hierarchical or ultrametric trees) for the analysis of ordinal proximity data are plagued by potential degenerate solutions if estimated using traditional nonmetric procedures (i.e., procedures which optimize a STRESS-based criteria of fit and whose solutions are invariant under a monotone transformation of the input data). This paper proposes a new parametric, maximum likelihood based procedure for estimating ultrametric trees for the analysis of conditional rank order proximity data. We present the technical aspects of the model and the estimation algorithm. Some preliminary Monte Carlo results are discussed. A consumer psychology application is provided examining the similarity of fifteen types of snack/breakfast items. Finally, some directions for future research are provided.     

The problem of measurement model misspecification in behavioral and organizational research and some recommended solutions

The purpose of this study was to review the distinction between formative- and reflective-indicator measurement models, articulate a set of criteria for deciding whether measures are formative or reflective, illustrate some commonly researched constructs that have formative indicators, empirically test the effects of measurement model misspecification using a Monte Carlo simulation, and recommend new scale development procedures for latent constructs with formative indicators. Results of the Monte Carlo simulation indicated that measurement model misspecification can inflate unstandardized structural parameter estimates by as much as 400% or deflate them by as much as 80% and lead to Type I or Type II errors of inference, depending on whether the exogenous or the endogenous latent construct is misspecified. Implications of this research are discussed.     

Re-evaluation of monte carlo studies in nonmetric multidimensional scaling

Critical examination is made of the recent controversy over the value of Monte Carlo techniques in nonmetric multidimensional scaling procedures. The case is presented that the major relevance of Monte Carlo studies is not for the local minima problem but for the meaningfulness of the obtained solutions.     

Testing for robustness in Monte Carlo studies

Monte Carlo studies provide the information needed to help researchers select appropriate analytical procedures under design conditions in which the underlying assumptions of the procedures are not met. In Monte Carlo studies, the 2 errors that one could commit involve (a) concluding that a statistical procedure is robust when it is not or (b) concluding that it is not robust when it is. In previous attempts to apply standard statistical design principles to Monte Carlo studies, the less severe of these errors has been wrongly designated the Type I error. In this article, a method is presented for controlling the appropriate Type I error rate; the determination of the number of iterations required in a Monte Carlo study to achieve desired power is described; and a confidence interval for a test's true Type I error rate is derived. A robustness criterion is also proposed that is a compromise between W. G. Cochran's (1952) and J. V. Bradley's (1978) criteria.     

Bayesian factor analysis for multilevel binary observations

Multilevel covariance structure models have become increasingly popular in the psychometric literature in the past few years to account for population heterogeneity and complex study designs. We develop practical simulation based procedures for Bayesian inference of multilevel binary factor analysis models. We illustrate how Markov Chain Monte Carlo procedures such as Gibbs sampling and Metropolis-Hastings methods can be used to perform Bayesian inference, model checking and model comparison without the need for multidimensional numerical integration. We illustrate the proposed estimation methods using three simulation studies and an application involving student's achievement results in different areas of mathematics. The authors thank Ian Westbury, University of Illinois at Urbana Champaign for kindly providing the SIMS data for the application.     

几种常用非补偿型认知诊断模型的比较与选用：基于属性层级关系的考量

当前国际上开发了60多种认知诊断计量模型(Fu &; Li, 2007), 各种模型各具特点, 实际应用者应根据实际情况选用恰当的模型。本研究以属性层级关系为切入点, 采用Monte Carlo模拟的研究方法, 比较了属性层级关系正确及有误两种情况下, 当前国际上常用的五种认知诊断模型的性能, 以充分考察不同认知诊断模型对属性层级关系的依赖程度, 及属性层级关系的错误界定对各认知诊断模型诊断正确率的影响, 从而为实际运用者在认知诊断模型选用上提供借鉴和参考。     

一种基于Q矩阵理论朴素的认知诊断方法

现有的认知诊断方法均是在复杂的统计测量学知识基础上构建的, 需要经过大量的运算才可实现对被试的诊断分类。这使得相关研究者及一线教师在理解和运用某一认知诊断方法时困难重重。相比之下, 孙佳楠、张淑梅、辛涛和包钰(2011)提出的广义距离判别法(GDD)较其他认知诊断方法更简单易用且分类准确率高。本研究在改进的Q矩阵理论(丁树良, 祝玉芳, 林海菁, 蔡艳, 2009; 丁树良, 杨淑群, 汪文义, 2010)的基础上, 借鉴GDD的思路, 提出一种无需进行参数估计的朴素的认知诊断方法, 即海明距离判别法(HDD)。根据判别方式的不同将其分为R方法和B方法。采用Monte Carlo模拟的研究方法, 以模式判准率(PMR)和属性平均判准率(AAMR)作为衡量被试知识状态分类准确率的指标, 与GDD进行比较。结果表明, HDD具有更简便的操作步骤和更好的分类准确率。     

The effects of degree of learning,meaning, and invidual differences on the feeling-of-knowing

Research on the feeling-of-knowing (FOK) phenomenon has established it as a valid indicator of the contents of memory and a reliable predictor of subsequent performance. It has been shown that degree of prior learning of items is one factor upon which FOK is based — the ‘overlearning’ effect. The present study was designed to determine, firstly, whether the overlearning effect is due to different numbers of presentations of physical attributes during learning, to encoding variability, or to both, and secondly, whether individual differences in the cognitive style variable of category width affect FOK judgments or accuracy. Using a delayed recall-FOK judgment-recognition paradigm, measures on these tests were obtained from narrow and broad categorizers, who learned paired-associate items to a criterion of 1 or 3 correct recalls. The target item, a homograph, to be produced in response to the cue, a number, was learned either as a word of single or multiple meaning. Results showed that a greater degree of learning did not increase FOK magnitude or accuracy, although it did increase recall and overall recognition. On the other hand, category width and encoding variability did not affect recall and recognition, but did influence magnitude and accuracy of FOK: Broad categorizers gave higher FOK ratings than narrow, and when single meanings of the target only were learned, broad categorizers gave higher ratings than narrow. This finding was not replicated in the accuracy of FOK responses: Broad categorizers were more accurate for multiple target encoding but less accurate for single. Thus FOK is influenced by semantic aspects of the target in different ways by subjects exhibiting different cognitive styles. The study places FOK firmly in the metamemory domain.     

9-26个月婴儿上位、基本和下位水平类别学习的发展顺序

该研究探索了9-26个月婴儿在上位水平类别、基本水平类别和下位水平类别的发展顺序。采用触摸屏式的序列触摸任务,36名婴儿参加实验,分9-14月、15-20月、21-26月三个年龄组。以平均序列长度作为组分析的因变量指标,以Dixon等（1998;2007）开发的Touchstat V3.0软件统计的归类者概率作为个别分析的因变量指标,结果表明,9-14个月、15-20个月和21-26个月三个年龄组的婴儿,都能显著地区分基本水平的类别;15-20个月和21-26个月婴儿,能够显著区分上位水平的类别;21-26个月的婴儿,能够区分下位水平的类别,说明了婴儿不同类别水平概念的学习,是按照基本水平（L2）,然后上位水平（L1）,最后才是下位水平（L3）的顺序发展的,即L2→L1→L3。     

Heuristic cognitive diagnosis when the Q‐matrix is unknown

Cognitive diagnosis models of educational test performance rely on a binary Q‐matrix that specifies the associations between individual test items and the cognitive attributes (skills) required to answer those items correctly. Current methods for fitting cognitive diagnosis models to educational test data and assigning examinees to proficiency classes are based on parametric estimation methods such as expectation maximization (EM) and Markov chain Monte Carlo (MCMC) that frequently encounter difficulties in practical applications. In response to these difficulties, non‐parametric classification techniques (cluster analysis) have been proposed as heuristic alternatives to parametric procedures. These non‐parametric classification techniques first aggregate each examinee's test item scores into a profile of attribute sum scores, which then serve as the basis for clustering examinees into proficiency classes. Like the parametric procedures, the non‐parametric classification techniques require that the Q‐matrix underlying a given test be known. Unfortunately, in practice, the Q‐matrix for most tests is not known and must be estimated to specify the associations between items and attributes, risking a misspecified Q‐matrix that may then result in the incorrect classification of examinees. This paper demonstrates that clustering examinees into proficiency classes based on their item scores rather than on their attribute sum‐score profiles does not require knowledge of the Q‐matrix, and results in a more accurate classification of examinees.     

Spontaneous categorizers retain more than spontaneous alphabetizers

Immediate and delayed free recall of unstructured verbal materials were assessed for alphabetizers and categorizers after varying study times. Alphabetizers and categorizers did not differ in immediate recall, but retention following a 6-day interval was consistently higher for categorizers than for alphabetizers. Both immediate and delayed recall increased with study time, which did not interact with learning strategy. In the next experiment alphabetizers and categorizers learned two lists, the second of which was categorically structured or unstructured. No retroactive interference on first-list recall was produced by the learning of the structured list, but the learning of the unstructured list produced retroactive interference for the alphabetizers only. It was suggested that, although both alphabetizers and categorizers probably use their respective strategies for other verbal tasks, the categoric strategy should create less interference because specific 'categories are rarely encountered in successive tasks.     

Bayesian Model Comparison for the Order Restricted RC Association Model

Association models constitute an attractive alternative to the usual log-linear models for modeling the dependence between classification variables. They impose special structure on the underlying association by assigning scores on the levels of each classification variable, which can be fixed or parametric. Under the general row-column (RC) association model, both row and column scores are unknown parameters without any restriction concerning their ordinality. However, when the classification variables are ordinal, order restrictions on the scores arise naturally. Under such restrictions, we adopt an alternative parameterization and draw inferences about the equality of adjacent scores using the Bayesian approach. To achieve that, we have constructed a reversible jump Markov chain Monte Carlo algorithm for moving across models of different dimension and estimate accurately the posterior model probabilities which can be used either for model comparison or for model averaging. The proposed methodology is evaluated through a simulation study and illustrated using actual datasets.     

Variability and its sources in infant categorization

Variability of infants’ categorization performance and potential sources of this variability were investigated. Using data from 13 categorization studies employing a habituation-of-looking paradigm with infants 3, 5, 6, and 9 months of age, a method for establishing a categorization criterion was developed and then used to classify individual infants as “categorizers” for particular tasks. Logistic regression analyses were then used to identify demographic and information-processing variables that predicted “categorizer” classification. Variables that increased the odds of being classified as a categorizer were gender, number of habituation trials, and duration of peak look during habituation; total looking time during habituation decreased the odds of categorizer classification. These findings are discussed in the context of individual differences in information processing.