The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality

To assess the reliability of congeneric tests, specifically designed reliability measures have been proposed. This paper emphasizes that such measures rely on a unidimensionality hypothesis, which can neither be confirmed nor rejected when there are only three test parts, and will invariably be rejected when there are more than three test parts. Jackson and Agunwamba's (1977) greatest lower bound to reliability is proposed instead. Although this bound has a reputation for overestimating the population value when the sample size is small, this is no reason to prefer the unidimensionality-based reliability. Firstly, the sampling bias problem of the glb does not play a role when the number of test parts is small, as is often the case with congeneric measures. Secondly, glb and unidimensionality based reliability are often equal when there are three test parts, and when there are more test parts, their numerical values are still very similar. To the extent that the bias problem of the greatest lower bound does play a role, unidimensionality-based reliability is equally affected. Although unidimensionality and reliability are often thought of as unrelated, this paper shows that, from at least two perspectives, they act as antagonistic concepts. A measure, based on the same framework that led to the greatest lower bound, is discussed for assessing how close is a set of variables to unidimensionality. It is the percentage of common variance that can be explained by a single factor. An empirical example is given to demonstrate the main points of the paper. The authors are obliged to Henk Kiers for commenting on a previous version. Gregor Sočan is now at the University of Ljubljana.     

On kristof's test for a linear relation between true scores of two measures

The hypothesis that two variables have a perfect disattenuated correlation and hence measure the same trait, except for errors of measurement, is discussed. Equivalently, the underlying variables, the true scores, are related linearly. We show that several previously proposed ad hoc tests are in fact likelihood ratio tests. The cases when the linear relation is specified and when it is unspecified are both discussed.This work was done while the author was at Purdue University Under Air Force Grant AFOSR-72-2350B.     

Some improved diagnostics for failure of the Rasch model

Although several goodness of fit tests have been developed for the Rasch model for dichotomous items, most of them are of a global, asymptotic, and confirmatory type. This paper, based on ideas from a recent thesis by Van den Wollenberg, offers some suggestions for local, small sample, and exploratory techniques: difficulty plots for person groups scoring right and wrong on a specific item, a slope test per item based on a binomial distribution per score group, and a unidimensionality check based on an extended hypergeometric distribution per score group. This paper owes much to the inspiring and pioneering work of Arnold Van den Wollenberg, of which only minor aspects are criticized. Thanks go to Charles Lewis for stimulating discussions and for solutions to some programming problems.     

Identifiability of nonlinear logistic test models

The linear logistic test model (LLTM) specifies the item parameters as a weighted sum of basic parameters. The LLTM is a special case of a more general nonlinear logistic test model (NLTM) where the weights are partially unknown. This paper is about the identifiability of the NLTM. Sufficient and necessary conditions for global identifiability are presented for a NLTM where the weights are linear functions, while conditions for local identifiability are shown to require a model with less restrictions. It is also discussed how these conditions are checked using an algorithm due to Bekker, Merckens, and Wansbeek (1994). Several illustrations are given.This article was written while the first author was a post doctoral fellow at the university of Twente. He gratefully acknowledges the university's hospitality and the financial support by NWO (project nr. 30002).     

A new interpretation of stochastic test models

A new look at latent trait models is proposed. The event of an item being solved by a person is related to the event that the momentary value of a person-specific random component is at least as large as the corresponding value of an item-specific random component. The Birnbaum logistic test model is shown to be generated by a bivariate extreme value distribution for the components. Some consequences of this interpretation are outlined.I am indebted to David Strauss for calling the extreme value distribution (1) to my attention in a random utility context. The paper also benefited from discussions with H. C. Micko, H. H. Schulze and K. F. Wender.     

Model Selection of Nested and Non-Nested Item Response Models Using Vuong Tests

Abstract

In this paper, we apply Vuong’s general approach of model selection to the comparison of nested and non-nested unidimensional and multidimensional item response theory (IRT) models. Vuong’s approach of model selection is useful because it allows for formal statistical tests of both nested and non-nested models. However, only the test of non-nested models has been applied in the context of IRT models to date. After summarizing the statistical theory underlying the tests, we investigate the performance of all three distinct Vuong tests in the context of IRT models using simulation studies and real data. In the non-nested case we observed that the tests can reliably distinguish between the graded response model and the generalized partial credit model. In the nested case, we observed that the tests typically perform as well as or sometimes better than the traditional likelihood ratio test. Based on these results, we argue that Vuong’s approach provides a useful set of tools for researchers and practitioners to effectively compare competing nested and non-nested IRT models.     

Power of the likelihood ratio test in covariance structure analysis

A procedure for computing the power of the likelihood ratio test used in the context of covariance structure analysis is derived. The procedure uses statistics associated with the standard output of the computer programs commonly used and assumes that a specific alternative value of the parameter vector is specified. Using the noncentral Chi-square distribution, the power of the test is approximated by the asymptotic one for a sequence of local alternatives. The procedure is illustrated by an example. A Monte Carlo experiment also shows how good the approximation is for a specific case.This research was made possible by a grant from the Dutch Organization for Advancement of Pure Research (ZWO). The authors also like to acknowledge the helpful comments and suggestions from the editor and anonymous reviewers.     

On the asymptotic distributions of two statistics for two-level covariance structure models within the class of elliptical distributions

Since data in social and behavioral sciences are often hierarchically organized, special statistical procedures for covariance structure models have been developed to reflect such hierarchical structures. Most of these developments are based on a multivariate normality distribution assumption, which may not be realistic for practical data. It is of interest to know whether normal theory-based inference can still be valid with violations of the distribution condition. Various interesting results have been obtained for conventional covariance structure analysis based on the class of elliptical distributions. This paper shows that similar results still hold for 2-level covariance structure models. Specifically, when both the level-1 (within cluster) and level-2 (between cluster) random components follow the same elliptical distribution, the rescaled statistic recently developed by Yuan and Bentler asymptotically follows a chi-square distribution. When level-1 and level-2 have different elliptical distributions, an additional rescaled statistic can be constructed that also asymptotically follows a chi-square distribution. Our results provide a rationale for applying these rescaled statistics to general non-normal distributions, and also provide insight into issues related to level-1 and level-2 sample sizes. The authors thank an associate editor and three referees for their constructive comments, which led to an improved version of the paper. This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of Notre Dame faculty research grant.     

使用似然比D2统计量的题目属性定义方法

题目属性的定义是实施认知诊断评价的关键步骤, 通过有丰富经验的领域专家对题目的属性进行定义是当前的主要方法, 然而该方法受到许多主观经验因素的影响。寻找客观的题目属性定义或验证方法可以为主观定义过程提供策略支持或对结果进行改进, 因此已经引起研究者们的关注。本研究构建了一种简单高效的题目属性定义方法, 研究使用似然比D2统计量从作答数据中估计题目属性的方法, 实现属性掌握模式、题目参数和题目属性向量的联合估计。模拟研究结果表明, 使用似然比D2统计量可以有效地识别题目的属性向量, 该方法一方面可以实现新编制题目属性向量的在线估计, 另一方面可以验证已经定义的题目属性向量的准确性。     

Marginalized maximum a posteriori estimation for the four-parameter logistic model under a mixture modelling framework

The four-parameter logistic model (4PLM) has recently attracted much interest in various applications. Motivated by recent studies that re-express the four-parameter model as a mixture model with two levels of latent variables, this paper develops a new expectation–maximization (EM) algorithm for marginalized maximum a posteriori estimation of the 4PLM parameters. The mixture modelling framework of the 4PLM not only makes the proposed EM algorithm easier to implement in practice, but also provides a natural connection with popular cognitive diagnosis models. Simulation studies were conducted to show the good performance of the proposed estimation method and to investigate the impact of the additional upper asymptote parameter on the estimation of other parameters. Moreover, a real data set was analysed using the 4PLM to show its improved performance over the three-parameter logistic model.     

Construct validity of the Sivik Psychosomaticism test and test of Operational style: Correlations with four Minnesota Multiphasic Personality Inventory (MMPI) subscales

To evaluate the construct validity (convergent and divergent) of the Sivik Psycho Somaticism test (SPS) and test of Operationality (OPER), Pearson correlation coefficients between SPS scales and subscales, OPER and Minnesota Multiphasic Personality Inventory (MMPI) subscales Hypochondria (Hs), Depression (D), Hysteria (Hy) and Alexithymia (Al) were calculated. Eighty-eight healthy individuals and 285 psychosomatic patients completed the SPS and OPER tests and MMPI; Hs, D, Hy and Al. The results show that most of the SPS subscales and OPER are significantly correlated to several MMPI subscales in both a normal and a psychosomatic population. The results are in concordance with the theoretical hypotheses and confirm the validity of the SPS and OPER constructs.     

On equivalence between a partial credit item and a set of independent Rasch binary items

Given a Masters partial credit item withn known step difficulties, conditions are stated for the existence of a set of (locally) independent Rasch binary items such that their raw score and the partial credit raw score have identical probability density functions. The conditions are those for the existence ofn positive values with predetermined elementary symmetric functions and include the requirement that then step difficulties form an increasing sequence.     

On the existence and uniqueness of maximum-likelihood estimates in the Rasch model

Necessary and sufficient conditions for the existence and uniqueness of a solution of the so-called unconditional (UML) and the conditional (CML) maximum-likelihood estimation equations in the dichotomous Rasch model are given. The basic critical condition is essentially the same for UML and CML estimation. For complete data matricesA, it is formulated both as a structural property ofA and in terms of the sufficient marginal sums. In case of incomplete data, the condition is equivalent to complete connectedness of a certain directed graph. It is shown how to apply the results in practical uses of the Rasch model.Paper read at the European Meeting of the Psychometric Society, Groningen, June 19–21, 1980.Part of the research reported herein was done while the author was staying at the Pulmologisches Zentrum der Stadt Wien; he is indebted to Professor Dr. F. Muhar and Dr. R. Mutschlechner for providing excellent working conditions.     

Introducing Emotioncy as a Potential Source of Test Bias: A Mixed Rasch Modeling Study

This article attempts to present emotioncy as a potential source of test bias to inform the analysis of test item performance. Emotioncy is defined as a hierarchy, ranging from exvolvement (auditory, visual, and kinesthetic) to involvement (inner and arch), to emphasize the emotions evoked by the senses. This study hypothesizes that when individuals have high levels of emotioncy for specific words, their test performance may systematically change, resulting in test bias. To this end, 355 individuals were asked to take a 40-item vocabulary test along with the emotioncy scale. Mixed Rasch model was employed to flag differential item functioning items. Results illustrated that the test takers with high emotioncy toward specific words outperformed the ones in the low-emotioncy group, characterizing emotioncy as a potential source of test bias.     

四参数Logistic模型潜在特质参数的Warm加权极大似然估计

本文以四参数Logistic(4-parameter Logistic,4PL)模型为研究对象,根据Warm的加权极大似然估计技巧,提出了4PL模型潜在特质参数的加权极大似然估计方法,并借助模拟研究对加权极大似然估计的性质进行验证。研究结果表明,与通常的极大似然估计和后验期望估计相比,加权极大似然估计的偏差(bias)明显减小,并且具有良好的返真性能。此外,在测试的长度较短和项目的区分度较小的情况下,加权极大似然估计依然保持了良好的统计性质,表现出更加显著的优势。     

A Statistical Hypothesis Testing Method for the Rank Ordering of the Priorities of the Alternatives in the Analytic Hierarchy Process

In analytic hierarchy process (AHP), a ratio scale (π₁, π₂, ⋯, π_t) for the priorities of the alternatives {T₁, T₂, ⋯, T_t} is used for a decision problem in which π_i/π_j is used to quantify the ratio of the priority of T_i to that of T_j. In a group decision‐making setup, the subjective estimates of π_i/π_j are obtained as entries of a pairwise comparison matrix for each member of the group. On the basis of these pairwise comparison matrices, one of the topics of interest in some situation is the total rank ordering of the priorities of the alternatives. In this article, a statistical method is proposed for testing a specific total rank ordering of the priorities of the alternatives. The method developed is then illustrated using numerical examples. Copyright © 2014 John Wiley & Sons, Ltd.     

Spurious Latent Class Problem in the Mixed Rasch Model: A Comparison of Three Maximum Likelihood Estimation Methods under Different Ability Distributions

Recent research has shown that over-extraction of latent classes can be observed in the Bayesian estimation of the mixed Rasch model when the distribution of ability is non-normal. This study examined the effect of non-normal ability distributions on the number of latent classes in the mixed Rasch model when estimated with maximum likelihood estimation methods (conditional, marginal, and joint). Three information criteria fit indices (Akaike information criterion, Bayesian information criterion, and sample size adjusted BIC) were used in a simulation study and an empirical study. Findings of this study showed that the spurious latent class problem was observed with marginal maximum likelihood and joint maximum likelihood estimations. However, conditional maximum likelihood estimation showed no overextraction problem with non-normal ability distributions.     

Application of the bootstrap methods in factor analysis

A Monte Carlo experiment is conducted to investigate the performance of the bootstrap methods in normal theory maximum likelihood factor analysis both when the distributional assumption is satisfied and unsatisfied. The parameters and their functions of interest include unrotated loadings, analytically rotated loadings, and unique variances. The results reveal that (a) bootstrap bias estimation performs sometimes poorly for factor loadings and nonstandardized unique variances; (b) bootstrap variance estimation performs well even when the distributional assumption is violated; (c) bootstrap confidence intervals based on the Studentized statistics are recommended; (d) if structural hypothesis about the population covariance matrix is taken into account then the bootstrap distribution of the normal theory likelihood ratio test statistic is close to the corresponding sampling distribution with slightly heavier right tail.This study was carried out in part under the ISM cooperative research program (91-ISM · CRP-85, 92-ISM · CRP-102). The authors would like to thank the editor and three reviewers for their helpful comments and suggestions which improved the quality of this paper considerably.     

On coding the position of letters in words: a test of two models

Open-bigram and spatial-coding schemes provide different accounts of how letter position is encoded by the brain during visual word recognition. Open-bigram coding involves an explicit representation of order based on letter pairs, while spatial coding involves a comparison function operating over representations of individual letters. We identify a set of priming conditions (subset primes and reversed interior primes) for which the two types of coding schemes give opposing predictions, hence providing the opportunity for strong scientific inference. Experimental results are consistent with the open-bigram account, and inconsistent with the spatial-coding scheme.     

The rasch model,the law of comparative judgment and additive conjoint measurement

Relationships between the Rasch model and both the law of comparative judgment and additive conjoint measurement are discussed. The distance between the ability of Persona and the difficult of Itemi is, in the Rasch model, the baseline value corresponding to the probability thata will respond correctly toi, where this probability is interpreted as the area under a logistic curve (which is substantially equivalent to the normal curve) and is thus an application of the law of comparative judgment. Under certain assumptions, the Rasch model is also a special case of additive conjoint measurement and, properly reinterpreted, may be usefully applied in contexts other than individual differences.