Exact tests for the rasch model via sequential importance sampling

Rasch proposed an exact conditional inference approach to testing his model but never implemented it because it involves the calculation of a complicated probability. This paper furthers Rasch’s approach by (1) providing an efficient Monte Carlo methodology for accurately approximating the required probability and (2) illustrating the usefulness of Rasch’s approach for several important testing problems through simulation studies. Our Monte Carlo methodology is shown to compare favorably to other Monte Carlo methods proposed for this problem in two respects: it is considerably faster and it provides more reliable estimates of the Monte Carlo standard error.This Research was supported in part by National Science Foundation grant DMS-0203762 and a University of Pennsylvania Research Foundation grant.The authors are grateful to Don Burdick for helpful comments. In addition, the authors wish to thank the editor, the associate editor, and the referees for their helpful suggestions.This revised article was published online in August 2005 with the PDF paginated correctly.     

A note on the equivalence of the graded response model and the sequential model

This paper concerns items that consist of several item steps to be responded to sequentially. The item scoreX is defined as the number of correct responses until the first failure. Samejima's graded response model states that each steph=1,...,m is characterized by a parameterb _h , and, for a subject with ability, Pr(Xh; )=F(–b _h ). Tutz's general sequential model associates with each step a parameterdh, and it states that Pr(Xh;)= _r ^=1h G(–d _r ). Tutz's (1991, 1997) conjectures that the models are equivalent if and only ifF(x)=G(x) is an extreme value distribution. This paper presents a proof for this conjecture.     

On the asymptotic distributions of two statistics for two-level covariance structure models within the class of elliptical distributions

Since data in social and behavioral sciences are often hierarchically organized, special statistical procedures for covariance structure models have been developed to reflect such hierarchical structures. Most of these developments are based on a multivariate normality distribution assumption, which may not be realistic for practical data. It is of interest to know whether normal theory-based inference can still be valid with violations of the distribution condition. Various interesting results have been obtained for conventional covariance structure analysis based on the class of elliptical distributions. This paper shows that similar results still hold for 2-level covariance structure models. Specifically, when both the level-1 (within cluster) and level-2 (between cluster) random components follow the same elliptical distribution, the rescaled statistic recently developed by Yuan and Bentler asymptotically follows a chi-square distribution. When level-1 and level-2 have different elliptical distributions, an additional rescaled statistic can be constructed that also asymptotically follows a chi-square distribution. Our results provide a rationale for applying these rescaled statistics to general non-normal distributions, and also provide insight into issues related to level-1 and level-2 sample sizes. The authors thank an associate editor and three referees for their constructive comments, which led to an improved version of the paper. This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of Notre Dame faculty research grant.     

Cross-cultural differences in processes underlying sequential cognitive activity

Abstract: Similarities and differences in the processes underlying performance on the Stroop Color-Word Test (Card A, 100 color words/black ink; Card B, 100 color patches; Card C, 100 color words/incongruous ink color) were investigated with Japanese ( N = 96) and Americans ( N = 96) with stimulus items read: (a) vertically, top to bottom; and (b) horizontally, left to right. Total time as well as errors, namely, those involved with the underlying processes of identification and sequential organization, were significantly longer or more frequent on Card C than on Cards A and B for both nationalities and spatial arrangements. Further, greater frequency of serial organization errors in females than in males was obtained for the Japanese group, and the opposite for the American group. The implications and directions for future research are discussed.     

Asymptotic null distribution of person fit statistics with estimated person parameter

Person fit statistics are considered for dichotomous item response models. The asymptotic null distribution is derived for statistics which are linear in the item responses, and in which the ability parameter is replaced by an estimate. This allows the asymptotically correct standardization of linear person fit statistics with estimated ability parameter. The fact that the ability parameter is estimated usually decreases the asymptotic variance.I am indebted to Herbert Hoijtink and three anonymous referees for their comments on a previous version.     

Comparing the effects of two correction procedures on human acquisition of sequential behavior patterns

Thirty-one college undergraduates learned to touch abstract stimuli on a computer screen in arbitrarily designated “correct” sequential orders. Four sets of seven stimuli were used; the stimuli were arrayed horizontally on the screen in random sequences. A correct response (i.e., touching first the stimulus designated as first) resulted in that stimulus appearing near the top of the screen in its correct sequential position (left to right), and remaining there until the end of the trial. Incorrect responses (i.e., touching a stimulus out of sequence) terminated the trial. New trials displayed either the same sequence as the one on which an error had occurred (same-order correction procedure), or a new random sequence (new-order correction procedure). Whenever all responses occurred in the correct sequence, the next trial displayed a new random sequence. Each phase ended when five consecutive correct response sequences occurred. Initially, the same-order correction procedure increased control by the position as well as by the shape of the stimuli; also, it produced more errors, more total trials, more trials to mastery, and more individual patterns of reacquisition than were produced by the new-order procedure.     

The asymptotic distribution of commonality components

Commonality components have been defined as a method of partitioning squared multiple correlations. In this paper, the asymptotic joint distribution of all 2 ^k – 1 squared multiple correlations is derived. The asymptotic joint distribution of linear combinations of squared multiple correlations is obtained as a corollary. In particular, the asymptotic joint distribution of commonality components are derived as a special case. Simultaneous and nonsimultaneous asymptotic confidence intervals for commonality components can be obtained from this distribution.This work was supported in part by the Spencer Foundation and the National Science Foundation.The authors are grateful to Bryna Siegel-Gorlick for her help in obtaining the data used in Example 4.3, and to the referees for their comments and suggestions.     

On robusiness of the normal-theory based asymptotic distributions of three reliability coefficient estimates

This paper studies the asymptotic distributions of three reliability coefficient estimates: Sample coefficient alpha, the reliability estimate of a composite score following a factor analysis, and the estimate of the maximal reliability of a linear combination of item scores following a factor analysis. Results indicate that the asymptotic distribution for each of the coefficient estimates, obtained based on a normal sampling distribution, is still valid within a large class of nonnormal distributions. Therefore, a formula for calculating the standard error of the sample coefficient alpha, recently obtained by van Zyl, Neudecker and Nel, applies to other reliability coefficients and can still be used even with skewed and kurtotic data such as are typical in the social and behavioral sciences.This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of North Texas faculty research grant. We would like to thank the Associate Editor and two reviewers for suggestions that helped to improve the paper.     

The asymptotic posterior normality of the latent trait in an IRT model

It has long been part of the item response theory (IRT) folklore that under the usual empirical Bayes unidimensional IRT modeling approach, the posterior distribution of examinee ability given test response is approximately normal for a long test. Under very general and nonrestrictive nonparametric assumptions, we make this claim rigorous for a broad class of latent models.This research was partially supported by Office of Naval Research Cognitive and Neural Sciences Grant N0014-J-90-1940, 442-1548, National Science Foundation Mathematics Grant NSF-DMS-91-01436, and the National Center for Supercomputing Applications. We wish to thank Kumar Joag-dev and Zhiliang Ying for enlightening suggestions concerning the proof of the basic result.The authors wish to thank Kumar Joag-Dev, Brian Junker, Bert Green, Paul Holland, Robert Mislevy, and especially Zhiliang Ying for their useful comments and discussions.     

The asymptotic distribution of response probabilities in the Audley-Jonckheere learning model

The asymptotic distributions of response probabilities in the Audley-Jonckheere learning model, as applied to the two-choice learning situation, are described for (1) the experimenter-controlled events model, (2) the subject-controlled events model, and (3) the experimenter-subject-controlled events model.     

On the asymptotic standard error of a class of robust estimators of ability in dichotomous item response models

In item response theory, the classical estimators of ability are highly sensitive to response disturbances and can return strongly biased estimates of the true underlying ability level. Robust methods were introduced to lessen the impact of such aberrant responses on the estimation process. The computation of asymptotic (i.e., large‐sample) standard errors (ASE) for these robust estimators, however, has not yet been fully considered. This paper focuses on a broad class of robust ability estimators, defined by an appropriate selection of the weight function and the residual measure, for which the ASE is derived from the theory of estimating equations. The maximum likelihood (ML) and the robust estimators, together with their estimated ASEs, are then compared in a simulation study by generating random guessing disturbances. It is concluded that both the estimators and their ASE perform similarly in the absence of random guessing, while the robust estimator and its estimated ASE are less biased and outperform their ML counterparts in the presence of random guessing with large impact on the item response process.     

CLASSI: A classification model for the study of sequential processes and individual differences therein

In psychological research, one often aims at explaining individual differences in S-R profiles, that is, individual differences in the responses (R) with which people react to specific stimuli (S). To this end, researchers often postulate an underlying sequential process, which boils down to the specification of a set of mediating variables (M) and the processes that link these mediating variables to the stimuli and responses under study. Obviously, a crucial task is to chart how the individual differences in the S-R profiles are caused by individual differences in the S-M link and/or by individual differences in the M-R link. In this paper we propose a new model, called CLASSI, which was explicitly designed for this task. In particular, the key principle of CLASSI consists of reducing the S, M, and R nodes of a sequential process to a few mutually exclusive types and inducing an S-M and an M-R person typology from the data, with the S-M person types being characterized in terms of if S type then M type rules and the M-R person types in terms of if M type then R type rules. As such, the S-M and M-R person types and their associated if–then rules represent the important individual differences in the S-M and M-R links of the sequential process under study. An algorithm to fit the CLASSI model is described and evaluated in a simulation study. An application of CLASSI to data from the behavioral domain of anger and sadness is discussed. Finally, we relate CLASSI to other methods and discuss possible extensions. The first author is a post-doctoral fellow of the Fund for Scientific Research—Flanders (Belgium). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/05/04).     

Aging effects in sequential modulations of poorer-strategy effects during execution of memory strategies

In this study, we asked young adults and older adults to encode pairs of words. For each item, they were told which strategy to use, interactive imagery or rote repetition. Data revealed poorer-strategy effects in both young adults and older adults: Participants obtained better performance when executing better strategies (i.e., interactive-imagery strategy to encode pairs of concrete words; rote-repetition strategy on pairs of abstract words) than with poorer strategies (i.e., interactive-imagery strategy on pairs of abstract words; rote-repetition strategy on pairs of concrete words). Crucially, we showed that sequential modulations of poorer-strategy effects (i.e., poorer-strategy effects being larger when previous items were encoded with better relative to poorer strategies), previously demonstrated in arithmetic, generalise to memory strategies. We also found reduced sequential modulations of poorer-strategy effects in older adults relative to young adults. Finally, sequential modulations of poorer-strategy effects correlated with measures of cognitive control processes, suggesting that these processes underlie efficient trial-to-trial modulations during strategy execution. Differences in correlations with cognitive control processes were also found between older adults and young adults. These findings have important implications regarding mechanisms underlying memory strategy execution and age differences in memory performance.     

Optimal sequential designs for on-line item estimation

Replenishing item pools for on-line ability testing requires innovative and efficient data collection designs. By generating localD-optimal designs for selecting individual examinees, and consistently estimating item parameters in the presence of error in the design points, sequential procedures are efficient for on-line item calibration. The estimating error in the on-line ability values is accounted for with an item parameter estimate studied by Stefanski and Carroll. LocallyD-optimaln-point designs are derived using the branch-and-bound algorithm of Welch. In simulations, the overall sequential designs appear to be considerably more efficient than random seeding of items.This report was prepared under the Navy Manpower, Personnel, and Training R&D Program of the Office of the Chief of Naval Research under Contract N00014-87-0696. The authors wish to acknowledge the valuable advice and consultation given by Ronald Armstrong, Charles Davis, Bradford Sympson, Zhaobo Wang, Ing-Long Wu and three anonymous reviewers.     

On the geometric approach to multivariate selection

Multivariate selection can be represented as a linear transformation in a geometric framework. This approach has led to considerable simplification in the study of the effects of selection on factor analysis. In this note this approach is extended to describe the effects of selection on regression analysis and to adjust for the effects of selection using the inverse of the linear transformation.     

Statistical concepts in philosophy of science

This article focuses on the role of statistical concepts in both experiment and theory in various scientific disciplines, especially physics, including astronomy, and psychology. In Sect. 1 the concept of uncertainty in astronomy is analyzed from Ptolemy to Laplace and Gauss. In Sect. 2 theoretical uses of probability and statistics in science are surveyed. Attention is focused on the historically important example of radioactive decay. In Sect. 3 the use of statistics in biology and the social sciences is examined, with detailed consideration of various Chi-square statistical tests. Such tests are essential for proper evaluation of many different kinds of scientific hypotheses.     

Process-specific slowing with advancing age: evidence derived from the analysis of sequential effects

In the current study we examined the generalized slowing hypothesis on the mechanisms underlying sequential effects in serial two-choice reaction time tasks. For young adults, sequential effects of conditions with a high and a low stimulus presentation rate respectively pointed to an automatic and an expectancy mechanism. Older adults' low presentation rate data corroborated the general slowing hypothesis for expectancy, whereas the high presentation rate results did not. The observation of a differential influence of age on the automatic and the expectancy mechanism poses a problem for notions assuming that higher level processes are more vulnerable to advancing age than lower level processes.     

The comparison of interdependent correlations between optimal linear composites

A general procedure is provided for comparing correlation coefficients between optimal linear composites. The procedure allows computationally efficient significance tests on independent or dependent multiple correlations, partial correlations, and canonical correlations, with or without the assumption of multivariate normality. Evidence from some Monte Carlo studies on the effectiveness of the methods is also provided.This research was supported in part by an operating grant (#67-4640) to the first author from the National Sciences and Engineering Research Council of Canada. The authors would also like to acknowledge the helpful comments and encouragement of Alexander Shapiro, Stanley Nash, and Ingram Olkin.     

Describing the elephant: Structure and function in multivariate data

There is a unity underlying the diversity of models for the analysis of multivariate data. Essentially, they constitute a family models, most generally nonlinear, for structural/functional relations between variables drawn from a behavior domain.