A probabilistic threshold model: Analyzing semantic categorization data with the Rasch model

According to the Threshold Theory (Hampton, 1995, 2007) semantic categorization decisions come about through the placement of a threshold criterion along a dimension that represents items' similarity to the category representation. The adequacy of this theory is assessed by applying a formalization of the theory, known as the Rasch model (Rasch, 1960; Thissen & Steinberg, 1986), to categorization data for eight natural language categories and subjecting it to a formal test. In validating the model special care is given to its ability to account for inter- and intra-individual differences in categorization and their relationship with item typicality. Extensions of the Rasch model that can be used to uncover the nature of category representations and the sources of categorization differences are discussed.     

Fitting Rasch model using appropriateness measure statistics

In this paper, the distributional properties and power rates of the Lz, Eci2z, and Eci4z statistics when they are used as item fit statistics were explored. The results were compared to t-transformation of Outfit and Infit mean square. Four sample sizes were selected: 100, 250, 500, and 1000 examinees. The abilities were uniform and normal with mean 0 and standard deviation 1, and uniform and normal with mean -1 and standard deviation 1. The pseudo-guessing parameter was fixed at .25. Two ranges of difficulty parameters were selected: +/- 1 logits and +/- 2 logits. Two test lengths were selected: 15 and 30 items. The results showed important differences between the T-infit, T-outfit, Lz, Eci2z, and Eci4z statistics. The T-oufit, T-infit, and Lz statistics showed poor standardization with estimated parameters because their distributional properties were not close to the expected values. However, the Eci2z and Eci4z statistics showed satisfactory standardization on all conditions. Further, the power rates of Eci2z and Eci4z were 5% to 10% higher than the power rates of Lz, T-outfit, and T-infit to detect items that do not fit Rasch model.     

Prejudiced attitude measurement using the Rasch Rating Scale model

There have been two basic approaches for the study of minority group prejudice against the majority: to adapt instruments from the majority group, and to use qualitative techniques by analyzing the content of the discourse of the groups involved. Neither of these procedures solves the problem of measuring intergroup attitudes of majorities and minorities in interaction. This study shows the result of a prejudice scale which was developed to measure the attitude of both the minority and majority groups. Prejudice is conceived as an attitude which requires the beliefs or opinions about the out-group, the emotions it elicits, and the behavior or intentional behavior toward it to be known for its evaluation. The innovation in this work is that the psychometric development of the scale was based on the item response theory, and more specifically, the rating scale model.     

Analyzing Job Analysis Data Using Mixture Rasch Models

An important piece of validity evidence to support the use of credentialing exams comes from performing a job analysis of the profession. One common job analysis method is the task inventory method, where people working in the field are surveyed using rating scales about the tasks thought necessary to safely and competently perform the job. This article describes how mixture Rasch models can be used to analyze these data, and how results from these analyses can help to identify whether different groups of people may be responding to job tasks differently. Three examples from different credentialing programs illustrate scenarios that can be found when applying mixture Rasch models to job analysis data. Discussion of what these results may imply for the development of credentialing exams and other analyses of job analysis data is provided.     

Loglinear Rasch model tests

Existing statistical tests for the fit of the Rasch model have been criticized, because they are only sensitive to specific violations of its assumptions. Contingency table methods using loglinear models have been used to test various psychometric models. In this paper, the assumptions of the Rasch model are discussed and the Rasch model is reformulated as a quasi-independence model. The model is a quasi-loglinear model for the incomplete subgroup × score × item 1 × item 2 × ... × itemk contingency table. Using ordinary contingency table methods the Rasch model can be tested generally or against less restrictive quasi-loglinear models to investigate specific violations of its assumptions.     

A dynamic generalization of the Rasch model

In the present paper a model for describing dynamic processes is constructed by combining the common Rasch model with the concept of structurally incomplete designs. This is accomplished by mapping each item on a collection of virtual items, one of which is assumed to be presented to the respondent dependent on the preceding responses and/or the feedback obtained. It is shown that, in the case of subject control, no unique conditional maximum likelihood (CML) estimates exist, whereas marginal maximum likelihood (MML) proves a suitable estimation procedure. A hierarchical family of dynamic models is presented, and it is shown how to test special cases against more general ones. Furthermore, it is shown that the model presented is a generalization of a class of mathematical learning models, known as Luce's beta-model.     

An analysis of the Revised Olweus Bully/Victim Questionnaire using the Rasch measurement model

Background. Bullying is a problem in schools in many countries. There would be a benefit in the availability of a psychometrically sound instrument for its measurement, for use by teachers and researchers. The Olweus Bully/Victim Questionnaire has been used in a number of studies but comprehensive evidence on its validity is not available. Aims. To examine the conceptual design, construct validity and reliability of the Revised Olweus Bully/Victim Questionnaire (OBVQ) and to provide further evidence on the prevalence of different forms of bullying behaviour. Sample. All 335 pupils (160 [47.8%] girls; 175 [52.2%]) boys, mean age 11.9 years [range 11.2–12.8 years]), in 21 classes of a stratified sample of 7 Greek Cypriot primary schools. Method. The OBVQ was administered to the sample. Separate scales were created comprising (a) the items of the questionnaire concerning the extent to which pupils are being victimized; and (b) those concerning the extent to which pupils express bullying behaviour. Using the Rasch model, both scales were analysed for reliability, fit to the model, meaning, and validity. Both scales were also analysed separately for each of two sample groups (i.e. boys and girls) to test their invariance. Results. Analysis of the data revealed that the instrument has satisfactory psychometric properties; namely, construct validity and reliability. The conceptual design of the instrument was also confirmed. The analysis leads also to suggestions for improving the targeting of items against student measures. Support was also provided for the relative prevalence of verbal, indirect and physical bullying. As in other countries, Cypriot boys used and experienced more bullying than girls, and boys used more physical and less indirect forms of bullying than girls. Conclusions. The OBVQ is a psychometrically sound instrument that measures two separate aspects of bullying, and whose use is supported for international studies of bullying in different countries. However, improvements to the questionnaire were also identified to provide increased usefulness to teachers tackling this significant problem facing schools in many countries.     

Fitting the ratcliff diffusion model to experimental data

Many experiments in psychology yield both reaction time and accuracy data. However, no off-the-shelf methods yet exist for the statistical analysis of such data. One particularly successful model has been the diffusion process, but using it is difficult in practice because of numerical, statistical, and software problems. We present a general method for performing diffusion model analyses on experimental data. By implementing design matrices, a wide range of across-condition restrictions can be imposed on model parameters, in a flexible way. It becomes possible to fit models with parameters regressed onto predictors. Moreover, data analytical tools are discussed that can be used to handle various types of outliers and contaminants. We briefly present an easy-touse software tool that helps perform diffusion model analyses.     

Assessing the age-related effects of proactive interference on working memory tasks using the Rasch model

Inhibition-reduction theory (L. Hasher & R. Zacks, 1988) hypothesizes that the age-related decline in working memory (WM) span is a result of a decrease in the ability to inhibit irrelevant information in WM. Using the Rasch psychometric model, this study found that later trials on 2 WM span tasks were more difficult for older adults than for younger adults, consistent with inhibition-reduction theory's hypothesis that older adults are more susceptible to the effects of proactive interference (PI). Furthermore, after accounting for differential susceptibility to the effects of PI, age-related variance in WM span was reduced by about half. These results suggest that differential susceptibility to PI may account for a substantial portion, although not all, of the age-related decline in WM span.     

Robust estimation of ability in the Rasch model

Estimating ability parameters in latent trait models in general, and in the Rasch model in particular is almost always hampered by noise in the data. This noise can be caused by guessing, inattention to easy questions, and other factors which are unrelated to ability. In this study several alternative formulations which attempt to deal with these problems without a reparameterization are tested through a Monte Carlo simulation. It was found that although no one of the tested schemes is uniformly superior to all others, a modified jackknife stood out as the best one in general, it was also super efficient (more efficient than the asymptotically optimal estimator) for tests with forty or fewer items. It is proposed that this sort of jackknifing scheme for estimating ability be considered for practical work.This research was funded through a grant from the Law Enforcement Assistance Administration (78-NI-AX-0047) to the Bureau of Social Science Research, Howard Wainer, Principal Investigator. We would like to thank Ronald Mead, Anne Morgan and James Ramsay for kind, generous, and invaluable help at various stages of the project.     

Testing the assumptions and interpreting the results of the Rasch model using log-linear procedures in SPSS

This paper shows how to use the log-linear subroutine of SPSS to fit the Rasch model. It also shows how to fit less restrictive models obtained by relaxing specific assumptions of the Rasch model. Conditional maximum likelihood estimation was achieved by including dummy variables for the total scores as covariates in the models. This approach greatly simplifies the specification of the Rasch models. We illustrate these procedures in an analysis of four items selected from the Reiss Premarital Sexual Permissiveness Scale. We found that a modified version of the Rasch model with item dependencies fits the data significantly better than the simple Rasch model. We also found that the item difficulties are the same for men and women, but that the item dependencies are significantly greater for men. Apart from any substantive issues these results raise, the value of this exercise lies in its demonstration of how researchers can use the procedures of popular, accessible software packages to study an increasingly important set of measurement models.     

Construction of a depression scale with the Rasch model

This study examines the psychometric properties of a depression questionnaire. The goal was to improve the technical quality of traditional measures of depression in Spanish youth. 310 participants, aged 18-24 years, filled in the self-report questionnaire. The data were analyzed by means of the Rasch model. Results show that model fit, average item reliability (.97), and average person reliability (.88) are high. After deleting four indicators showing misfit and 12 showing sex bias, the resulting scale measures clinical depression objectively. Using this scale, the expected sex-related differences are found.     

Some improved diagnostics for failure of the Rasch model

Although several goodness of fit tests have been developed for the Rasch model for dichotomous items, most of them are of a global, asymptotic, and confirmatory type. This paper, based on ideas from a recent thesis by Van den Wollenberg, offers some suggestions for local, small sample, and exploratory techniques: difficulty plots for person groups scoring right and wrong on a specific item, a slope test per item based on a binomial distribution per score group, and a unidimensionality check based on an extended hypergeometric distribution per score group. This paper owes much to the inspiring and pioneering work of Arnold Van den Wollenberg, of which only minor aspects are criticized. Thanks go to Charles Lewis for stimulating discussions and for solutions to some programming problems.     

Semi-automated Rasch analysis using in-plus-out-of-questionnaire log likelihood

Rasch analysis is a popular statistical tool for developing and validating instruments that aim to measure human performance, attitudes and perceptions. Despite the availability of various software packages, constructing a good instrument based on Rasch analysis is still considered to be a complex, labour-intensive task, requiring human expertise and rather subjective judgements along the way. In this paper we propose a semi-automated method for Rasch analysis based on first principles that reduces the need for human input. To this end, we introduce a novel criterion, called in-plus-out-of-questionnaire log likelihood (IPOQ-LL). On artificial data sets, we confirm that optimization of IPOQ-LL leads to the desired behaviour in the case of multi-dimensional and inhomogeneous surveys. On three publicly available real-world data sets, our method leads to instruments that are, for all practical purposes, indistinguishable from those obtained by Rasch analysis experts through a manual procedure.     

Testing the Rasch model by means of the mixture fit index

Rudas, Clogg, and Lindsay (RCL) proposed a new index of fit for contingency table analysis. Using the overparametrized two‐component mixture, where the first component with weight 1?w represents the model to be tested and the second component with weight w is unstructured, the mixture index of fit was defined to be the smallest w compatible with the saturated two‐component mixture. This index of fit, which is insensitive to sample size, is applied to the problem of assessing the fit of the Rasch model. In this application, use is made of the equivalence of the semi‐parametric version of the Rasch model to specifically restricted latent class models. Therefore, the Rasch model can be represented by the structured component of the RCL mixture, with this component itself consisting of two or more subcomponents corresponding to the classes, and the unstructured component capturing the discrepancies between the data and the model. An empirical example demonstrates the application of this approach. Based on four‐item data, the one‐ and two‐class unrestricted latent class models and the one‐ to three‐class models restricted according to the Rasch model are considered, with respect to both their chi‐squared statistics and their mixture fit indices.     

A mixed Rasch model of dual-process conditional reasoning

A fine-grained dual-process approach to conditional reasoning is advocated: Responses to conditional syllogisms are reached through the operation of either one of two systems, each of which can rely on two different mechanisms. System1 relies either on pragmatic implicatures or on the retrieval of information from semantic memory; System2 operates first through inhibition of System1, then (but not always) through activation of analytical processes. It follows that reasoners will fall into one of four groups of increasing reasoning ability, each group being uniquely characterized by (a) the modal pattern of individual answers to blocks of affirming the consequent (AC), denying the antecedent (DA), and modus tollens (MT) syllogisms featuring the same conditional; and (b) the average rate of determinate answers to AC, DA, and MT. This account receives indirect support from the extant literature and direct support from a mixed Rasch model of responses given to 18 syllogisms by 486 adult reasoners.     

D-optimal design for the Rasch counts model with multiple binary predictors

In this paper we derive optimal designs for the Rasch Poisson counts model and its extended version of the (generalized) negative binomial counts model incorporating several binary predictors for the difficulty parameter. To efficiently estimate the regression coefficients of the predictors, locally D-optimal designs are developed. After an introduction to the Rasch Poisson counts model and its extension, we will specify these models as particular generalized linear models. Based on this embedding, optimal designs for both models including several binary explanatory variables will be presented. Therefore, we will derive conditions on the effect sizes for certain designs to be locally D-optimal. Finally, it is pointed out that the results derived for the Rasch Poisson models can be applied for more general Poisson regression models which should receive more attention in future psychological research.     

Using the Rasch model to quantify the causal effect of test instructions

This experimental study was designed to quantify, by means of the Rasch model (RM), the effects of three instruction/scoring conditions on student measures and on the reliability of an achievement multiple-choice test in a field context. Examinees performed the test in one of three conditions which differed only in the instructions provided. Predictions regarding performance indicators were fulfilled, and the expected differences in reliability favoring omission-inducing instructions did appear. This difference in reliability was found for both Rasch and raw data and thus it can be concluded that the fact that results from previous studies failed to corroborate this prediction must have been due to the lack of important consequences of test scores for the students. The RM has served to neatly quantify the differences between instructions promoting guessing and instructions promoting omission under uncertainty, showing that the recommendation to omit is not only educationally but also psychometrically sound.     

Parameter recovery and model selection in mixed Rasch models

This study examines the precision of conditional maximum likelihood estimates and the quality of model selection methods based on information criteria (AIC and BIC) in mixed Rasch models. The design of the Monte Carlo simulation study included four test lengths (10, 15, 25, 40), three sample sizes (500, 1000, 2500), two simulated mixture conditions (one and two groups), and population homogeneity (equally sized subgroups) or heterogeneity (one subgroup three times larger than the other). The results show that both increasing sample size and increasing number of items lead to higher accuracy; medium-range parameters were estimated more precisely than extreme ones; and the accuracy was higher in homogeneous populations. The minimum-BIC method leads to almost perfect results and is more reliable than AIC-based model selection. The results are compared to findings by Li, Cohen, Kim, and Cho (2009) and practical guidelines are provided.