A goodness of fit test for the rasch model

The Rasch model is an item analysis model with logistic item characteristic curves of equal slope,i.e. with constant item discriminating powers. The proposed goodness of fit test is based on a comparison between difficulties estimated from different scoregroups and over-all estimates. Based on the within scoregroup estimates and the over-all estimates of item difficulties a conditional likelihood ratio is formed. It is shown that—2 times the logarithm of this ratio isx ²-distributed when the Rasch model is true. The power of the proposed goodness of fit test is discussed for alternative models with logistic item characteristic curves, but unequal discriminating items from a scholastic aptitude test.     

In Spite of Indeterminacy Many Common Factor Score Estimates Yield an Identical Reproduced Covariance Matrix

It was investigated whether commonly used factor score estimates lead to the same reproduced covariance matrix of observed variables. This was achieved by means of Schönemann and Steiger’s (1976) regression component analysis, since it is possible to compute the reproduced covariance matrices of the regression components corresponding to different factor score estimates. It was shown that Thurstone’s, Ledermann’s, Bartlett’s, Anderson-Rubin’s, McDonald’s, Krijnen, Wansbeek, and Ten Berge’s, as well as Takeuchi, Yanai, and Mukherjee’s score estimates reproduce the same covariance matrix. In contrast, Harman’s ideal variables score estimates lead to a different reproduced covariance matrix.     

Why we don’t deserve credit for everything we know

A view of knowledge—what I call the Deserving Credit View of Knowledge(DCVK)—found in much of the recent epistemological literature, particularly among so-called virtue epistemologists, centres around the thesis that knowledge is something for which a subject deserves credit. Indeed, this is said to be the central difference between those true beliefs that qualify as knowledge and those that are true merely by luck—the former, unlike the latter, are achievements of the subject and are thereby creditable to her. Moreover, it is often further noted that deserving credit is what explains the additional value that knowledge has over merely lucky true belief. In this paper, I argue that the general conception of knowledge found in the DCVK is fundamentally incorrect. In particular, I show that deserving credit cannot be what distinguishes knowledge from merely lucky true belief since knowledge is not something for which a subject always deserves credit.     

Regression among factor scores

Structural equation models with latent variables are sometimes estimated using an intuitive three-step approach, here denoted factor score regression. Consider a structural equation model composed of an explanatory latent variable and a response latent variable related by a structural parameter of scientific interest. In this simple example estimation of the structural parameter proceeds as follows: First, common factor models areseparately estimated for each latent variable. Second, factor scores areseparately assigned to each latent variable, based on the estimates. Third, ordinary linear regression analysis is performed among the factor scores producing an estimate for the structural parameter. We investigate the asymptotic and finite sample performance of different factor score regression methods for structural equation models with latent variables. It is demonstrated that the conventional approach to factor score regression performs very badly. Revised factor score regression, using Regression factor scores for the explanatory latent variables and Bartlett scores for the response latent variables, produces consistent estimators for all parameters.     

Approaches to univocal factor scores

Two current methods of deriving common-factor scores from tests are briefly examined and rejected. One of these estimates a score from a multiple-regression equation with as many terms as there are tests in the battery. The other limits the equation to a few tests heavily saturated with the desired factor, with or without tests used to suppress the undesired factors. In the proposed methods, the single best test for each common factor is the starting point. Such a test ordinarily has a very few undesired factors to be suppressed, frequently only one. The suppression test should be univocal, or nearly so. Fortunately, there are relatively univocal tests for factors that commonly require suppression. Equations are offered by which the desired-factor test and a single suppression test can be weighted in order to achieve one or more objectives. Among the objectives are (1) maximizing the desired factor variance, (2) minimizing the undesired factor variance, (3) a compromise, in which the undesired variance is materially reduced without loss in desired variance, and (4) a change to any selected ratio of desired to undesired variance. A more generalized solution is also suggested. The methods can be extended in part to the suppression of more than one factor. Equations are derived for the suppression of two factors.     

Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches

Multiple-baseline studies are prevalent in behavioral research, but questions remain about how to best analyze the resulting data. Monte Carlo methods were used to examine the utility of multilevel models for multiplebaseline data under conditions that varied in the number of participants, number of repeated observations per participant, variance in baseline levels, variance in treatment effects, and amount of autocorrelation in the Level 1 errors. Interval estimates of the average treatment effect were examined for two specifications of the Level 1 error structure (σ² I and first-order autoregressive) and for five different methods of estimating the degrees of freedom (containment, residual, between—within, Satterthwaite, and Kenward—Roger). When the Satterthwaite or Kenward—Roger method was used and an autoregressive Level 1 error structure was specified, the interval estimates of the average treatment effect were relatively accurate. Conversely, the interval estimates of the treatment effect variance were inaccurate, and the corresponding point estimates were biased.     

Bayesian inference and the classical test theory model: Reliability and true scores

A general one-way analysis of variance components with unequal replication numbers is used to provide unbiased estimates of the true and error score variance of classical test theory. The inadequacy of the ANOVA theory is noted and the foundations for a Bayesian approach are detailed. The choice of prior distribution is discussed and a justification for the Tiao-Tan prior is found in the particular context of the “n-split” technique. The posterior distributions of reliability, error score variance, observed score variance and true score variance are presented with some extensions of the original work of Tiao and Tan. Special attention is given to simple approximations that are available in important cases and also to the problems that arise when the ANOVA estimate of true score variance is negative. Bayesian methods derived by Box and Tiao and by Lindley are studied numerically in relation to the problem of estimating true score. Each is found to be useful and the advantages and disadvantages of each are discussed and related to the classical test-theoretic methods. Finally, some general relationships between Bayesian inference and classical test theory are discussed. Supported in part by the National Institute of Child Health and Human Development under Research Grant 1 PO1 HDO1762. Reproduction, translation, use or disposal by or for the United States Government is permitted.     

Generating correlation matrices with model error for simulation studies in factor analysis: A combination of the Tucker-Koopman-Linn model and Wijsman’s algorithm

Most simulation studies in factor analysis follow a process of constructing population correlation matrices from the common-factor model and generating sample correlation matrices from the population matrices. In the common-factor model, the population correlation matrix is perfectly fit by the model’s containing common and unique factors. However, since no mathematical model accounts exactly for the real-world phenomena that it is intended to represent, the Tucker-Koopman-Linn model (1969) is more realistic for generating correlation matrices than the conventional common-factor model because the former incorporates model error. In this paper, a procedure for generating population and sample correlation matrices with model error by combining the Tucker-Koopman-Linn model and Wijsman’s algorithm (1959) is presented. The SAS/ IML program for generating correlation matrices is described, and an example is also provided.     

Relations of factor score estimates to their use

Considerations of factor score estimates have concentrated on internal characteristics. This report considers external characteristics of four methods for determining factor score estimates; that is, relations of these estimates to measures on attributes not entered into the factor analysis. These external characteristics are important for many uses of factor score estimates. Findings are that different ones of the methods are appropriate for different uses.Supported in part by the Personnel and Training Branch of the Office of Naval Research under contract number 00014-67-A-0305-0003.     

Modality and domain specific components in auditory and visual working memory tasks

In the tripartite model of working memory (WM) it is postulated that a unique part system—the visuo-spatial sketchpad (VSSP)—processes non-verbal content. Due to behavioral and neurophysiological findings, the VSSP was later subdivided into visual object and visual spatial processing, the former representing objects’ appearance and the latter spatial information. This distinction is well supported. However, a challenge to this model is the question how spatial information from non-visual sensory modalities, for example the auditory one, is processed. Only a few studies so far have directly compared visual and auditory spatial WM. They suggest that the distinction of two processing domains—one for object and one for spatial information—also holds true for auditory WM, but that only a part of the processes is modality specific. We propose that processing in the object domain (the item’s appearance) is modality specific, while spatial WM as well as object-location binding relies on modality general processes.     

Virtue and Disagreement

One of the most prominent strands in contemporary work on the virtues consists in the attempt to develop a distinctive—and compelling—account of practical reason on the basis of Aristotle’s ethics. In response to this project, several eminent critics have argued that the Aristotelian account encourages a dismissive attitude toward moral disagreement. Given the importance of developing a mature response to disagreement, the criticism is devastating if true. I examine this line of criticism closely, first elucidating the features of the Aristotelian account that motivate it, and then identifying two further features of the account that the criticism overlooks. These further features show the criticism to be entirely unwarranted. Once these features are acknowledged, a more promising line of criticism suggests itself—namely, that the Aristotelian account does too little to help us to resolve disputes—but that line of objection will have to be carried out on quite different grounds.     

Common and distinctive features in stimulus similarity: A modified version of the contrast model

Featural representations of similarity data assume that people represent stimuli in terms of a set of discrete properties. In this article, we consider the differences in featural representations that arise from making four different assumptions about how similarity is measured. Three of these similarity models— the common features model, the distinctive features model, and Tversky’s seminal contrast model—have been considered previously. The other model is new and modifies the contrast model by assuming that each individual feature only ever acts as a common or distinctive feature. Each of the four models is tested on previously examined similarity data, relating to kinship terms, and on a new data set, relating to faces. In fitting the models, we have used the geometric complexity criterion to balance the competing demands of data-fit and model complexity. The results show that both common and distinctive features are important for stimulus representation, and we argue that the modified contrast model combines these two components in a more effective and interpretable way than Tversky’s original formulation.     

In defence of virtue epistemology

In a number of recent papers Duncan Pritchard argues that virtue epistemology’s central ability condition—one knows that p if and only if one has attained cognitive success (true belief) because of the exercise of intellectual ability—is neither necessary nor sufficient for knowledge. This paper discusses and dismisses a number of responses to Pritchard’s objections and develops a new way of defending virtue epistemology against them.     

How to fit a response time distribution

Among the most valuable tools in behavioral science is statistically fitting mathematical models of cognition to data—response time distributions, in particular. However, techniques for fitting distributions vary widely, and little is known about the efficacy of different techniques. In this article, we assess several fitting techniques by simulating six widely cited models of response time and using the fitting procedures to recover model parameters. The techniques include the maximization of likelihood and least squares fits of the theoretical distributions to different empirical estimates of the simulated distributions. A running example is used to illustrate the different estimation and fitting procedures. The simulation studies reveal that empirical density estimates are biased even for very large sample sizes. Some fitting techniques yield more accurate and less variable parameter estimates than do others. Methods that involve least squares fits to density estimates generally yield very poor parameter estimates.     

Changes in common-factor loadings as tests are altered homogeneously in length

Formulas are derived by which, given the factor loadings and the internal reliability of a test of unit length, the following estimates can be made: (1) the common-factor loadings for a similar (homogeneous) test of lengthn; (2) the number of times (n) that a test needs to be lengthened homogeneously to achieve a factor loading of a desired magnitude; and (3) the correlation between two tests, either or both of which have been altered in length, as a function of (a) the new factor loadings in the altered tests or (b) the original loadings in the unit-length tests. The appropriate use of the derived formulas depends upon the fulfillment of four assumptions enumerated.This article is based on a paper read by the authors at the annual meeting of the Western Psychological Association in Eugene, Oregon, June 25, 1949.     

Statistical inference based on latent ability estimates

The quality of approximations to first and second order moments (e.g., statistics like means, variances, regression coefficients) based on latent ability estimates is being discussed. The ability estimates are obtained using either the Rasch, or the two-parameter logistic model. Straightforward use of such statistics to make inferences with respect to true latent ability is not recommended, unless we account for the fact that the basic quantities are estimates. In this paper true score theory is used to account for the latter; the counterpart of observed/true score being estimated/true latent ability. It is shown that statistics based on the true score theory are virtually unbiased if the number of items presented to each examinee is larger than fifteen. Three types of estimators are compared: maximum likelihood, weighted maximum likelihood, and Bayes modal. Furthermore, the (dis)advantages of the true score method and direct modeling of latent ability is discussed.     

How True It Is = Who Says It’s True

This is a largely expository paper in which the following simple idea is pursued. Take the truth value of a formula to be the set of agents that accept the formula as true. This means we work with an arbitrary (finite) Boolean algebra as the truth value space. When this is properly formalized, complete modal tableau systems exist, and there are natural versions of bisimulations that behave well from an algebraic point of view. There remain significant problems concerning the proper formalization, in this context, of natural language statements, particularly those involving negative knowledge and common knowledge. A case study is presented which brings these problems to the fore. None of the basic material presented here is new to this paper—all has appeared in several papers over many years, by the present author and by others. Much of the development in the literature is more general than here—we have confined things to the Boolean case for simplicity and clarity. Most proofs are omitted, but several of the examples are new. The main virtue of the present paper is its coherent presentation of a systematic point of view—identify the truth value of a formula with the set of those who say the formula is true.     

Cartwright On Explanation And Idealization

Nancy Cartwright (1983, 1999) argues that (1) the fundamental laws of physics are true when and only when appropriate ceteris paribus modifiers are attached and that (2) ceteris paribus modifiers describe conditions that are almost never satisfied. She concludes that when the fundamental laws of physics are true, they don't apply in the real world, but only in highly idealized counterfactual situations. In this paper, we argue that (1) and (2) together with an assumption about contraposition entail the opposite conclusion — that the fundamental laws of physics do apply in the real world. Cartwright extracts from her thesis about the inapplicability of fundamental laws the conclusion that they cannot figure in covering-law explanations. We construct a different argument for a related conclusion — that forward-directed idealized dynamical laws cannot provide covering-law explanations that are causal. This argument is neutral on whether the assumption about contraposition is true. We then discuss Cartwright's simulacrum account of explanation, which seeks to describe how idealized laws can be explanatory. This revised version was published online in July 2006 with corrections to the Cover Date.     

Beyond statistical inference: A decision theory for science

Traditional null hypothesis significance testing does not yield the probability of the null or its alternative and, therefore, cannot logically ground scientific decisions. The decision theory proposed here calculates the expected utility of an effect on the basis of (1) the probability of replicating it and (2) a utility function on its size. It takes significance tests—which place all value on the replicability of an effect and none on its magnitude—as a special case, one in which the cost of a false positive is revealed to be an order of magnitude greater than the value of a true positive. More realistic utility functions credit both replicability and effect size, integrating them for a single index of merit. The analysis incorporates opportunity cost and is consistent with alternate measures of effect size, such as r2 and information transmission, and with Bayesian model selection criteria. An alternate formulation is functionally equivalent to the formal theory, transparent, and easy to compute.