Reliability of test scores in nonparametric item response theory

Three methods for estimating reliability are studied within the context of nonparametric item response theory. Two were proposed originally by Mokken (1971) and a third is developed in this paper. Using a Monte Carlo strategy, these three estimation methods are compared with four classical lower bounds to reliability. Finally, recommendations are given concerning the use of these estimation methods.The authors are grateful for constructive comments from the reviewers and from Charles Lewis.     

Disruption of proofreading by irrelevant speech: Effects of attention,arousal or memory?

A series of five experiments examined the effects of irrelevant speech on proofreading and memory. Four of the experiments used a proofreading task and showed that the deleterious effects of irrelevant speech: (1) depend on the speech being meaningful, (2) are only present when the burden on short-term memory is low and (3) are manifested in a lower detection rate for non-contextual as opposed to contextual errors. Neither the spatial location of the speech (either in terms of spatial dispersion of sources or spatial movement of a single source) nor the intensity of the speech (in a range bounded by 50 dB(A) and 70 dB(A)) had any effect on proofreading. Late selection models of attention are favoured by the results in preference to models having arousal, short-term memory or early selection in attention as their basis. A final experiment showed serial recall for visual lists to be impaired by the presence of any speech-like sound (including reversed speech and speech in an unfamiliar language) which suggests a set of phenomena qualitatively different from those associated with proofreading. Throughout the article the practical consequences of the findings are emphasized.     

The many null distributions of person fit indices

This paper deals with the situation of an investigator who has collected the scores ofn persons to a set ofk dichotomous items, and wants to investigate whether the answers of all respondents are compatible with the one parameter logistic test model of Rasch. Contrary to the standard analysis of the Rasch model, where all persons are kept in the analysis and badly fittingitems may be removed, this paper studies the alternative model in which a small minority ofpersons has an answer strategy not described by the Rasch model. Such persons are called anomalous or aberrant. From the response vectors consisting ofk symbols each equal to 0 or 1, it is desired to classify each respondent as either anomalous or as conforming to the model. As this model is probabilistic, such a classification will possibly involve false positives and false negatives. Both for the Rasch model and for other item response models, the literature contains several proposals for a person fit index, which expresses for each individual the plausibility that his/her behavior follows the model. The present paper argues that such indices can only provide a satisfactory solution to the classification problem if their statistical distribution is known under the null hypothesis that all persons answer according to the model. This distribution, however, turns out to be rather different for different values of the person's latent trait value. This value will be called ability parameter, although our results are equally valid for Rasch scales measuring other attributes.As the true ability parameter is unknown, one can only use its estimate in order to obtain an estimated person fit value and an estimated null hypothesis distribution. The paper describes three specifications for the latter: assuming that the true ability equals its estimate, integrating across the ability distribution assumed for the population, and conditioning on the total score, which is in the Rasch model the sufficient statistic for the ability parameter.Classification rules for aberrance will be worked out for each of the three specifications. Depending on test length, item parameters and desired accuracy, they are based on the exact distribution, its Monte Carlo estimate and a new and promising approximation based on the moments of the person fit statistic. Results for the likelihood person fit statistic are given in detail, the methods could also be applied to other fit statistics. A comparison of the three specifications results in the recommendation to condition on the total score, as this avoids some problems of interpretation that affect the other two specifications.The authors express their gratitude to the reviewers and to many colleagues for comments on an earlier version.     

Stochastic ordering using the latent trait and the sum score in polytomous IRT models

In a restricted class of item response theory (IRT) models for polytomous items the unweighted total score has monotone likelihood ratio (MLR) in the latent trait. MLR implies two stochastic ordering (SO) properties, denoted SOM and SOL, which are both weaker than MLR, but very useful for measurement with IRT models. Therefore, these SO properties are investigated for a broader class of IRT models for which the MLR property does not hold.In this study, first a taxonomy is given for nonparametric and parametric models for polytomous items based on the hierarchical relationship between the models. Next, it is investigated which models have the MLR property and which have the SO properties. It is shown that all models in the taxonomy possess the SOM property. However, counterexamples illustrate that many models do not, in general, possess the even more useful SOL property.Hemker's research was supported by the Netherlands Research Council, Grant 575-67-034. Junker's research was supported in part by the National Institutes of Health, Grant CA54852, and by the National Science Foundation, Grant DMS-94.04438.     

Organizational factors in the effect of irrelevant speech: The role of spatial location and timing

Typically, hearing a repeated syllable produces minimal disruption of serial recall of visual lists, but a sequence of different syllables impairs performance markedly. Two conditions for presenting anidentical sequence of three syllables are compared: one, in which, by means of stereophony, each syllable is assigned to the left, center, or right auditory locus (three streamsnot changing in state), and another, in which the same syllable sequence occurs in one location only (one streamwith changing state). Disruption was significantly less in the stereophonic than in the monophonic condition. There was a joint effect of changing state and location, not an effect of the number of locations alone. In Experiment 2, temporal predictability was used to manipulate changing state. The disruptive effect ofregular presentation of a repeated syllable was markedly increased when it was presentedirregularly. The results are discussed in the context of organizational factors in short-term memory.     

Dynamic models of development and the mechanistic-organismic controversy

Starting with a prototypical model of Newtonian mechanics, a sequence of dynamic models with increasing structural complexity is presented. The implicated smooth variation along a dimension of structural complexity yields dynamic models which display organismic properties such as stagewise development and emergent self-organization. Consequently, it is concluded that dynamic models of development are inherently neutral with respect to the mechanism-organicism controversy. Accordingly, a unitary research program is proposed in which these dynamic models serve as building blocks for the construction of a coherent explanatory system of development.     

Perceived control in continuous loud noise

The effects of two types of perceived control were assessed for performance during and following exposure to continuous loud noise at 90 dBC. Subjects performed a tracking task during exposure and the Stroop test in the period following exposure. Control could be effected either (1) by attenuating the intensity of the noise or (2) by reducing the difficulty of the tracking task. In both cases subjects were encouraged not to exercise control. Performance in treatments involving loud (90 dBC) and soft (55 dBC) noise served as baseline conditions. Perceived control of task difficulty accentuated the tendency for root-mean-square error to rise during the tracking task. There were no significant effects of noise intensi ty on tracking performance. Loud noise reduced the incidence of errors in the Stroop test but perceived control failed to influence this effect. The results are discussed in terms of the generality of perceived control phenomena.     

Modeling Asymmetry in the Time–Distance Relation of Ordinal Personality Items

In analyzing responses and response times to personality questionnaire items, models have been proposed which include the so-called “inverted-U effect.” These models predict that response times to personality test items decrease as the latent trait value of a given person gets closer to the attractiveness of an item. Initial studies into these models have focused on dichotomous personality items, and more recently, models for Likert-type scale items have been proposed. In all these models, it is assumed that the inverted-U effect is symmetrical around 0, while, as will be explained in this article, there are substantive and statistical reasons to study this assumption. Therefore, in this article, a general inverted-U model is proposed which accommodates two sources of asymmetry between the response times and the attractiveness of the items. The viability of this model is demonstrated in a simulation study, and the model is applied to the responses and response times of the Temperament and Character Inventory–Revised, covering a broad range of personality dimensions.     

An Ethical Framework for the Design,Development, Implementation,and Assessment of Drones Used in Public Healthcare

Science and Engineering Ethics - The use of drones in public healthcare is suggested as a means to improve efficiency under constrained resources and personnel. This paper begins by framing drones...