A nonparametric approach for assessing latent trait unidimensionality

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.The referees' comments were remarkably detailed and greatly enhanced the writeup and sensitized the author to certain pertinent issues. Discussions with Fritz Drasgow, Lloyd Humphreys, Dennis Jennings, Brian Junker, Robert Linn, Ratna Nandakumar, and Robin Shealy were also very useful.This research was supported by the Office of Naval Research under grant N00014-84-K-0186; NR 150-533, and by the National Science Foundation under grant DMS 85-03321.     

Reliability of test scores in nonparametric item response theory

Three methods for estimating reliability are studied within the context of nonparametric item response theory. Two were proposed originally by Mokken (1971) and a third is developed in this paper. Using a Monte Carlo strategy, these three estimation methods are compared with four classical lower bounds to reliability. Finally, recommendations are given concerning the use of these estimation methods.The authors are grateful for constructive comments from the reviewers and from Charles Lewis.     

Superimposition of response-independent reinforcement

Studies that have superimposed response-independent reinforcement (or reinforcers scheduled by contingencies placed on the absence of responding) upon conventional response-dependent schedules are reviewed. In general, providing alternative sources of reinforcement reduced response rates below the levels observed when alternative reinforcement was absent. However, response-rate elevation was sometimes found, particularly when rates of superimposed response-independent reinforcement were low. Superimposition of schedules providing reinforcers contingent on the absence of responding usually produced more severe response-rate decrements than superimposition of response-independent reinforcement. A variant of Herrnstein's equation, which assumes that some of the alternative reinforcers function as if they were delivered by baseline response-dependent source of reinforcement, is in qualitative agreement with the overall body of results obtained, and can predict both increases and decreases in response rate as resulting from superimposed reinforcers.     

The effect of negative stimulus presentations on observing-response rates

Theories of observing differ in predicting whether or not a signal for absence of reinforcement (S−) is capable of reinforcing observing responses. Experiments in which S− was first removed from and then restored to the procedure have yielded mixed results. The present experiments suggest that failure to control for the direct effect of presenting S− may have been responsible. Pigeons and operant procedures were used. Experiment 1 showed that presentations of S−, even when not contingent on observing, can raise the rate of an observing response that was reinforced only by presentations of a signal (S+) that accompanied a schedule of food delivery. Experiment 2 showed that this effect resulted from bursts of responding that followed offsets of S−. Experiment 3 showed that, when the presence of S− was held constant, lower rates occurred when S− was dependent on, rather than independent of, observing. These results support theories that characterize S− as incapable of reinforcing observing responses.     

Units of analysis and kinetic structure of behavioral repertoires

It is suggested that molar streams of behavior are constructed of various arrangements of three elementary constituents (elicited, evoked, and emitted response classes). An eight-cell taxonomy is elaborated as a framework for analyzing and synthesizing complex behavioral repertoires based on these functional units. It is proposed that the local force binding functional units into a smoothly articulated kinetic sequence arises from temporally arranged relative response probability relationships. Behavioral integration is thought to reflect the joint influence of the organism's hierarchy of relative response probabilities, fluctuating biological states, and the arrangement of environmental and behavioral events in time.     

Latent trait models and dichotomization of graded responses

This paper discusses thecompatibility of the polychotomous Rasch model with dichotomization of the response continuum. It is argued that in the case of graded responses, the response categories presented to the subject are essentially an arbitrary polychotomization of the response continuum, ranging for example from total rejection or disagreement to total acceptance or agreement of an item or statement. Because of this arbitrariness, the measurement outcome should be independent of the specific polychotomization applied, for example, presenting a specific multicategory response format should not affect the measurement outcome. When such is the case, the original polychotomous model is called compatible with dichotomization.A distinction is made between polychotomization or dichotomization before the fact, that is, in constructing the response format, and polycho- or dichotomization after the fact, for example in dichotomizing existing graded response data.It is shown that, at least in case of dichotomization after-the-fact, the polychotomous Rasch model is not compatible with dichotomization, unless a rather special condition of the model parameters is met. Insofar as it may be argued that dichotomization before the fact is not essentially different from dichotomization after the fact, the value of the unidimensional polychotomous Rasch model is consequently questionable. The impact of our conclusion on related models is also discussed.     

A taxonomy of item response models

A number of models for categorical item response data have been proposed in recent years. The models appear to be quite different. However, they may usefully be organized as members of only three distinct classes, within which the models are distinguished only by assumptions and constraints on their parameters. “Difference models” are appropriate for ordered responses, “divide-by-total” models may be used for either ordered or nominal responses, and “left-side added” models are used for multiple-choice responses with guessing. The details of the taxonomy and the models are described in this paper. The present study was supported in part by two postdoctoral fellowships awarded to Lynne Steinberg: an Educational Testing Service Postdoctoral Fellowship at ETS, Princeton, NJ and an NIMH Individual National Research Service Award at Stanford University, Stanford, CA. Helpful comments by the editor and three anonymous reviewers are gratefully acknowledged.     

Self-selection biases in correlational studies based on questionnaires

It is commonly held that even where questionnaire response is poor, correlational studies are affected only by loss of degrees of freedom or precision. We show that this supposition is not true. If the decision to respond is correlated with a substantive variable of interest, then regression or analysis of variance methods based upon the questionnaire results may be adversely affected by self-selection bias. Moreover such bias may arise even where response is 100%. The problem in both cases arises where selection information is passed to the score indirectly via the disturbance or individual effects, rather than entirely via the observable explanatory variables. We suggest tests for the ensuing self-selection bias and possible ways of handling the ensuing problems of inference.The author would like to thank members of the Psychology Department at UWA for references on the illustrative example, namely the effects of sex-stereotyping on achievements in mathematics; thanks are also due to the anonymous reviewers whose comments resulted in material improvements to the paper.     

Psychopathology subtypes and symptom correlates among former prisoners of war

Psychopathology and symptom patterns were studied in 60 former prisoners-of-war (POWs) by administering standardized tests including the Minnesota Multiphasic Personality Inventory (MMPI), an adjustment problem checklist, and a structured clinical interview. Most POWs showed marked psychological impairment, but modal profile analysis identified two prototypic MMPI patterns, which differed in pervasiveness and type of psychopathology. Profile subtypes were defined by unique clusters of clinical symptoms and differed in confinement stress severity. The typology of symptoms argues against a homogeneous conceptualization of stress-induced disorders and suggests the need for definition of the severity and subtype of stress phenomena and individual difference factors in responding to trauma.     

Concept learning,memory and transfer of dot pattern categories

In the first experiment, subjects rated the similarity of a sample of pairs of random dot patterns generated from 3 prototypes consisting of either 4, 6, or 9 dots. The results of the experiment tended to confirm the structure imposed on the patterns by the construction method. In the second experiment, subjects learned to classify random patterns generated from 2 prototypes with either short or long interprototype distance and consisting of either 4, 6, or 9 dots in a traditional concept identification task. After acquisition, memory for old and new distortions and the prototype, which was not presented during learning, was assessed. The tendency to call a pattern “new” increased with the distance between the pattern and its prototype. In a subsequent transfer test, all subjects saw patterns of either 4, 6, or 9 dots. Performance was above chance and the transfer from larger to smaller patterns tended to be greater than the transfer from smaller to larger patterns. This is consistent with a process of schema formation based on features common to most exemplars of the category. The results are discussed in relation to several theories of concept learning and schema abstraction.