Ordinal consistency and ordinal true scores

This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.Part of this work was done while the author was a Visiting Fellow at Macquarrie University. The paper has benefitted from discussions with Professors Thomas J. Reynolds and Roderick P. McDonald and from the comments of several anonymous reviewers.     

Criterion-related construct validity

Established results on latent variable models are applied to the study of the validity of a psychological test. When the test predicts a criterion by measuring a unidimensional latent construct, not only must the total score predict the criterion, but the joint distribution of criterion scores and item responses must exhibit a certain pattern. The presence of this population pattern may be tested with sample data using the stratified Wilcoxon rank sum test. Often, criterion information is available only for selected examinees, for instance, those who are admitted or hired. Three cases are discussed: (i) selection at random, (ii) selection based on the current test, and (iii) selection based on other measures of the latent construct. Discriminant validity is also discussed.This work was supported in part by Grant SES-87-01890 from the Measurement Methods and Data Improvement Program of the U.S. National Science Foundation.     

Sequential effects in dimensional contrast

Two experiments examined pigeons' response rates during short trials signaled by stimuli closely spaced along a wavelength continuum. In Experiment 1 separate halves of the continuum were correlated with different reinforcement schedules. In Experiment 2, the middle stimulus was accompanied by a lower probability of reinforcement than were the remaining wavelengths. Both procedures resulted in dimensional contrast “shoulders,” seen as relatively enhanced or depressed response rates in the presence of stimuli between the extreme of the continuum and the border separating the positive and negative stimuli. Sequential analyses addressed possible contributions of the following interactions: (a) local contrast, seen when rate during a given schedule depends on the schedule in the just-preceding trial; (b) modification of local contrast by the similarity of the signaling stimuli (P. Blough, 1983); and (c) schedule-independent rate contrast, seen when rate in a given trial depends on the rate controlled by the stimulus that accompanied the just-preceding trial (Malone & Rowe, 1981). Dimensional contrast functions were similar when isolated according to the schedule, to the similarity of the signaling stimulus, and to the response rate of the just-preceding trial. The interactions noted above do not appear to make important contributions to this effect.     

A nonparametric approach for assessing latent trait unidimensionality

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.The referees' comments were remarkably detailed and greatly enhanced the writeup and sensitized the author to certain pertinent issues. Discussions with Fritz Drasgow, Lloyd Humphreys, Dennis Jennings, Brian Junker, Robert Linn, Ratna Nandakumar, and Robin Shealy were also very useful.This research was supported by the Office of Naval Research under grant N00014-84-K-0186; NR 150-533, and by the National Science Foundation under grant DMS 85-03321.     

Transfer effects from language processing to visual attention dynamics: The impact of orthographic transparency

The consistency between letters and sounds varies across languages. These differences have been proposed to be associated with different reading mechanisms (lexical vs. phonological), processing grain sizes (coarse vs. fine) and attentional windows (whole words vs. individual letters). This study aimed to extend this idea to writing to dictation. For that purpose, we evaluated whether the use of different types of processing has a differential impact on local windowing attention: phonological (local) processing in a transparent language (Spanish) and lexical (global) processing of an opaque language (English). Spanish and English monolinguals (Experiment 1) and Spanish–English bilinguals (Experiment 2) performed a writing to dictation task followed by a global–local task. The first key performance showed a critical dissociation between languages: the response times (RTs) from the Spanish writing to dictation task was modulated by word length, whereas the RTs from the English writing to dictation task was modulated by word frequency and age of acquisition, as evidence that language transparency biases processing towards phonological or lexical strategies. In addition, after a Spanish task, participants more efficiently processed local information, which resulted in both the benefit of global congruent information and the reduced cost of incongruent global information. Additionally, the results showed that bilinguals adapt their attentional processing depending on the orthographic transparency.     

Separability of item and person parameters in response time models

This paper discusses two forms of separability of item and person parameters in the context of response time (RT) models. The first is separate sufficiency: the existence of sufficient statistics for the item (person) parameters that do not depend on the person (item) parameters. The second is ranking independence: the likelihood of the item (person) ranking with respect to RTs does not depend on the person (item) parameters. For each form a theorem stating sufficient conditions, is proved. The two forms of separability are shown to include several (special cases of) models from psychometric and biometric literature. Ranking independence imposes no restrictions on the general distribution form, but on its parametrization. An estimation procedure based upon ranks and pseudolikelihood theory is discussed, as well as the relation of ranking independence to the concept of double monotonicity.I am indebted to Wim van der Linden for bringing Thissen's (1983) paper to my notice, and to Martijn Berger, Frans Tan, and the anonymous reviewers for their constructive comments on earlier drafts of this paper.     

Isotonic ordinal probabilistic models (ISOP)

The concept of an ordinal instrumental probabilistic comparison is introduced. It relies on an ordinal scale given a priori and on the concept of stochastic dominance. It is used to define a weakly independently ordered system, or isotonic ordinal probabilistic (ISOP) model, which allows the construction of separate sample-free ordinal scales on a set of subjects and a set of items. The ISOP-model is a common nonparametric theoretical structure for unidimensional models for quantitative, ordinal and dichotomous variables.Fundamental theorems on dichotomous and polytomous weakly independently ordered systems are derived. It is shown that the raw score system has the same formal properties as the latent system, and therefore the latter can be tested at the observed empirical level.I wish to thank 3 reviewers and 2 editors who contributed a lot to the readability and precision of the article.     

The percentage bend correlation coefficient

A well-known result is that the usual correlation coefficient,, is highly nonrobust: very slight changes in only one of the marginal distributions can alter by a substantial amount. There are a variety of methods for correcting this problem. This paper identifies one particular method which is useful in psychometrics and provides a simple test for independence. It is not recommended that the new test replace the usual test ofH ₀: = 0, but the new test has important advantages over the usual test in terms of both Type I errors and power.     

Effects of component length and of the transitions among components in multiple schedules

Pigeons received equal variable-interval reinforcement during presentations of two line-orientation stimuli while five other orientations appeared in extinction. Component duration was 30 seconds for all orientations and the sequence was arranged so that each orientation preceded itself and each other orientation equally often. The duration of one component (0°) was shortened to 10 seconds and the other (90°) was lengthened to 50 seconds. All animals showed large increases in response rate in the shortened component and this increase was recoverable after an interpolated condition in which all components were again 30 seconds in duration. This effect was replicated in a second experiment in which component duration was changed from 150 seconds to 50 seconds and 250 seconds. An examination of local contrast effects during the first experiment showed that the shortened component produced local contrast during subsequent presentations of the lengthened component, just as would a component associated with more frequent reinforcement. When the presentation sequence was changed so that the lengthened component was always followed by the shortened component, response rates generally increased during the lengthened component. When the sequence was arranged so that the shortened component always preceded the longer component, response rate decreased in the former. These effects, as well as the increases in response rate following change in component length, seem not to be the product of local contrast effects among components.     

Local contrast and maintained generalization.

Pigeons received variable-interval reinforcement for key pecking during presentations of horizontal and vertical line-orientation stimuli, while pecks during five intermediate orientations were extinguished. Lowest peck rates were observed during presentations of negative stimuli adjacent to the positive orientations while peck rate during 45 degrees (the intermediate negative orientation) was relatively high, i.e., there were negative contrast shoulders. When peck rates were manipulated in the positive orientations, peck rate in neithboring orientations changed in the opposite direction. Contrast shoulders faded after prolonged training. A second type of contrast, local contrast, was correlated with similarity of preceding stimulus and different average peck rates during different stages of the discrimination process. The data suggest that sequential local contrast accompanying the formation of a discrimination contributes to the form of generalization gradients. Blough's model of stimulus control predicts the changes in gradient form described here, but may not accurately depict the underlying process responsible for gradient form.