Test theory without an answer key

A general model is presented for homogeneous, dichotomous items when the answer key is not known a priori. The model is structurally related to the two-class latent structure model with the roles of respondents and items interchanged. For very small sets of respondents, iterative maximum likelihood estimates of the parameters can be obtained by existing methods. For other situations, new estimation methods are developed and assessed with Monte Carlo data. The answer key can be accurately reconstructed with relatively small sets of respondents. The model is useful when a researcher wants to study objectively the knowledge possessed by members of a culturally coherent group that the researcher is not a member of.This research was supported by NSF Grant No. SES-8320173 to the authors. We gratefully acknowledge comments and suggestions from John Boyd, Tarow Indow, and Kathy Maher as well as the editor and several anonymous referees.     

SPELLING AND EMERGENT PICTURE-PRINTED WORD RELATIONS ESTABLISHED WITH DELAYED IDENTITY MATCHING TO COMPLEX SAMPLES

Students with academic deficits learned delayed matching-to-sample tasks that used complex sample stimuli, each consisting of a picture and a printed word. A touch to the sample complex removed it from the computer display and produced either picture comparisons or a choice pool of letters. If the comparisons were pictures, selecting the picture identical to the preceding sample was reinforced. If the letters appeared, letter-by-letter construction of the preceding printed word sample was reinforced. The procedure engendered new constructed-response spelling performances and arbitrary relations among pictures and printed words in matching to sample. The emergence of relations among different sets of printed words (paired with the same pictures) suggested classes of equivalent stimuli. Outcome tests involving spoken words as sample stimuli suggested expansion of subjects' spelling repertoires and stimulus classes.     

USING STIMULUS EQUIVALENCE PROCEDURES TO TEACH NAME-FACE MATCHING TO ADULTS WITH BRAIN INJURIES

On pretests, 3 men with brain injuries matched dictated names of three therapists to written names, but did not match dictated or written names to photos, produce correct names in response to photos, locate offices given written names, or name therapists on sight. Match-to-sample training established conditional relations between dictated names and photos. Posttests showed the emergence of untrained conditional relations involving photos and written names, indicating development of three classes of equivalent stimuli (each containing a dictated name, photo, and written name). For 1 participant, conditional relations involving photos of office nameplates were also examined, but did not emerge pre- or posttraining. Two participants produced names orally when given photos and sorted written names and faces together after training; the 3rd participant was unavailable for these posttests. After training, 1 participant located and named all three therapists in their offices.     

An automated test development of parallel tests from a seed test

Binary programming models are presented to generate parallel tests from an itembank. The parallel tests are created to match item for item an existing seed test and match user supplied taxonomic specifications. The taxonomic specifications may be either obtained from the seed test or from some other user requirement. An algorithm is presented along with computational results to indicate the overall efficiency of the process. Empirical findings based on an itembank for the Arithmetic Reasoning section of the Armed Services Vocational Aptitude Battery are given.The Office of Naval Research, Program in Cognitive Science, N00014-87-C-0696 partially supported the work of Douglas H. Jones. The Rutgers Research Resource Committee of the Graduate School of Management partially supported the work of Douglas H. Jones and Ing-Long Wu. A Thomas and Betts research fellowship partially supported the work of Ing-Long Wu. The Human Resources Laboratory, United States Air Force, partially supported the work of Ronald Armstrong. The authors benefited from conversations with Dr. Wayne Shore, Operational Technologies, San Antonio, Texas. The order of authors' names is alphabetical and denotes equal authorship.     

Ordinal consistency and ordinal true scores

This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.Part of this work was done while the author was a Visiting Fellow at Macquarrie University. The paper has benefitted from discussions with Professors Thomas J. Reynolds and Roderick P. McDonald and from the comments of several anonymous reviewers.     

Functional classes and equivalence relations

Three adult subjects were taught a set of two-choice simultaneous discriminations, with three positive and three negative stimuli; all possible combinations of positive and negative stimuli yielded nine different pairs. The discriminations were repeatedly reversed and rereversed, the former positive stimuli becoming negative and the former negative stimuli becoming positive. With all subjects, a reversal of the contingencies for one pair of stimuli became sufficient to change their responses to all of the other pairs. The reversals had produced functional stimulus classes. Then, all subjects showed conditional discriminations emerging between members of a functional class; given a sample from one class and comparisons from both classes, they selected the comparison that was in the same class as the sample. Next, 2 of the subjects showed that the within-class conditional relations possessed the symmetric and transitive properties of equivalence relations; after having been taught to relate new stimuli to existing class members, the subjects then matched other class members to the new stimuli. Subsequent tests of two-choice discriminations showed that the conditional discriminations had transferred functional class membership to the new stimuli. The 3rd subject, who did not show equivalence relations among functional class members, was also found to have lost the within-class conditional relations after the equivalence tests.     

Multifactor measurement of bodily feelings: Conceptual development,scale construction,and empirical validation

A series of studies is described demonstrating a coherent exploratory and confirmatory factor-analytic approach to the scaling of self-reported bodily feelings. This approach attempts to optimize the orthogonality, parsimony, and meaningfulness of a multifactor inventory. Psychometric properties of the resulting 35-item Bodily Feeling Scale (BFS) are presented and its comparability with other inventories is discussed. A validity study using item-content-based scales of the Minnesota Multiphasic Personality Inventory showed both concurrent and discriminative validity of the BFS. Implications for clinical and research utility are discussed.Portions of this research were conducted while the first author was supported by a University Graduate Fellowship awarded by the Graduate School of Arts and Sciences at Vanderbilt University.Professor Nunnally passed away in August 1982. His major contributions to the several stages of this research program are recognized through coauthorship.     

Conditional relations by monkeys: Reflexivity, symmetry, and transitivity

Two cynomolgous macaques categorized six colors into two groups of three after conditional discrimination training (zero-delay symbolic match-to-sample). The procedures resulted in the establishment of relations among the elements of each set-relations that were not specifically trained and that can be characterized by the properties of reflexivity, symmetry, and transitivity. Each set of colors was related to a characteristic pattern of responding: One response pattern involved temporal duration (press and hold the response keys); the second response pattern entailed repeated pressing and releasing of the response keys (fixed ratio 8). Six combinations of two colors were trained, three combinations from each set. After discriminative performance stabilized for each monkey, they were tested with 10 additional color combinations, all of which differed from the training combinations. The conditional relations established between test combinations can be characterized as stimulus equivalence. The training procedures were analogous to the procedure of using category names, and have implications for understanding the function of language in the formation of equivalence classes.     

A nonparametric approach for assessing latent trait unidimensionality

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.The referees' comments were remarkably detailed and greatly enhanced the writeup and sensitized the author to certain pertinent issues. Discussions with Fritz Drasgow, Lloyd Humphreys, Dennis Jennings, Brian Junker, Robert Linn, Ratna Nandakumar, and Robin Shealy were also very useful.This research was supported by the Office of Naval Research under grant N00014-84-K-0186; NR 150-533, and by the National Science Foundation under grant DMS 85-03321.