Some standard errors in item response theory

The mathematics required to calculate the asymptotic standard errors of the parameters of three commonly used logistic item response models is described and used to generate values for some common situations. It is shown that the maximum likelihood estimation of a lower asymptote can wreak havoc with the accuracy of estimation of a location parameter, indicating that if one needs to have accurate estimates of location parameters (say for purposes of test linking/equating or computerized adaptive testing) the sample sizes required for acceptable accuracy may be unattainable in most applications. It is suggested that other estimation methods be used if the three parameter model is applied in these situations.The research reported here was supported, in part, by contract #F41689-81-6-0012 from the Air Force Human Resources Laboratory to McFann-Gray & Associates, Benjamin A. Fairbank, Jr., Principal Investigator. Further support of Wainer's effort was supplied by the Educational Testing Service, Program Statistics Research Project.     

Canonical/redundancy factoring analysis

The interrelationships between two sets of measurements made on the same subjects can be studied by canonical correlation. Originally developed by Hotelling [1936], the canonical correlation is the maximum correlation betweenlinear functions (canonical factors) of the two sets of variables. An alternative statistic to investigate the interrelationships between two sets of variables is the redundancy measure, developed by Stewart and Love [1968]. Van Den Wollenberg [1977] has developed a method of extracting factors which maximize redundancy, as opposed to canonical correlation.A component method is presented which maximizes user specified convex combinations of canonical correlation and the two nonsymmetric redundancy measures presented by Stewart and Love. Monte Carlo work comparing canonical correlation analysis, redundancy analysis, and various canonical/redundancy factoring analyses on the Van Den Wollenberg data is presented. An empirical example is also provided.Wayne S. DeSarbo is a Member of Technical Staff at Bell Laboratories in the Mathematics and Statistics Research Group at Murray Hill, N.J. I wish to express my appreciation to J. Kettenring, J. Kruskal, C. Mallows, and R. Gnanadesikan for their valuable technical assistance and/or for comments on an earlier draft of this paper. I also wish to thank the editor and reviewers of this paper for their insightful remarks.     

WHAT IS THE ROLE OF SCIENCE IN THE DIALOGUE PROPOSED BY WILLIAM KLINK?

Abstract. Klink rejects the use of ecological models in environmental decision making because their predictions cannot be tested by rigorous scientific methods. I argue that models that cannot be tested according to the rigorous standards of the physical sciences can still be considered “scientific”; they are useful (and, in practice, used) for assessing the impacts of human actions on the environment and choosing between alternative courses of action. It is, however, important to be aware of the uncertainties and to make corrections as new data and insights become available. The interplay between (1) model-based decisions and action and (2) their consequences and subsequent corrections can be regarded as a dialogue between humans and nature (or God) in the sense proposed by Klink. Klink also claims that future actions should be informed by the larger vision of theology and should not be based on science. I suggest that science has an indispensible role. The larger vision is needed to respond to the fundamental religious question: How should I live—and why? But this question cannot be answered without first addressing the fundamental scientific question: How does the world work? I suggest that responses to the first question can be formulated as visions of a future state of existence that we feel compelled to strive to realize, and that science is necessary to provide “maps of reality” needed to realize visions. I also suggest that Christian traditions can probably provide adequate visions; the crucial need is for improving our “maps of reality.”     

A simple Gauss-Newton procedure for covariance structure analysis with high-level computer languages

An implementation of the Gauss-Newton algorithm for the analysis of covariance structures that is specifically adapted for high-level computer languages is reviewed. With this procedure one need only describe the structural form of the population covariance matrix, and provide a sample covariance matrix and initial values for the parameters. The gradient and approximate Hessian, which vary from model to model, are computed numerically. Using this approach, the entire method can be operationalized in a comparatively small program. A large class of models can be estimated, including many that utilize functional relationships among the parameters that are not possible in most available computer programs. Some examples are provided to illustrate how the algorithm can be used.We are grateful to M. W. Browne and S. H. C. du Toit for many invaluable discussions about these computing ideas. Thanks also to Scott Chaiken for providing the data in the first example. They were collected as part of the U.S. Air Force's Learning Ability Measurement Project (LAMP), sponsored by the Air Force Office of Scientific Research (AFOSR) and the Human Resource Directorate of the Armstrong Laboratory (AL/HRM).     

Analyzing preventive trials with generalized additive models

Described a new class of nonparametric regression procedures called generalized additive models (Hastie and Tibshirani, 1991) for assessing intervention effects in mental health preventive field trials. Such models are often better than analysis of covariance models for examining intervention effects adjusted for one or more baseline characteristics and for assessing potential interactions between the intervention and baseline characteristics. Because of these advantages, generalized additive models are important tools analysts should consider in evaluating preventive field trials. We apply generalized additive models as well as more standard linear models to data from a preventive trial aimed at improving mental health and school performance outcomes through a universal intervention in first and second grades. Practical guidance is given to researchers regarding when generalized additive models would be beneficial.     

A multidimensional latent trait model for measuring learning and change

A latent trait model is presented for the repeated measurement of ability based on a multidimensional conceptualization of the change process. A simplex structure is postulated to link item performance under a given measurement condition or occasion to initial ability and to one or more modifiabilities that represent individual differences in change. Since item discriminations are constrained to be equal within a measurement condition, the model belongs to the family of multidimensional Rasch models. Maximum likelihood estimators of the item parameters and abilities are derived, and an example provided that shows good recovery of both item and ability parameters. Properties of the model are explored, particularly for several classical issues in measuring change.     

Measuring clinical pain

There are two goals in attempts to measure, the intention to describe and the intention to compare. These two functions work hand in hand and are brought to fruition in psychological and educational testing only by psychometrics that can produce person-free item calibrations and item-free person measures. The only such psychometrics that currently exist is that based on George Rasch's work. One of the consequences of using the tools Rasch left us is the Pain Scale, a rating scale that asks the person in pain to compare the intensityof his/her own pain with that he/she imagines is described by some 25 adjectives. Analyses of the responses of 53 people with chronic lower back pain to the Pain Scale using the Rasch partial-credit latent-trait model show that they were very much in agreement about the amount of hurt implied by each of the adjectives. An examination of person fit to the partial-credit model gave no hint of a response set in the data but two people did differ very much from their peers in their operational definitions of back pain and were studied separately. The evidence from these two people suggested that they either could not understand the task before them or could not get enough perspective on their pain to describe its intensity.I would like to thank Benjamin Wright and Geoff Masters for their help with this research. Drs. David J. Smith and James B. Boscardin were the first to bring my attention to the problem of measuring pain and were generous in their financial support of this work.     

Testing the conditional independence and monotonicity assumptions of item response theory

When item characteristic curves are nondecreasing functions of a latent variable, the conditional or local independence of item responses given the latent variable implies nonnegative conditional covariances between all monotone increasing functions of a set of item responses given any function of the remaining item responses. This general result provides a basis for testing the conditional independence assumption without first specifying a parametric form for the nondecreasing item characteristic curves. The proposed tests are simple, have known asymptotic null distributions, and possess certain optimal properties. In an example, the conditional independence hypothesis is rejected for all possible forms of monotone item characteristic curves.The author acknowledges Paul W. Holland for valuable conversations on the subject of this paper; Henry Braun and Fred Lord for comments at a presentation on this subject which led to improvements in the paper; Carl H. Haag for permission to use the data in §4; Bruce Kaplan for assistance with computing; and two referees for helpful suggestions. Requests for reprints should be sent to Paul R. Rosenbaum     

A nonrandomized minimax solution for passing scores in the binomial error model

A nonrandomized minimax solution is presented for passing scores in the binomial error model. The computation does not require prior knowledge regarding an individual examinee or group test data for a population of examinees. The optimum passing score minimizes the maximum risk which would be incurred by misclassifications. A closed-form solution is provided for the case of constant losses, and tables are presented for a variety of situations including linear and quadratic losses. A scheme which allows for correction for guessing is also described.This work was performed pursuant to Grant No. NIE-G-78-0087 with the National Institute of Education, Department of Health, Education, and Welfare, Huynh Huynh, Principal Investigator. Points of view or opinions stated do not necessarily reflect NIE position or policy and no official endorsement should be inferred. The editorial assistance and comments of Anthony J. Nitko and of Joseph C. Saunders are gratefully acknowledged.