The validity of the successive intervals method of psychometric scaling

The degree to which scale values computed by the method of successive intervals diverge from theoretically true values is seen to be due to three types of error: error due to inequalities in variances of the distributions from which the scale values are computed, error due to non-normality of the distributions, and sampling error. The contribution of each type of error to the total error is evaluated; the latter is seen to be surprisingly small under appropriate conditions. Certain aspects of the formal methodology underlying scaling procedures are also briefly considered.This paper reports research undertaken in cooperation with the Quartermaster Food and Container Institute for the Armed Forces, and has been assigned number 475 in the series of papers approved for publication. The views or conclusions contained in this report are those of the authors. They are not to be construed as necessarily reflecting the views or indorsement of the Department of Defense.     

Approximating maximum test validity by a non-parametric method

The Gleser-DuBois conditions for selecting from a number of test items those which will maximize the correlation between total test score and criterion will degenerate into expressions requiring only item counts on total distributions and the upper halves of distributions. A grouping convention for scores near medians is recommended. The inefficiency of the method is easily compensated for, because, regardless of the size of the sample, only standard test-scoring equipment and brief computations are required. A procedure is outlined, and some applications are discussed.     

A modification of the method of successive intervals

A modification of the method of successive intervals is presented which yields scale values correlating .995 with those from Thurstone's method described by Saffir. Values yielded by the present method can be obtained in 25 per cent of the time required by the older method and are shown to be, on apriori grounds, more reliable as well.     

A matching method of successive approximations and its instrumentation

The present paper reports a method of successive approximations. The technique is optimal for matching stimuli with respect to time and all Ss are forced to use the same strategy. The S’s answer to the question of which is more intense, a reference or a matching stimulus, initiates a new matching stimulus according to an algorithm that bisects intervals up or down in successive trials and gives a new matching stimulus to judge. A circuitry that governs the method of successive approximations is reported. The method is exemplified by an experimental set-up for odor intensity matching.     

Multidimensional successive categories scaling: A maximum likelihood method

A single-step maximum likelihood estimation procedure is developed for multidimensional scaling of dissimilarity data measured on rating scales. The procedure can fit the euclidian distance model to the data under various assumptions about category widths and under two distributional assumptions. The scoring algorithm for parameter estimation has been developed and implemented in the form of a computer program. Practical uses of the method are demonstrated with an emphasis on various advantages of the method as a statistical procedure.The research reported here was partly supported by Grant A6394 to the author by Natural Sciences and Engineering Research Council of Canada. Portions of this research were presented at the Psychometric Society meeting in Uppsala, Sweden, in June, 1978. MAXSCAL-2.1, a program to perform the computations discussed in this paper may be obtained from the author. Thanks are due to Jim Ramsay for his helpful comments.     

Fluctuations in recall across successive test trials

The purpose of the experiment was to determine whether item fluctuation (intertrial forgetting and spontaneous recovery) depends upon the time allowed for recall and/or the demand characteristics of the experimental situation (manipulated via instructions to S). One hundred. and forty-four university undergraduates studied a 36-item list and then had two successive recall trials which were 9, 36, or 144 sec in duration. On the second recall trial, Ss were instructed to recall previously recalled items only (“old”), previously nonrecalled items only (“new”), or both previously recalled items and previously nonrecalled items (“both”). The results suggest two conclusions: (1) Although recall trial duration affects the overall amount of recall, it does not affect the amount of item fluctuation; and (2) an instruction to recall only “old” items or to recall only “new” items results in suppression of the nonrequested items without a corresponding facilitation for the requested items.     

A field study of frame-of-reference effects on personality test validity

As part of a test validation study at a major U.S.-based airline, the authors tested the effects of providing an "at work" frame-of-reference on the validity of the NEO Five-Factor Inventory among a sample of customer service supervisors (N = 206). Frame-of-reference moderated the validity of the Extraversion and Openness to Experience subscales after controlling for cognitive ability. In addition, the frame-of-reference personality test showed incremental validity over cognitive ability (deltaR2 = .16), but the standard personality test did not (deltaR2 = .05). The authors' discussion focuses on implications for personality theory and research and on implications for increasing the validity of personality tests in organizational settings.     

Choice behavior in a discrete-trial concurrent VI-VR: A test of maximizing theories of matching

Rats were trained on a discrete-trial procedure in which one alternative (VR) was correlated with a constant probability of reinforcement while the other was correlated with a VI schedule which ran during the intertrial intervals and held the scheduled reinforcer until they were obtained by the next VI response. Relative reinforcement rate was varied in series of conditions in which the VR schedule was varied and in series in which the VI was varied. Choice behavior was described well by the generalized matching law, although moderate undermatching occurred for all subjects. Contrary to the predictions of molar maximizing (optimality) theories, there was no consistent bias in favor of the ratio alternative, and the sensitivity to reinforcement allocation was not systematically affected by whether the ratio or interval schedule was varied. The results were also contrary to momentary maximizing accounts, as there was no correspondence between the probability of a changeover to the VI behavior and the time since the last response to the VI alternative. Neither variety of maximizing theory appears to provide a general explanation of matching in concurrent schedules.     

Alpha if item deleted: A note on loss of criterion validity in scale development if maximizing coefficient alpha

This note is concerned with a validity‐related limitation of the widely available and routinely used index ‘alpha if item deleted’ in the process of construction and development of multiple‐component measuring instruments. Attention is drawn to the fact that this statistic can suggest dispensing with such scale components, whose removal leads to loss of criterion validity while maximizing the popular coefficient alpha. As an alternative, a latent variable modelling approach is discussed that can be used for point and interval estimation of composite criterion validity (as well as reliability) after deletion of single components. The method can also be utilized to test conventional or minimum level hypotheses about associated population change in measurement quality indices.     

The validity of a persistence test

In order to raise the predictive efficiency of its college entrance test battery, the Educational Testing Service is working on the development of non-academic measures to supplement the standard aptitude and achievement examinations. A test of difficult number series problems was set up to measure persistence by tempting the students to give up early; the students were informed that some of the problems had no solution, and that full credit would be received by so marking them. This test was tried out and found to have some correlation with grades, while having no correlation with the other tests. Adding this test to the battery showed an appreciable rise in the battery's multiple correlation with grades.     

A method of successive cumulations for the scaling of pair-comparison preference judgments

The present model treats the scaling of pair-comparison preference judgments among a unidimensional set of stimuli across a population of individuals. Given a setS ofn stimuli,S = {S ₁,S ₂, ,S _n }, the model yields a partially ordered metric on the interstimulus distances which may be used to construct an interval scale of values forS. Obtained also are a set of predictionsP = {P ₁,P ₂, ,P _n } whereP _i is the proportion of individuals in the population whose first choice among the elements ofS isS _i . A numerical illustration is offered and comparisons are drawn with Coombs' unfolding technique.This work was supported in part by Grant GB 2345 from the National Science Foundation. An earlier version of this paper was prepared while the author was a consultant to Proctor & Gamble Co. during the summer of 1964.Now with Proctor & Gamble Co.     

Maximizing test validity by item selection

The exact condition for discardingk items from a test in order to obtain a residual test with higher validity is derived. A proof that validity always increases is given for the casek=1. The lack of uniqueness of maximum validity when achieved by use of the condition is discussed. With the use of additional restrictions on items to be included in the initial test, a practical test construction procedure which has several advantages over previous methods is developed. The homogeneity of tests constructed by the method is discussed, and applications are given.     

Incremental validity principles in test construction

The authors articulate 5 basic principles for enhancing incremental validity, both among elements within a test and between tests, during test construction: (a) careful, precise articulation of each element or facet within the content domain; (b) reliable measurement of each facet through use of multiple, alternate-form items; (c) examination of incremental validity at the facet level rather than the broad construct level; (d) use of items that represent single facets rather than combinations of facets; and (e) empirical examination of whether there is a broad construct or a combination of separate constructs. Using these principles, the authors offer specific suggestions for modifications in 3 classic test construction approaches; (a) criterion keying, (b) inductive test construction, and (c) deductive test construction. Implementation of these suggestions is likely to provide theoretical clarification and improved prediction.