Item difficulty of the tactual performance test location score.

The item difficulty and discrimination index values for the Tactual Performance Test Location scores are presented. The difference between this and the previous item analysis of the Location score is that in this analysis the subjects' Location scores were used only if the subject correctly remembered the block.     

Validity of Interpretation: A user validity perspective beyond the test score

This paper introduces the concept of user validity and provides a new perspective on the validity of interpretations from tests. Test interpretation is based on outputs such as test scores, profiles, reports, spreadsheets of multiple candidates' scores, etc. The user validity perspective focuses on the interpretations a test user makes given the purpose of the test and the information provided in the test output. This innovative perspective focuses on how user validity can be extended to content, criterion, and to some extent construct‐related validity. It provides a basis for researching the validity of interpretations and an improved understanding of the appropriateness of different approaches to score interpretation, as well as how to design test outputs and assessments that are pragmatic and optimal.     

A process dissociation approach to objective-projective test score interrelationships

Even when self-report and projective measures of a given trait or motive both predict theoretically related features of behavior, scores on the 2 tests correlate modestly with each other. This article describes a process dissociation framework for personality assessment, derived from research on implicit memory and learning, which can resolve these ostensibly conflicting results. Research on interpersonal dependency is used to illustrate 3 key steps in the process dissociation approach: (a) converging behavioral predictions, (b) modest test score intercorrelations, and (c) delineation of variables that differentially affect self-report and projective test scores. Implications of the process dissociation framework for personality assessment and test development are discussed.     

Interrater reliability of scores derived from two methods for scoring the clock drawing test

Scores on the Clock Drawing Test have long been considered a useful screening tool for neuropsychological dysfunction, and a number of scoring methods have been developed to evaluate various aspects of performance. This study compared quantitative and qualitative scoring by briefly trained students on 145 clock drawings produced by patients in a geriatric psychiatry outpatient clinic to estimate the interrater reliability of the methods, user's acceptance of the methods, and whether the methods provide differential diagnosis. Both systems showed acceptable interrater reliability. Using the quantitative method, raters scored drawings by patients with organic mental disease as more impaired than those patients diagnosed as depressed or schizophrenic. Results suggest that the Clock Drawing Test is a reliable screening tool for cognitive impairment in a geropsychiatric population, but the scoring methods examined do not yet appear psychometrically sound enough to provide a differential diagnosis.     

Diagnosing item score patterns on a test using item response theory-based person-fit statistics

Person-fit statistics have been proposed to investigate the fit of an item score pattern to an item response theory (IRT) model. The author investigated how these statistics can be used to detect different types of misfit. Intelligence test data were analyzed using person-fit statistics in the context of the G. Rasch (1960) model and R. J. Mokken's (1971, 1997) IRT models. The effect of the choice of an IRT model to detect misfitting item score patterns and the usefulness of person-fit statisticsfor diagnosis of misfit are discussed. Results showed that different types of person-fit statistics can be used to detect different kinds of person misfit. Parametric person-fit statistics had more power than nonparametric person-fit statistics.     

Validity of test score interpretations for three self-concept measures based on differing theoretical models

Three theoretical models have been proposed to represent self-concept: (a) unidimensional; (b) multidimensional; (c) multidimensional hierarchical. Inventories have been developed under each of the three competing theoretical models; which model best represents self-concept is unclear. Typically, self-concept construct validation has utilized various approaches including correlational, multitrait-multimethod, and factor analytic methods. Another method, however, for assessing validity would be to determine the consequences of score interpretations using different measures specific to each of the theoretical models. This paper examined Messick's notion (1989) of the validity of test-score interpretations as applied to three of the most widely used measures derived under each of the three different theoretical models of self-concept. Results suggest that overall multidimensional measures are more consistent in classifying individual's self-concept than unidimensional measures.     

The effects on the predictive variance of a new subject’s score for a new test

Abstract: It is often required to predict the scores or their variations under interest. Ishii and Watanabe (2001) investigated, in the context of psychological measurement, the Bayesian predictive distribution of a new subject’s scores for tests and subjects’ scores for a new test. In this paper, the Bayesian posterior predictive distribution of a new subject’s scores for a new parallel test were considered. And the effects of the number of subjects, the number of the tests, and the test reliability were investigated. Then, it was found that, under assumptions that (co)variance parameters are known, the predictive variance of a new subject’s score for a new test was equal to the predictive variances of the new subject’s scores for the existent tests. It was also found that the effect of the number of subjects was relatively large and the effect of the number of tests was relatively small, when a new subject’s scores for existent tests were not observed.     

Procedures for obtaining separate set and content components of a test score

Using two distinct models, several formulas for obtaining separate set and content components of a test score have been derived. Comparisons among the methods are made algebraically and through their application to a set of test data apparently affected by response sets.The author wishes to thank Dr. Norman Frederiksen, who brought this problem to his attention, and Drs. Ledyard R Tucker and Frederic M. Lord, who suggested three of the procedures. All have contributed materially to this paper through many helpful suggestions and criticisms.This paper was written while the author was an Associate in Research at the Educational Testing Service, Princeton, New Jersey.     

The estimation of true score variance and error variance in the classical test theory model

The use of one-way analysis of variance tables for obtaining unbiased estimates of true score variance and error score variance in the classical test theory model is discussed. Attention is paid to both balanced (equal numbers of observations on each person) and unbalanced designs, and estimates provided for both homoscedastic (common error variance for all persons) and heteroscedastic cases. It is noted that optimality properties (minimum variance) can be claimed for estimates derived from analysis of variance tables only in the balanced, homoscedastic case, and that there they are essentially a reflection of the symmetry inherent in the situation. Estimates which might be preferable in other cases are discussed. An example is given where a natural analysis of variance table leads to estimates which cannot be derived from the set of statistics which is sufficient under normality assumptions. Reference is made to Bayesian studies which shed light on the difficulties encountered. Work on this paper was carried out at the headquarters of the American College Testing Program, Iowa City, Iowa, while the author was on leave from the University College of Wales.     

Applicant reactions to test score banding in entry-level and promotional contexts.

This series of field studies used a fairness framework to investigate applicant reactions to test score banding in 3 police selection contexts. Studies 1 (N = 85) and 2 (N = 369) involved applicants for entry-level positions, and Study 3 (N = 39) involved applicants for promotion. Across all 3 studies, race interacted with applicants' belief that banding is associated with affirmative action to affect measures of fairness and organizational outcomes such as attractiveness and perceived employee relations. Reactions were also related to applicants' perceived outcomes as a result of banding. Results are explained in terms of self-interest and suggest that reactions to banding are largely a function of the association of banding with affirmative action.     

Elicitation of spontaneous mirror reversal drawing on the forehead

32 Japanese participants drew symmetrical and asymmetrical letters on their own foreheads. By observing their drawing strokes, it was judged that 29 of them drew symmetric letters in reverse, that is, as if these were viewed from inside the head. However, 14 of these 29 seemed to have noticed their reversed perception because they drew asymmetric letters intentionally reversed so they would appear correct when viewed from outside. Only three participants drew the letters in a normal way irrespective of the letter symmetry.