A note on ROC analysis and non-parametric estimate of sensitivity

In the signal detection paradigm, the non-parametric index of sensitivity A′, as first introduced by Pollack and Norman (1964), is a popular alternative to the more traditional d′ measure of sensitivity. Smith (1995) clarified a confusion about the interpretation of A′ in relation to the area beneath proper receiver operating characteristic (ROC) curves, and provided a formula (which he called A′′) for this commonly held interpretation. However, he made an error in his calculations. Here, we rectify this error by providing the correct formula (which we call A) and compare the discrepancy that would have resulted. The corresponding measure for bias b is also provided. Since all such calculations apply to “proper” ROC curves with non-decreasing slopes, we also prove, as a separate result, the slope-monotonicity of ROC curves generated by likelihood-ratio criterion.This revised article was published online in August 2005 with the PDF paginated correctly.     

A non-parametric test of correlation using rank orders within subgroups

Kendall's rank order test for association between two variables is generalized to the case where the total sample is made up of several subgroups and the data on one or both variables consist of the rank order within each subgroup. The test involves no assumptions concerning scales of measurement, shapes of distributions, or relative level of excellence or amount of variability of the different subgroups. Two empirical examples indicate that the normal approximation to the exact test of significance can be considered adequate for most practical situations. Special consideration is given to the case of tied ranks. If ties occur in but one variable within any given subgroup, only a slight modification in procedure is needed. Extensive ties in both variables within subgroups lead to difficulties in determining the appropriate correction for continuity.     

A note on moran's measure of multiple rank correlation

A measure of multiple rank correlation,T _y.12 ², is proposed for the situation with no tied observations in the variables. The measure is a weighted average of two squared Kendall taus. It is shown thatT _y.12 ² is equivalent to a statistic previously proposed by Moran and thus a new interpretation is given to Moran's statistic.The author wishes to thank Nancy Anderson, Willard Larkin, and Kent Norman for their helpful comments.     

A note on item analysis and the criterion of internal consistency

The implications contained in Richardson's article on item analysis in March 1936 issue ofPsychometrika are examined in the light of multiple factor theory. It is shown that item analysis is a necessary, but not a sufficient condition for the construction of a test which shall measure a single trait. The intercorrelations of certain items selected by a method of item analysis are examined, found to contain many zero and some negative correlations. Multiple factor analysis showed that eight traits were measured by the items which had been asserted to measure only one.     

A note on factor analysis: Arbitrary orthogonal transformations

A modification of the Gram-Schmidt process yields an easily constructed orthogonal transformation matrix which may be used to rotate a centroid, principal axis, or maximum likelihood factor matrix in a manner such that one of the new axes has predetermined direction. The procedure is illustrated by rotating a centroid factor matrix into an abbreviated bifactor matrix, the general factor being defined as the centroid of a specified subgroup of reasoning tests.     

A note on the internal consistency of the sexual orientation method

The S.O.M. was administered to two samples of homosexuals. There was some reduction in the number of inconsistent response patterns when a modified scoring system was used. Further examination of the data indicated that an inconsistent style of response might not be simply an index of faking.     

A note on preliminary tests of equality of variances

Preliminary tests of equality of variances used before a test of location are no longer widely recommended by statisticians, although they persist in some textbooks and software packages. The present study extends the findings of previous studies and provides further reasons for discontinuing the use of preliminary tests. The study found Type I error rates of a two‐stage procedure, consisting of a preliminary Levene test on samples of different sizes with unequal variances, followed by either a Student pooled‐variances t test or a Welch separate‐variances t test. Simulations disclosed that the twostage procedure fails to protect the significance level and usually makes the situation worse. Earlier studies have shown that preliminary tests often adversely affect the size of the test, and also that the Welch test is superior to the t test when variances are unequal. The present simulations reveal that changes in Type I error rates are greater when sample sizes are smaller, when the difference in variances is slight rather than extreme, and when the significance level is more stringent. Furthermore, the validity of the Welch test deteriorates if it is used only on those occasions where a preliminary test indicates it is needed. Optimum protection is assured by using a separate‐variances test unconditionally whenever sample sizes are unequal.     

A note on the uniqueness of minimum rank solutions in factor analysis

A revised theorem is presented concerning uniqueness of minimum rank solutions in common factor analysis.     

A note on the consistency of estimators in the analysis of moment structures

This paper is concerned with the consistency of estimators in the analysis of moment structures. An example is given where in spite of identifiability of the parameters the associated estimator is inconsistent. Consistency is shown to hold under the additional condition of compactness of the parameter space.     

A note on correcting for chance success in objective tests

The conventional scoring formula to correct for guessing is derived and is compared with a regression method for scoring which has been recently proposed by Hamilton. It is shown that the usual formula,S=R–W/(n–1), yields a close approximation (correct within one point) to the maximum-likelihood estimate of an individual's true score on the test, if we assume that the individual knows or does not know the answer to each item, that guessing at unknown items is random, and that success at guessing is governed by the binomial law. It is also shown that the usual scoring formula yields an unbiased estimate of the individual's true score, when the true score is defined as the mean score over an indefinitely large number of independent attempts at the test or at equivalent (parallel) tests.     

A note on permutation tests of significance for multiple regression coefficients

In the vast majority of psychological research utilizing multiple regression analysis, asymptotic probability values are reported. This paper demonstrates that asymptotic estimates of standard errors provided by multiple regression are not always accurate. A resampling permutation procedure is used to estimate the standard errors. In some cases the results differ substantially from the traditional least squares regression estimates.     

Internal consistency of tests: Analyses old and new

A coefficient derived from communalities of test parts has been proposed as greatest lower bound to Guttman's immediate retest reliability. The communalities have at times been calculated from covariances between itemsets, which tends to underestimate appreciably. When items are experimentally independent, a consistent estimate of the greatest defensible internal-consistency coefficient is obtained by factoring item covariances. In samples of modest size, this analysis capitalizes on chance; an estimate subject to less upward bias is suggested. For estimating alternate-forms reliability, communality-based coefficients are less appropriate than stratified alpha.I thank Edward Haertel for comments and suggestions, and Andrew Comrey for data.