The reliability coefficient

The reliability coefficient is unlike other measures of correlation in that it is a quantitative statement of an act of judgment,—usually the test maker's, — that the things correlated are similar measures. Attempts to divorce it from this act of judgment are misdirected, just as would be an attempt to eliminate judgment of sameness of function of items when a test is originally drawn up. A coefficient of cohesion, entirely devoid of judgment, measuring the singleness of test function is proposed as an essential datum with reference to a test, but not as a substitute for the similar-form reliability co-efficient.     

A note on the estimation of test reliability by the Kuder-Richardson formula (20)

The Kuder-Richardson formula (20) is rewritten to be identical with the simplest formula, (21), except for the addition of a term involving the standard deviation, _p , of the itemp's. If _p can be estimated, a rapid and superior estimate of test reliability is possible in contrast to the simpler formula (21) used when the number of items and mean and standard deviation of test scores are known.Kuder, G. F. and Richardson, M. W. The theory of the estimation of test reliability.Psychometrika. 1937, 2, 151–160.     

The Kuder-Richardson formula (21) as a split-half coefficient,and some remarks on its basic assumption

Case IV of the Kuder-Richardson series, their formula (21), is derived as a generalized split-half Spearman-Brown coefficient. The basic assumption employed is shown to be sufficient to justify the various assumptions used in derivations by other authors. Some of the implications of this assumption are discussed.     

The reliability of a point biserial coefficient of correlation

An attempt is made to elaborate upon Lev's results concerning the reliability of the point biserial coefficient of correlation in a manner that will be helpful to the psychological statistician. Procedures required in the use of the non-centralt tables prepared by Johnson and Welch are described as they relate to the determination of the fiducial limits for a point biserial coefficient. A normal approximation technique for the estimation of fiducial limits is also suggested. Numerical evidence is presented which shows that relative to a given level of significance the width of the fiducial interval estimated from a point biserial coefficient of any size is smaller than that of the fiducial interval corresponding to an ordinary Pearsonian coefficient of the same magnitude.     

Some remarks on the kuder-richardson reliability coefficient

The Kuder-Richardson reliability coefficient is derived in a manner independent of that originally given. Various alternative forms applicable to special situations are exhibited with the purpose of making them available to others interested in using this formula. A simplification in computation is suggested for use with a calculating machine.     

A reliability coefficient for maximum likelihood factor analysis

Maximum likelihood factor analysis provides an effective method for estimation of factor matrices and a useful test statistic in the likelihood ratio for rejection of overly simple factor models. A reliability coefficient is proposed to indicate quality of representation of interrelations among attributes in a battery by a maximum likelihood factor analysis. Usually, for a large sample of individuals or objects, the likelihood ratio statistic could indicate that an otherwise acceptable factor model does not exactly represent the interrelations among the attributes for a population. The reliability coefficient could indicate a very close representation in this case and be a better indication as to whether to accept or reject the factor solution. This research was supported by the Personnel and Training Research Programs Office of the Office of Naval Research under contract US NAVY/00014-67-A-0305-0003. Critical review of the development and suggestions by Richard Montanelli were most helpful.     

A practical use for the KR-21 reliability coefficient

KR-21 provides a lower limit for the computed value of KR-20. KR-20 is equivalent to coefficient alpha when a test is composed of dichotomous items scored 0 or 1. Therefore, KR-21 coefficients, computed from simple summary statistics, can be used in cases in which journal authors do not provide the test score reliability. Use of KR-21 in these cases will provide the reader with a lower limit for the value of KR-20.     

On determining the reliability and significance of a tetrachoric coefficient of correlation

In this note are presented facilitating tables for the estimation of the standard error of a tetrachoric and also tables providing significant and very significant tetrachoric coefficients for various sizes of samples and various combinations of proportions in the dichotomized distributions.The task of computing the values in the accompanying tables should be credited to Mr. Lyons.     

A reliability generalization study of coefficient alpha for the UCLA loneliness scale

Loneliness is a psychological construct that has been reported in a variety of populations and associated with a number of other negative psychological problems. This study was an examination of coefficient alpha of a prominent measure of loneliness: the University of California, Los Angeles (UCLA) Loneliness Scale (Russell, Peplau, & Cutrona, 1980; Russell, 1996). We utilized reliability generalization to provide an aggregate estimate of the reliability of the scale over time and in a variety of populations as well as to assess and identify sampling and demographic characteristics associated with variability in coefficient alpha. Of the 213 studies examined, 80 had reported alpha estimates, and we used them in this analysis. We discuss conditions associated with variability in coefficient alpha along with pertinent implications for practice and future research.     

On robusiness of the normal-theory based asymptotic distributions of three reliability coefficient estimates

This paper studies the asymptotic distributions of three reliability coefficient estimates: Sample coefficient alpha, the reliability estimate of a composite score following a factor analysis, and the estimate of the maximal reliability of a linear combination of item scores following a factor analysis. Results indicate that the asymptotic distribution for each of the coefficient estimates, obtained based on a normal sampling distribution, is still valid within a large class of nonnormal distributions. Therefore, a formula for calculating the standard error of the sample coefficient alpha, recently obtained by van Zyl, Neudecker and Nel, applies to other reliability coefficients and can still be used even with skewed and kurtotic data such as are typical in the social and behavioral sciences.This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of North Texas faculty research grant. We would like to thank the Associate Editor and two reviewers for suggestions that helped to improve the paper.     

The memory drum theory after twenty years

The difference between the Henry “memory-drum” theory and our version is that ours includes an additional assumption that, after programming has occurred, the resultant representation can be stored in short-term memory. Otherwise, the essential ideas are the same in the two theories. Implications of the presently available data for the distinction between the theories are discussed. Regardless of how one evaluates our added assumption, it is clear that the essential insight of the Henry theory has fared very well in the 20 yr since the theory first appeared in print.     

A Monte Carlo study of the Hoyt reliability coefficient in the Bower-Trabasso model

Monte Carlo procedures are used to study the sampling distribution of the Hoyt reliability coefficient. Estimates of mean, variance, and skewness are made for the case of the Bower-Trabasso concept identification model. Given the Bower-Trabasso assumptions, the Hoyt coefficient of a particular concept identification experiment is shown to be statistically unlikely.     

The relation of the reliability of multiple-choice tests to the distribution of item difficulties

Under certain assumptions an expression, in terms of item difficulties and intercorrelations, is derived for the curvilinear correlation of test score on the ability underlying the test, this ability being defined as the common factor of the item tetrachoric intercorrelations corrected for guessing. It is shown that this curvilinear correlation is equal to the square root of the test reliability. Numerical values for these curvilinear correlations are presented for a number of hypothetical tests, defined in terms of their item parameters. These numerical results indicate that the reliability and the curvilinear correlation will be maximized by (1) minimizing the variability of item difficulty and (2) making the level of item difficulty somewhat easier than the halfway point between a chance percentage of correct answers and 100 per cent correct answers.     

Task and distribution sampling affect auditory category learning

There is substantial evidence that two distinct learning systems are engaged in category learning. One is principally engaged when learning requires selective attention to a single dimension (rule-based), and the other is drawn online by categories requiring integration across two or more dimensions (information-integration). This distinction has largely been drawn from studies of visual categories learned via overt category decisions and explicit feedback. Recent research has extended this model to auditory categories, the nature of which introduces new questions for research. With the present experiment, we addressed the influences of incidental versus overt training and category distribution sampling on learning information-integration and rule-based auditory categories. The results demonstrate that the training task influences category learning, with overt feedback generally outperforming incidental feedback. Additionally, distribution sampling (probabilistic or deterministic) and category type (information-integration or rule-based) both affect how well participants are able to learn. Specifically, rule-based categories are learned equivalently, regardless of distribution sampling, whereas information-integration categories are learned better with deterministic than with probabilistic sampling. The interactions of distribution sampling, category type, and kind of feedback impacted category-learning performance, but these interactions have not yet been integrated into existing category-learning models. These results suggest new dimensions for understanding category learning, inspired by the real-world properties of auditory categories.     

Tactual performance test trials: internal consistency reliability using the Gilmer-Feldt coefficient.

Internal consistency reliabilities using the Gilmer-Feldt coefficient were calculated for the Tactual Performance Test for the Preferred Hand (n=298), Nonpreferred Hand (n = 302), and Both Hands (n = 314) trials, and total time. Reliabilities were reported for the total sample and three groups: 118 normal volunteers, 95 inpatient alcoholics, and 103 undiagnosed patients sent for neuropsychological assessment. The reliabilities ranged from .5584 to .8953.