Methods for averaging alpha coefficients in reliability generalization studies

The reliability generalization (RG) approach is a kind of meta-analysis that aims to statistically integrate a set of independent reliability coefficients obtained in several applications of a test, with the purpose of characterizing the measurement error and determining which factors related to the studies and samples can explain its variability. Diverse procedures have been proposed in the literature for averaging a set of independent alpha coefficients, and there is no consensus about which methods are best. Here, we present the results of a Monte Carlo simulation study, comparing the performance of twelve procedures proposed in Feldt and Charter, in terms of bias and mean square error. These procedures differ from each other in the transformation (or not) of the coefficients, and in the application or not of a weighting scheme based on sample size. Our results recommend using weighted methods in contrast to unweighted ones, and transforming the coefficients by the Hakstian and Whalen's proposal or by the proposal based on the square root of the inverse alpha coefficient. Lastly, we discuss the relations between the diverse procedures for averaging alpha coefficients with fixed-effects, random-effects, and varying coefficients models.     

The estimation of reliability when several trials are available

In certain types of experiment it frequently happens that we have a block of data consisting of several trials on the same individuals. Using the methods of estimation provided by the analysis of variance, estimates of reliability are derived for this case, and the conditions under which each is valid are discussed. Various relations between these estimates and the product-moment coefficient of correlation are obtained.From the Laboratory of Physiological Hygiene, University of Minnesota. This work was supported in part under the terms of a contract between the Regents of the University of Minnesota and the Office of Scientific Research and Development. Important financial assistance was also provided by the Nutrition Foundation, Inc.: The U. S. Sugar Cane Refiners Association, N. Y.; the Corn Industries Research Foundation, N. Y.: Swift and Co., Chicago; the National Confectioners' Association; the National Daily Council, Chicago; and the Graduate Medical Research Fund. University of Minnesota. The author is indebted for assistance and encouragement to all the members of the staff of the Laboratory of Physiological Hygiene, and especially to Mr. Harold Guetzkow and Mr. Rich and Seymour.     

Reliability formulas for independent decision data when reliability data are matched

A distinction is made between reliability data and decision data. Each of these sets of data may be matched or independent, depending on whether the same instruments (tests, judges, etc.) are applied to every individual in the group or the instruments to be applied to each individual are selected independently for him. Reliability formulas are developed (for both single observations and for composites ofk observations) for the case where reliability data are matched but decision data are independent. Formulas previously reported in the literature are inappropriate for this case.This paper was prepared under USPHS Grant M-1839, in which the author is associated with Lee J. Cronbach and Goldine C. Gleser, and is an outgrowth of a more comprehensive conceptualization of reliability being developed in that study. The author is indebted to Dr. Frederic M. Lord for numerous suggestions.An unpublished paper prepared in 1958 entitled A generalization of Kuder-Richardson reliability formula 21.     

The statistical theory of stepped-up reliability coefficients when a test has been divided into several equivalent parts

This paper is a presentation of the statistical sampling theory of stepped-up reliability coefficients when a test has been divided into any number of equivalent parts. Maximum-likelihood estimators of the reliability are obtained and shown to be biased. Their sampling distributions are derived and form the basis of the definition of new unbiased estimators with known sampling distributions. These unbiased estimators have a smaller sampling variance than the maximum-likelihood estimators and are, because of this and some other favorable properties, recommended for general use. On the basis of the variances of the unbiased estimators the gain in accuracy in estimating reliability connected with further division of a test can be expressed explicitly. The limits of these variances and thus the limits of accuracy of estimation are derived. Finally, statistical small sample tests of the reliability coefficient are outlined. This paper also covers the sampling distribution of Cronbach's coefficient alpha.     

The effect of statistical learning on internal stimulus representations: Predictable items are enhanced even when not predicted

Statistical learning is the automatic and unconscious learning of environmental regularities and is a basic mechanism of learning in a variety of human perceptual and cognitive domains. Previous studies have mainly focused on the associative mechanisms of statistical learning. However, an unexplored question is whether the internal representations of individual stimuli are altered as their associations are learned. Using a temporal statistical learning paradigm, we examine this question across three experiments and find clear evidence that the internal representations of individual stimuli are differentially altered according to their degree of temporal predictability. These findings complement previous accounts of statistical learning and reveal an enriched mechanism of human learning, such that learning to associate items also enhances the representations of certain items relative to others.