Coefficient alpha and related internal consistency reliability coefficients

The author studied the conditions under which coefficient alpha and 10 related internal consistency reliability coefficients underestimate the reliability of a measure. Simulated data showed that alpha, though reasonably robust when computed on n components in moderately heterogeneous data, can under certain conditions seriously underestimate the reliability of a measure. Consequently, alpha, when used in corrections for attenuation, can result in nontrivial overestimation of the corrected correlation. Most of the coefficients studied, including lambda 2, did not improve the estimate to any great extent when the data were heterogeneous. The exceptions were stratified alpha and maximal reliability, which performed well when the components were grouped into two subsets, each measuring a different factor, and maximized lambda 4, which provided the most consistently accurate estimate of the reliability in all simulations studied.     

Coefficient alpha and the internal structure of tests

A general formula (α) of which a special case is the Kuder-Richardson coefficient of equivalence is shown to be the mean of all split-half coefficients resulting from different splittings of a test. α is therefore an estimate of the correlation between two random samples of items from a universe of items like those in the test. α is found to be an appropriate index of equivalence and, except for very short tests, of the first-factor concentration in the test. Tests divisible into distinct subtests should be so divided before using the formula. The index , derived from α, is shown to be an index of inter-item homogeneity. Comparison is made to the Guttman and Loevinger approaches. Parallel split coefficients are shown to be unnecessary for tests of common types. In designing tests, maximum interpretability of scores is obtained by increasing the first-factor concentration in any separately-scored subtest and avoiding substantial group-factor clusters within a subtest. Scalability is not a requisite. The assistance of Dora Damrin and Willard Warrington is gratefully acknowledged. Miss Damrin took major responsibility for the empirical studies reported. This research was supported by the Bureau of Research and Service, College of Education.     

On the reliability of a weighted composite

A general formula for the reliability of a weighted composite has been derived by which that reliability can be estimated from a knowledge of the weights whatever their source, reliabilities, dispersions, and intercorrelations of the components. The Spearman-Brown formula has been shown to be a special case of the general statement. The effect of the internal consistency or intercorrelation of the components has been investigated and the conditions defining the set of weights yielding maximum reliability shown to be that the weight of a component is proportional to the sum of its intercorrelations with the remaining components and inversely proportional to its error variance.     

A note on using stratified alpha to estimate the composite reliability of a test composed of interrelated nonhomogeneous items

The relationship between stratified alpha (alpha(s)) and the reliability of a test composed of interrelated nonhomogeneous items is examined. It is mathematically demonstrated that when there is congeneric equivalence within the strata or subtests, the difference between the coefficients is a function of the variances of the loadings within the strata. When the items within each stratum are essentially tau equivalent, these variances are 0, and alpha(s) and true reliability are equal, provided errors of measurement are uncorrelated. If errors of measurement are positively correlated and there is essential tau equivalence within strata, stratified alpha will overestimate reliability. These findings indicate that recent studies involving stratified alpha (A. Kamata, A. Turhan, & E. Darandari, 2003; H. G. Osburn, 2000) need to be interpreted with some degree of caution. Nevertheless, the hypothetical population data presented in this article suggest that under certain circumstances, stratified alpha can be considerably greater than alpha and closer to the true reliability. Because the former is easily computed, it is recommended that with stratified tests, practicing researchers should routinely calculate both alpha and stratified alpha coefficients.     

Generalized reliability estimation using repeated measurements

Reliability can be studied in a generalized way using repeated measurements. Linear mixed models are used to derive generalized test–retest reliability measures. The method allows for repeated measures with a different mean structure due to correction for covariate effects. Furthermore, different variance–covariance structures between measurements can be implemented. When the variance structure reduces to a random intercept (compound symmetry), classical methods are recovered. With more complex variance structures (e.g. including random slopes of time and/or serial correlation), time‐dependent reliability functions are obtained. The effect of time lag between measurements on reliability estimates can be evaluated. The methodology is applied to a psychiatric scale for schizophrenia.     

Coefficient alpha: some terminological ambiguities and related misconceptions

Three basic concepts in psychometric theory are homogeneity, consistency, and stability. These terms, however, are not singular in meaning as used in the literature and as a result often lead to confusion. The pluralities involved are discussed and alternatives are recommended.     

A note on using alpha and stratified alpha to estimate the reliability of a test composed of item parcels

Several authors have suggested that prior to conducting a confirmatory factor analysis it may be useful to group items into a smaller number of item ‘parcels’ or ‘testlets’. The present paper mathematically shows that coefficient alpha based on these parcel scores will only exceed alpha based on the entire set of items if W, the ratio of the average covariance of items between parcels to the average covariance of items within parcels, is greater than unity. If W is less than unity, however, and errors of measurement are uncorrelated, then stratified alpha will be a better lower bound to the reliability of a measure than the other two coefficients. Stratified alpha are also equal to the true reliability of a test when items within parcels are essentially tau‐equivalent if one assumes that errors of measurement are not correlated.     

A unifying expression for the maximal reliability of a linear composite

A formally simple expression for the maximal reliability of a linear composite is provided, its theoretical implications and its relation to existing results for reliability are discussed.     

The factor structure and composite reliability of the Rosenberg Self-Esteem Scale among ex-prisoners

The aim of this study was to examine the factor structure and composite reliability of the Rosenberg Self-Esteem Scale (RSES) using a sample of 669 ex-prisoners identified in the National Survey of American Life. Six distinct factor models, with uncorrelated measurement error terms, were specified and tested using confirmatory factor analysis (CFA). Results indicated that the two-factor model consisting of positive and negative latent variables provided a better fit to the data than the alternative models. Moreover, only positive self-esteem was a significant predictor of recidivism. Composite reliability indicated that the two factors were measured with very good reliability. The results consequently provide additional support for a two-dimensional model of the RSES within offender populations.     

A reliability generalization study of coefficient alpha for the UCLA loneliness scale

Loneliness is a psychological construct that has been reported in a variety of populations and associated with a number of other negative psychological problems. This study was an examination of coefficient alpha of a prominent measure of loneliness: the University of California, Los Angeles (UCLA) Loneliness Scale (Russell, Peplau, & Cutrona, 1980; Russell, 1996). We utilized reliability generalization to provide an aggregate estimate of the reliability of the scale over time and in a variety of populations as well as to assess and identify sampling and demographic characteristics associated with variability in coefficient alpha. Of the 213 studies examined, 80 had reported alpha estimates, and we used them in this analysis. We discuss conditions associated with variability in coefficient alpha along with pertinent implications for practice and future research.     

Methods for averaging alpha coefficients in reliability generalization studies

The reliability generalization (RG) approach is a kind of meta-analysis that aims to statistically integrate a set of independent reliability coefficients obtained in several applications of a test, with the purpose of characterizing the measurement error and determining which factors related to the studies and samples can explain its variability. Diverse procedures have been proposed in the literature for averaging a set of independent alpha coefficients, and there is no consensus about which methods are best. Here, we present the results of a Monte Carlo simulation study, comparing the performance of twelve procedures proposed in Feldt and Charter, in terms of bias and mean square error. These procedures differ from each other in the transformation (or not) of the coefficients, and in the application or not of a weighting scheme based on sample size. Our results recommend using weighted methods in contrast to unweighted ones, and transforming the coefficients by the Hakstian and Whalen's proposal or by the proposal based on the square root of the inverse alpha coefficient. Lastly, we discuss the relations between the diverse procedures for averaging alpha coefficients with fixed-effects, random-effects, and varying coefficients models.     

The relationship between unstandardized and standardized alpha, true reliability, and the underlying measurement model

Popular computer programs print 2 versions of Cronbach's alpha: unstandardized alpha, α(Σ), based on the covariance matrix, and standardized alpha, α(R), based on the correlation matrix. Sources that accurately describe the theoretical distinction between the 2 coefficients are lacking, which can lead to the misconception that the differences between α(R) and α(Σ) are unimportant and to the temptation to report the larger coefficient. We explore the relationship between α(R) and α(Σ) and the reliability of the standardized and unstandardized composite under 3 popular measurement models; we clarify the theoretical meaning of each coefficient and conclude that researchers should choose an appropriate reliability coefficient based on theoretical considerations. We also illustrate that α(R) and α(Σ) estimate the reliability of different composite scores, and in most cases cannot be substituted for one another.     

Cronbach alpha reliability and concurrent validity of the Collett-Lester Fear of Death Scale in a Turkish sample

In a sample of 100 Turkish undergraduates, the Collett-Lester Fear of Death Scale had adequate interitem reliability and adequate concurrent validity with Templer's Death Anxiety Scale.     

Short measurements of personality - Validity and reliability of the GSOEP Big Five Inventory (BFI-S)

In the last decade, an upward trend in the use of short measurements for personality can be observed. The goal of this study was to explore the psychometric characteristics of the GSOEP Big Five Inventory (BFI-S; Gerlitz & Schupp, 2005), a 15-item instrument. We compared the BFI-S with the NEO-PI-R ( [Costa and McCrae, 1992a] and [Costa and McCrae, 1992b]) in a sample of 598 German adults (mean age = 42 years). Despite shortcomings for Agreeableness, the short scales generally showed acceptable levels of: (1) internal consistency, (2) stability over a period of 18 months, (3) convergent validity in relation to the NEO-PI-R and (4) discriminant validity. We conclude that in research settings with a pronounced need for parsimony, the BFI-S offers a sufficient level of utility.     

Validity and reliability of electroencephalographic frontal alpha asymmetry and frontal midline theta as biomarkers for depression

Electroencephalographic (EEG) frontal alpha asymmetry (FAA) and frontal midline (FM) theta have been suggested as biomarkers for depression and anxiety, but have mostly been assessed in small and non‐clinical studies. In a clinical sample of 79 adults with depression (ICD‐10: F32), resting EEG and scales of depression (MADRS) and anxiety (HADS‐A) were measured at intake and after 3 months. FAA and FM theta values were referenced to a normative population database. Internal consistency, test‐retest reliability, and correlations with psychiatric tests were examined. Reliability was sufficient. However, FAA and FM theta values were close to the general population, and correlations with psychiatric tests were mostly small and non‐significant, with the exception of FAA on F7–F8 z‐scores and HADS‐A. We conclude that the validity of FAA and FM theta and therefore their potential as biomarkers for depression and anxiety remain unclear.     

The reliability of aggregated measurements: Lessons to be learned from psychometric theory

In a recent article, H. D. Day, D. Marshall, B. Hamilton, and J. Christy (1983, Journal of Research in Personality, 17, 97–109) have reported on a series of simulation studies designed to investigate properties of various coefficients of behavior stability as applied to cross-situational data. The results of their research have prompted them to suggest precautions regarding the measurement of behavioral regularity with conventional indices of response consistency. A critical review of their paper reveals aspects of the data, procedures, and interpretations of Day et al. (1983) which undermine many of their conclusions. Notable problems include (a) inappropriate comparisons of internal consistency reliability coefficients based on nonparallel analyses, (b) anomalous data which may be due to errors of rounding or recording, and (c) the failure to recognize certain assumptions underlying reliability estimation. More importantly, it is shown that the investigation of Day et al. (1983) represents primarily an empirical verification of several measurement principles for which algebraic proofs are amply documented.     

Sample homogeneity and coefficient alpha: comment on MJI and Alkhateeb's reliability generalization study and a theoretical clarification

Reliability generalization studies are increasingly frequently reported in the literature, including Mji and Alkhateeb's 2005 study of the Conceptions of Mathematics Questionnaire. The present article comments on an issue of reliability generalization as used in that study and clarifies a theoretical point regarding the meaning of coefficient alpha and what data features tend to influence it as a measure of score reliability.