Premature conclusions about the signal-to-noise ratio in structural equation modeling research: A commentary on Yuan and Fang (2023)

In a recent article published in this journal, Yuan and Fang (British Journal of Mathematical and Statistical Psychology, 2023) suggest comparing structural equation modeling (SEM), also known as covariance-based SEM (CB-SEM), estimated by normal-distribution-based maximum likelihood (NML), to regression analysis with (weighted) composites estimated by least squares (LS) in terms of their signal-to-noise ratio (SNR). They summarize their findings in the statement that “[c]ontrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the [SNR].” In our commentary, we show that Yuan and Fang have made several incorrect assumptions and claims. Consequently, we recommend that empirical researchers not base their methodological choice regarding CB-SEM and regression analysis with composites on the findings of Yuan and Fang as these findings are premature and require further research.     

Shrinkage estimation of linear combinations of true scores

This paper is concerned with combining observed scores from sections of tests. It is demonstrated that in the presence of population information a linear combination of true scores can be estimated more efficiently than by the same linear combination of the observed scores. Three criteria for optimality are discussed, but they yield the same solution which can be described and motivated as a multivariate shrinkage estimator.Input from Eric Bradlow, Charles Lewis, and Linda Zeger is acknowledged. Research for this paper was funded by the Program Research Council (ETS). Suggestions of the Editor and of anonymous referees were instrumental in several improvements of the paper.     

The effect of additional variables on factor indeterminacy in models with a single common factor

Determinate solutions for the indeterminate common factor ofp variables satisfying the single common factor model are not unique. Therefore an infinite sequence of additional variables that conform jointly with the originalp variables to the original single common factor model does not determine a unique solution for the indeterminate factor of thep variables (although the solution is unique for the factor of the infinite sequence). Other infinite sequences may be found to determine different solutions for the factor of the originalp variables. The paper discusses a number of theorems about the effects of additional variables on factor indeterminacy in a model with a single common factor and draws conclusions from them for factor theory in general.     

On the base-free measure of change proposed by Tucker,Damarin and Messick

Tucker, Damarin and Messick proposed a base-free measure of change which involves the computation of residual scores that are uncorrelated with true scores on the pretest. The present note discusses this change measure and demonstrates that, because of an incorrect derivation by the authors, properties they attribute to (the coefficient for the regression of true scores from the second testing on true scores from the first testing) are, in fact, properties ofb, the ordinary regression coefficient. A correct derivation of the initial position—change correlation in terms of is obtained, and Tucker et al.'s discussion of the law of initial values is reconsidered.The preparation of this paper was supported by the Learning Research and Development Center supported in part as a research and development center by funds from the National Institute of Education.     

The effects of primary reward on the I.Q. performance of grade-school children as a function of initial I.Q. level

The effect of candy reward on I.Q. scores was investigated in 72 first- and second-grade children. All subjects were administered Form A of the Peabody Picture Vocabulary Test and, based upon these scores, were divided into three blocks: low, middle, and high. From each block, subjects were randomly assigned to one of three conditions (contingent reward, noncontingent reward, or no reward) that were in effect during administration of Form B. Results showed that candy given contingent upon each correct response increased I.Q. scores for the initially low scoring subjects, but had no influence on the scores of middle and high scoring subjects.     

Comments on “the measurement of factorial indeterminacy”

Guttman's index of indeterminacy (2² – 1) measures the potential amount of uncertainty in picking the right alternative interpretation for a factor. When alternative solutions for a factor are equally likely to be correct, then the squared multiple correlation ² for predicting the factor from the observed variables is the average correlation _AB between independently selected alternative solutionsA andB, while var ( _AB )=(1 – ²)²/s, wheres is the dimensionality of the space in which unpredicted components of alternative solutions are to be found. When alternative solutions for the factor are not equally likely to be chosen, ² is the lower bound for E( _AB ); however, E( _AB ) need not be a modal value in the distribution of _AB . Guttman's index and E( _AB ) measure different aspects of the same indeterminacy problem.     

The Academic Advantage of Devotion: Measuring Variation in the Value of Weekly Worship in Late Adolescence on Educational Attainment Using Propensity Score Matching

This study measures the effect of regular worship attendance at age 17 on total years of schooling by age 25, using data from the National Longitudinal Survey of Youth 1997. Expanding on previous work, this study estimates differences in the impact of worship attendance by race and family income status using propensity score matching. Individuals who frequently attend religious services complete .69 more years of schooling than similar individuals who do not frequently attend services. There are significantly greater returns to attendance for low‐income youth and no significant difference in returns by religious affiliation. These findings suggest that religious observance provides greater benefits for low‐income individuals or perhaps provides resources high‐income individuals have access to elsewhere. Moreover, this study extends previous work by examining a more recent and nationally representative sample of youth and by using methods that allow for greater causal inference.     

Latent variables should remain as such: Evidence from a Monte Carlo study

Use of subject scores as manifest variables to assess the relationship between latent variables produces attenuated estimates. This has been demonstrated for raw scores from classical test theory (CTT) and factor scores derived from factor analysis. Conclusions on scores have not been sufficiently extended to item response theory (IRT) theta estimates, which are still recommended for estimation of relationships between latent variables. This is because IRT estimates appear to have preferable properties compared to CTT, while structural equation modeling (SEM) is often advised as an alternative to scores for estimation of the relationship between latent variables. The present research evaluates the consequences of using subject scores as manifest variables in regression models to test the relationship between latent variables. Raw scores and three methods for obtaining theta estimates were used and compared to latent variable SEM modeling. A Monte Carlo study was designed by manipulating sample size, number of items, type of test, and magnitude of the correlation between latent variables. Results show that, despite the advantage of IRT models in other areas, estimates of the relationship between latent variables are always more accurate when SEM models are used. Recommendations are offered for applied researchers.