Premature conclusions about the signal-to-noise ratio in structural equation modeling research: A commentary on Yuan and Fang (2023)

In a recent article published in this journal, Yuan and Fang (British Journal of Mathematical and Statistical Psychology, 2023) suggest comparing structural equation modeling (SEM), also known as covariance-based SEM (CB-SEM), estimated by normal-distribution-based maximum likelihood (NML), to regression analysis with (weighted) composites estimated by least squares (LS) in terms of their signal-to-noise ratio (SNR). They summarize their findings in the statement that “[c]ontrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the [SNR].” In our commentary, we show that Yuan and Fang have made several incorrect assumptions and claims. Consequently, we recommend that empirical researchers not base their methodological choice regarding CB-SEM and regression analysis with composites on the findings of Yuan and Fang as these findings are premature and require further research.     

Shrinkage estimation of linear combinations of true scores

This paper is concerned with combining observed scores from sections of tests. It is demonstrated that in the presence of population information a linear combination of true scores can be estimated more efficiently than by the same linear combination of the observed scores. Three criteria for optimality are discussed, but they yield the same solution which can be described and motivated as a multivariate shrinkage estimator.Input from Eric Bradlow, Charles Lewis, and Linda Zeger is acknowledged. Research for this paper was funded by the Program Research Council (ETS). Suggestions of the Editor and of anonymous referees were instrumental in several improvements of the paper.     

The effect of additional variables on factor indeterminacy in models with a single common factor

Determinate solutions for the indeterminate common factor ofp variables satisfying the single common factor model are not unique. Therefore an infinite sequence of additional variables that conform jointly with the originalp variables to the original single common factor model does not determine a unique solution for the indeterminate factor of thep variables (although the solution is unique for the factor of the infinite sequence). Other infinite sequences may be found to determine different solutions for the factor of the originalp variables. The paper discusses a number of theorems about the effects of additional variables on factor indeterminacy in a model with a single common factor and draws conclusions from them for factor theory in general.     

On the base-free measure of change proposed by Tucker,Damarin and Messick

Tucker, Damarin and Messick proposed a base-free measure of change which involves the computation of residual scores that are uncorrelated with true scores on the pretest. The present note discusses this change measure and demonstrates that, because of an incorrect derivation by the authors, properties they attribute to (the coefficient for the regression of true scores from the second testing on true scores from the first testing) are, in fact, properties ofb, the ordinary regression coefficient. A correct derivation of the initial position—change correlation in terms of is obtained, and Tucker et al.'s discussion of the law of initial values is reconsidered.The preparation of this paper was supported by the Learning Research and Development Center supported in part as a research and development center by funds from the National Institute of Education.     

The effects of primary reward on the I.Q. performance of grade-school children as a function of initial I.Q. level

The effect of candy reward on I.Q. scores was investigated in 72 first- and second-grade children. All subjects were administered Form A of the Peabody Picture Vocabulary Test and, based upon these scores, were divided into three blocks: low, middle, and high. From each block, subjects were randomly assigned to one of three conditions (contingent reward, noncontingent reward, or no reward) that were in effect during administration of Form B. Results showed that candy given contingent upon each correct response increased I.Q. scores for the initially low scoring subjects, but had no influence on the scores of middle and high scoring subjects.     

Comments on “the measurement of factorial indeterminacy”

Guttman's index of indeterminacy (2² – 1) measures the potential amount of uncertainty in picking the right alternative interpretation for a factor. When alternative solutions for a factor are equally likely to be correct, then the squared multiple correlation ² for predicting the factor from the observed variables is the average correlation _AB between independently selected alternative solutionsA andB, while var ( _AB )=(1 – ²)²/s, wheres is the dimensionality of the space in which unpredicted components of alternative solutions are to be found. When alternative solutions for the factor are not equally likely to be chosen, ² is the lower bound for E( _AB ); however, E( _AB ) need not be a modal value in the distribution of _AB . Guttman's index and E( _AB ) measure different aspects of the same indeterminacy problem.     

Latent variables should remain as such: Evidence from a Monte Carlo study

Use of subject scores as manifest variables to assess the relationship between latent variables produces attenuated estimates. This has been demonstrated for raw scores from classical test theory (CTT) and factor scores derived from factor analysis. Conclusions on scores have not been sufficiently extended to item response theory (IRT) theta estimates, which are still recommended for estimation of relationships between latent variables. This is because IRT estimates appear to have preferable properties compared to CTT, while structural equation modeling (SEM) is often advised as an alternative to scores for estimation of the relationship between latent variables. The present research evaluates the consequences of using subject scores as manifest variables in regression models to test the relationship between latent variables. Raw scores and three methods for obtaining theta estimates were used and compared to latent variable SEM modeling. A Monte Carlo study was designed by manipulating sample size, number of items, type of test, and magnitude of the correlation between latent variables. Results show that, despite the advantage of IRT models in other areas, estimates of the relationship between latent variables are always more accurate when SEM models are used. Recommendations are offered for applied researchers.     

Health Outcomes by Closeness of Sexual Abuse Perpetrator: A Test of Betrayal Trauma Theory

Summary

In this chapter, practicing attorney Brooks Cooper illuminates how the legal tradition of applying different standards to the ad-missibility of expert and lay opinions as evidence in litigation complicates delayed-discovery sexual abuse cases. Admissibility standards have long been a source of controversy, and recent developments in law, such as the pre-trial evidentiary hearings that invoke the “gatekeeper” function of the judge under the standard enunciated in Daubert v. Mer-rell-Dow Pharmaceuticals, have become a favored strategy by the defense in an effort to bar delayed memory testimony by characterizing it as the product of unreliable science. To overcome this litigation hurdle, Mr. Cooper contextualizes the present controversy in the history of scientific evidence admissibility, tracing emerging trends in judicial opinion on scientific evidence from expert witnesses from the 1923 case Frye v. United States through the present, and offers strategies and techniques to maintain admissibility of the evidence before and during all phases of the trial.