Premature conclusions about the signal-to-noise ratio in structural equation modeling research: A commentary on Yuan and Fang (2023)

In a recent article published in this journal, Yuan and Fang (British Journal of Mathematical and Statistical Psychology, 2023) suggest comparing structural equation modeling (SEM), also known as covariance-based SEM (CB-SEM), estimated by normal-distribution-based maximum likelihood (NML), to regression analysis with (weighted) composites estimated by least squares (LS) in terms of their signal-to-noise ratio (SNR). They summarize their findings in the statement that “[c]ontrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the [SNR].” In our commentary, we show that Yuan and Fang have made several incorrect assumptions and claims. Consequently, we recommend that empirical researchers not base their methodological choice regarding CB-SEM and regression analysis with composites on the findings of Yuan and Fang as these findings are premature and require further research.     

Isotonic ordinal probabilistic models (ISOP)

The concept of an ordinal instrumental probabilistic comparison is introduced. It relies on an ordinal scale given a priori and on the concept of stochastic dominance. It is used to define a weakly independently ordered system, or isotonic ordinal probabilistic (ISOP) model, which allows the construction of separate sample-free ordinal scales on a set of subjects and a set of items. The ISOP-model is a common nonparametric theoretical structure for unidimensional models for quantitative, ordinal and dichotomous variables.Fundamental theorems on dichotomous and polytomous weakly independently ordered systems are derived. It is shown that the raw score system has the same formal properties as the latent system, and therefore the latter can be tested at the observed empirical level.I wish to thank 3 reviewers and 2 editors who contributed a lot to the readability and precision of the article.     

Behavioral coaching in the development of skills in football, gymnastics, and tennis

A review of the literature indicates that methods of skill acquisition based on the operant paradigm have been scientifically validated with many motor behaviors. However, these procedures have been limited to the use of positive reinforcement for correct performance when applied to the acquisition of complex sports skills in natural settings. To find complementary procedures to enhance skill acquisition, a coaching method involving several behavioral techniques was developed that focused on remediation of errors. This coaching method combined the following components: (1) systematic use of verbal instructions and feedback, (2) positive and negative reinforcement, (3) positive practice, and (4) time out. Three sports, football, gymnastics, and tennis, were selected to determine the effectiveness and generality of this behavioral coaching method. A total of 23 male and female subjects, ranging in age from 11 to 35, was included in this study. Baseline data were first collected for each sport under standard coaching conditions. Next, the behavioral coaching method was evaluated depending on the sport in either a multiple baseline or a reversal design. The behavioral dimensions selected were blocking in football; backward walkovers, front hand springs, and reverse kips in gymnastics; and the forehand, backhand, and serve in tennis. Behavioral coaching was immediately effective in increasing the correct execution of complex skills in all three sports. Gains of up to 10 times the baseline performance were achieved in each sport. In football, behavioral coaching resulted in an increase in correct blocking performance from a baseline average of 5% to 51.3%. Gymnasts' performances increased from baseline averages of 2.7% to 52.6% across the three skills. In tennis, standard coaching produced an average of 6% correct performance which increased dramatically under behavioral coaching to 57% across the three strokes. The success of the behavioral coaching package used here suggests that a technology of behavior may offer additional and complementary strategies to the acquisition of motor skills in the natural environment.     

A stochastic model for test-retest correlations

A simple stochastic model is formulated in order to determine the optimal time between the first test and the second test when the test-retest method of assessing reliability is used. A forgetting process and a change in true score process are postulated. The optimal time between tests is derived by maximizing the probability that the respondent has not remembered the response on the first test and has not had a change in true score. The resulting test-retest correlation is then found to be a linear function of the true reliability of the test, where the slope of this function is the key probability of not remembering and having no change in true score. Some numerical examples and suggestions for using the results in empirical studies are given. Specific recommendations are presented for improved design and analysis of intentions data.This research was made possible by a grant from the Center for Food Policy Research, Graduate School of Business, Columbia University, New York, New York, 10027.     

Essential independence and likelihood-based ability estimation for polytomous items

A definition ofessential independence is proposed for sequences of polytomous items. For items satisfying the reasonable assumption that the expected amount of credit awarded increases with examinee ability, we develop a theory ofessential unidimensionality which closely parallels that of Stout. Essentially unidimensional item sequences can be shown to have a unique (up to change-of-scale) dominant underlying trait, which can be consistently estimated by a monotone transformation of the sum of the item scores. In more general polytomous-response latent trait models (with or without ordered responses), anM-estimator based upon maximum likelihood may be shown to be consistent for under essentially unidimensional violations of local independence and a variety of monotonicity/identifiability conditions. A rigorous proof of this fact is given, and the standard error of the estimator is explored. These results suggest that ability estimation methods that rely on the summation form of the log likelihood under local independence should generally be robust under essential independence, but standard errors may vary greatly from what is usually expected, depending on the degree of departure from local independence. An index of departure from local independence is also proposed.This work was supported in part by Office of Naval Research Grant N00014-87-K-0277 and National Science Foundation Grant NSF-DMS-88-02556. The author is grateful to William F. Stout for many helpful comments, and to an anonymous reviewer for raising the questions addressed in section 2. A preliminary version of section 6 appeared in the author's Ph.D. thesis.     

Two simple classes of mastery scores based on the beta-binomial model

Two simple classes of mastery scores which are suitable for hand calculations are presented for beta-binomial test score distributions combined with linear and cubic referral success. The models provide a simple way to explore the consequences of selecting an arbitrary mastery score. Such assessment would be useful whenever the test user is not willing to posta priori a loss ratio, but wishes to look at the various consequences before aiming at a particular score.     

A look,by simulation,at the validity of some asymptotic distribution results for rotated loadings

In the last few years, a number of asymptotic results for the distribution of unrotated and rotated factor loadings have been given. This paper investigates the validity of some of these results based on simulation techniques. In particular, it looks at principal component extraction and quartimax rotation on a problem with 13 variables. The indication is that the asymptotic results are quite good.     

A note on computing Louis’ observed information matrix identity for IRT and cognitive diagnostic models

Using Louis’ formula, it is possible to obtain the observed information matrix and the corresponding large-sample standard error estimates after the expectation–maximization (EM) algorithm has converged. However, Louis’ formula is commonly de-emphasized due to its relatively complex integration representation, particularly when studying latent variable models. This paper provides a holistic overview that demonstrates how Louis’ formula can be applied efficiently to item response theory (IRT) models and other popular latent variable models, such as cognitive diagnostic models (CDMs). After presenting the algebraic components required for Louis’ formula, two real data analyses, with accompanying numerical illustrations, are presented. Next, a Monte Carlo simulation is presented to compare the computational efficiency of Louis’ formula with previously existing methods. Results from these presentations suggest that Louis’ formula should be adopted as a standard method when computing the observed information matrix for IRT models and CDMs fitted with the EM algorithm due to its computational efficiency and flexibility.     

Concise formulas for the standard errors of component loading estimates

Concise formulas for the asymptotic standard errors of component loading estimates were derived. The formulas cover the cases of principal component analysis for unstandardized and standardized variables with orthogonal and oblique rotations. The formulas can be used under any distributions for observed variables as long as the asymptotic covariance matrix for sample covariances/correlations is available. The estimated standard errors in numerical examples were shown to be equivalent to those by the methods using information matrices.The author is indebted to anonymous reviewers for the corrections and suggestions on this study, which have led to improvements of earlier versions of this article.