Assessing Approximate Fit in Categorical Data Analysis

A family of Root Mean Square Error of Approximation (RMSEA) statistics is proposed for assessing the goodness of approximation in discrete multivariate analysis with applications to item response theory (IRT) models. The family includes RMSEAs to assess the approximation up to any level of association of the discrete variables. Two members of this family are RMSEA₂, which uses up to bivariate moments, and the full information RMSEA_n. The RMSEA₂ is estimated using the M₂ statistic of Maydeu-Olivares and Joe (2005, 2006), whereas for maximum likelihood estimation, RMSEA_n is estimated using Pearson's X² statistic. Using IRT models, we provide cutoff criteria of adequate, good, and excellent fit using the RMSEA₂. When the data are ordinal, we find a strong linear relationship between the RMSEA₂ and the Standardized Root Mean Squared Residual goodness-of-fit index. We are unable to offer cutoff criteria for the RMSEA_n as its population values decrease as the number of variables and categories increase.