An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data

Confirmatory factor analysis (CFA) is widely used for examining hypothesized relations among ordinal variables (e.g., Likert-type items). A theoretically appropriate method fits the CFA model to polychoric correlations using either weighted least squares (WLS) or robust WLS. Importantly, this approach assumes that a continuous, normal latent process determines each observed variable. The extent to which violations of this assumption undermine CFA estimation is not well-known. In this article, the authors empirically study this issue using a computer simulation study. The results suggest that estimation of polychoric correlations is robust to modest violations of underlying normality. Further, WLS performed adequately only at the largest sample size but led to substantial estimation difficulties with smaller samples. Finally, robust WLS performed well across all conditions.     

An empirical evaluation of a scale to measure occupational aspiration level

Data from 34,118 American high school students are used to evaluate the Occupational Aspiration Scale (OAS). The OAS is successively reevaluated for each of 16 subsamples (cells) generated by cross-classifying respondents by grade in school (9–12), sex, and socioeconomic status (SES). In each cell the OAS is found to be essentially unifactorial, and that factor is identified as level of occupational aspiration (LOA). The reliability of the OAS is slightly lower among females ($\text{r}$_kk = .681) than among males ($\text{r}$_kk = .756); it does not vary appreciably by grade or SES. The mean scores are lower for youth from low SES families than for those from high SES families, in accord with previous research. Mean OAS differences due to sex and grade are small. No important differences by age, sex, or SES are found in the standard deviations of the test scores. This and previously published data from small, local samples indicate that the reliability and validity of the OAS are sufficient for research on high school youth of both sexes and from both higher and lower SES levels.     

An entropy approach to the scaling of ordinal categorical data

This paper studies the problem of scaling ordinal categorical data observed over two or more sets of categories measuring a single characteristic. Scaling is obtained by solving a constrained entropy model which finds the most probable values of the scales given the data. A Kullback-Leibler statistic is generated which operationalizes a measure for the strength of consistency among the sets of categories. A variety of data of two and three sets of categories are analyzed using the entropy approach.This research was partially supported by the Air Force Office of Scientific Research under grant AFOSR 83-0234. The support by the Air Force through grant AFOSR-83-0234 is gratefully acknowledged. The comments of the editor and referees have been most helpful in improving the paper, and in bringing several additional references to our attention.     

An ordinal evaluation of categorical judgement data by random utilities and a corresponding correlation analysis

Categorical judgement data are analyzed along the lines of random utility theory. A class of orders is introduced (categorical weak orders); their characteristic vectors are regarded as points in a Euclidean space; their convex hull forms a polytope whose facets are fully characterized. This polytope is shown to correspond to an order polytope. Furthermore, its relation to the biorder polytope is pointed out. The convex representations of a given point of the polytope are discussed. The impact of these results on the methods of analyzing data arising from a categorical judgement procedure is outlined. In particular, some consequences are drawn with respect to the usual evaluation of correlations of such data.     

A problem for information theoretic semantics

Information theoretic semantics proposes to construe predicate reference in terms of nomological relations between distal properties and properties of representational mental events. Research on the model has largely concentrated on the problem of choosing the nomological relation in terms of which distal properties are to be singled out. I argue that, in addition to this, an information theoretic account has to provide a specification of which properties of representational mental events will play a role in determining reference,qua bearers of nomological relations. I contend that this task poses a serious additional challenge to the viability of the model.I am indebted to Paul Boghossian, Allan Gibbard, Eric Lormand, Peter Railton, Gideon Rosen, Crispin Wright, and Steve Yablo for their comments on drafts of this paper.     

One-sided nonparametric tests for ordinal data

Baumgartner, Weiss, and Schindler (1998) introduced a novel non-parametric test for the two-sample comparison that is superior to commonly used tests such as the Wilcoxon rank-sum test. A modification of the novel test statistic can be used for one-sided comparisons based on ordinal data. Such comparisons frequently occur in psychological research, and the Wilcoxon test is often recommended for their analysis. Here, the two tests were compared in a simulation study. According to this study the tests have a similar type I error rate, but the modified Baumgartner-Weiss-Schindler test is more powerful than the Wilcoxon test.     

Methods for describing ordinal data with cardinal models

Several methods, both new and old, for describing ordinal data with a cardinal model are discussed in a newly developed and general context which broadens their applicability beyond their traditional use in multidimensional scaling. The relationships between the methods is investigated. It is shown that the two most commonly used methods (Guttman's rank-image principle and Kruskal's least-square monotonic transformation) are the boundary conditions of a newly proposed single parameter family of methods. An additional method is proposed which is shown to yield an equal-density model space. The commonly made distinction between “transformation” methods and “transformation-free” methods is shown to be a pseudodistinction. It is observed that all the methods can be stated as matrix operations on the model space with the conclusion that they try to optimize a linear combination of the model space.     

Analysis of ordinal categorical data with misclassification

We develop a method for the analysis of multivariate ordinal categorical data with misclassification based on the latent normal variable approach. Misclassification arises if a subject has been classified into a category that does not truly reflect its actual state, and can occur with one or more variables. A basic framework is developed to enable the analysis of two types of data. The first corresponds to a single sample that is obtained from a fallible design that may lead to misclassified data. The other corresponds to data that is obtained by double sampling. Double sampling data consists of two parts: a sample that is obtained by classifying subjects using the fallible design only and a sample that is obtained by classifying subjects using both fallible and true designs, which is assumed to have no misclassification. A unified expectation–maximization approach is developed to find the maximum likelihood estimate of model parameters. Simulation studies and examples that are based on real data are used to demonstrate the applicability and practicability of the proposed methods.     

ThurCatD: A tool for analyzing ratings on an ordinal category scale

An algorithm for analyzing ordinal scaling results is described. Frequency data on ordinal categories are modeled for unidimensional psychological attributes according to Thurstone’s judgment scaling model. The algorithm applies maximum likelihood estimation of model parameters. The Cramér-Rao bounds of the standard errors of the estimated parameters are calculated, and a stress measure and a goodness-of-fit measure are supplied.     

Covariance structure analysis of ordinal ipsative data

Data are ipsative if they are subject to a constant-sum constraint for each individual. In the present study, ordinal ipsative data (OID) are defined as the ordinal rankings across a vector of variables. It is assumed that OID are the manifestations of their underlying nonipsative vector y, which are difficult to observe directly. A two-stage estimation procedure is suggested for the analysis of structural equation models with OID. In the first stage, the partition maximum likelihood (PML) method and the generalized least squares (GLS) method are proposed for estimating the means and the covariance matrix of A_cy, where A_c is a known contrast matrix. Based on the joint asymptotic distribution of the first stage estimator and an appropriate weight matrix, the generalized least squares method is used to estimate the structural parameters in the second stage. A goodness-of-fit statistic is given for testing the hypothesized covariance structure. Simulation results show that the proposed method works properly when a sufficiently large sample is available.This research was supported by National Institute on Drug Abuse Grants DA01070 and DA10017. The authors are indebted to Dr. Lee Cooper, Dr. Eric Holman, Dr. Thomas Wickens for their valuable suggestions on this study, and Dr. Fanny Cheung for allowing us to use her CPAI data set in this article. The authors would also like to acknowledge the helpful comments from the editor and the two anonymous reviewers.     

An ordinal coefficient of relational agreement for multiple judges

In a recent article, Fagot proposed a generalized family of coefficients of relational agreement for multiple judges, focusing on the concept of empirically meaningful relationships. In this paper an ordinal coefficient of relational agreement, based on ranking data, is presented as a special case of the generalized family. It is shown that the proposed ordinal coefficient encompasses other ordinal coefficients, such as the Kendall coefficient of concordance, the average Spearman rank-order coefficient, and intraclass correlation based on ranks. It is also shown that the Kendall coefficient of concordance, corrected for chance agreement, is equivalent to the ordinal coefficient proposed in this paper.     

An evaluation of the minimal constraining information during observation for movement reproduction

An important question in the field of imitation is what information is used for movement reproduction. While it is argued that relative motion information is perceived and minimised, direct evidence is lacking. In this experiment relative motion was manipulated to convey a novel kicking action. Twenty-four adults were assigned to one of the three impoverished relative motion display groups showing only the TOE, FOOT or LEG. After practise with partial information, participants watched a full-body display and in a final condition performed the action with an additional context constraint (i.e., a ball). Movement kinematics were collected and difference scores between participants and the model were analysed. The groups did not differ in terms of knee-ankle coordination. For hip-knee, the TOE group performed more like the model than the FOOT and LEG group. When transferred to the full-body display, there were no significant improvements. End-point trajectory information can provide sufficient information to reproduce key characteristics of the movement form.     

DifScal: A tool for analyzing difference ratings on an ordinal category scale

An algorithm for analyzing difference scaling results is described. Frequency data on ordered categories that represent perceived differences for a unidimensional psychological attribute are modeled according to Thurstone’s judgment scaling model. The algorithm applies the gradient method for the maximum likelihood estimation of the model parameters. Two ways to calculate the start configuration for the model parameters are elaborated. The algorithm also provides asymptotic values for the standard errors of the estimates and three measures for the goodness of the model fit. An additional feature of DifScal is that it is suited to analyze incomplete data.     

An evaluation of the perceived stress scale in two clinical populations

The relation of perceived stress and psychosocial dysfunction was examined in two clinical samples identifying stress as a personal problem. Perceived stress was significantly correlated with self-reported negative affect and physical symptoms. Negative and total major life event scores also were significantly correlated with self-reported negative affect and physical symptoms. However, the perceived stress scale and the life events measures were moderately correlated, indicating that the two scales assess different features of the stress experience. A major finding of this study was that perceived stress scores were significantly correlated with affective and physical symptoms even after the variance associated with life events had been partialed out.     

A method of optimal scaling for multivariate ordinal data and its extensions

This paper develops a method of optimal scaling for multivariate ordinal data, in the framework of a generalized principal component analysis. This method yields a multidimensional configuration of items, a unidimensional scale of category weights for each item and, optionally, a multidimensional configuration of subjects. The computation is performed by alternately solving an eigenvalue problem and executing a quasi-Newton projection method. The algorithm is extended for analysis of data with mixed measurement levels or for analysis with a combined weighting of items. Numerical examples and simulations are provided. The algorithm is discussed and compared with some related methods.Earlier results of this research appeared in Saito and Otsu (1983). The authors would like to acknowledge the helpful comments and encouragement of the editor.     

A measure of coherence for human information filters

When an information processing system is faced with an excess of input information the task of selecting the items which are to get immediate processing is frequently assigned to a human being. A quantitative measure of the extent to which a man avoids random activity during such filtering operations is derived in terms of two parameters (normalized overload and correct proportion of selections) which are determined from experimentally available quantities. This coherence measure may be used for studies of random behavior, comparison of rules for selecting items, and perhaps prediction of human performance at filtering tasks.The author is particularly indebted to Dr. Harold Glaser, of the U. S. Naval Research Laboratory, for many helpful suggestions during the development of this measure.