An analysis and synthesis of multiple correspondence analysis,optimal scaling,dual scaling,homogeneity analysis and other methods for quantifying categorical multivariate data

We discuss a variety of methods for quantifying categorical multivariate data. These methods have been proposed in many different countries, by many different authors, under many different names. In the first major section of the paper we analyze the many different methods and show that they all lead to the same equations for analyzing the same data. In the second major section of the paper we introduce the notion of a duality diagram, and use this diagram to synthesize the many superficially different methods into a single method.The ideas in this paper were worked out by the first author, with some suggestions provided by the second. The current version of this paper has evolved from three previous versions, the first two written by the first author.     

Linear structural equations with latent variables

An interdependent multivariate linear relations model based on manifest, measured variables as well as unmeasured and unmeasurable latent variables is developed. The latent variables include primary or residual common factors of any order as well as unique factors. The model has a simpler parametric structure than previous models, but it is designed to accommodate a wider range of applications via its structural equations, mean structure, covariance structure, and constraints on parameters. The parameters of the model may be estimated by gradient and quasi-Newton methods, or a Gauss-Newton algorithm that obtains least-squares, generalized least-squares, or maximum likelihood estimates. Large sample standard errors and goodness of fit tests are provided. The approach is illustrated by a test theory model and a longitudinal study of intelligence.This investigation was supported in part by a Research Scientist Development Award (KO2-DA00017) and a research grant (DA01070) from the U. S. Public Health Service.     

Homogeneity analysis withk sets of variables: An alternating least squares method with optimal scaling features

Homogeneity analysis, or multiple correspondence analysis, is usually applied tok separate variables. In this paper we apply it to sets of variables by using sums within sets. The resulting technique is called OVERALS. It uses the notion of optimal scaling, with transformations that can be multiple or single. The single transformations consist of three types: nominal, ordinal, and numerical. The corresponding OVERALS computer program minimizes a least squares loss function by using an alternating least squares algorithm. Many existing linear and nonlinear multivariate analysis techniques are shown to be special cases of OVERALS. An application to data from an epidemiological survey is presented.This research was partly supported by SWOV (Institute for Road Safety Research) in Leidschendam, The Netherlands.     

Describing the elephant: Structure and function in multivariate data

There is a unity underlying the diversity of models for the analysis of multivariate data. Essentially, they constitute a family models, most generally nonlinear, for structural/functional relations between variables drawn from a behavior domain.     

The choice of constraints in correspondence analysis

A discussion of alternative constraint systems has been lacking in the literature on correspondence analysis and related techniques. This paper reiterates earlier results that an explicit choice of constraints has to be made which can have important effects on the resulting scores. The paper also presents new results on dealing with missing data and probabilistic category assignment.I am most grateful to the following for their helpful comments. Arto Demirjian, Michael Greenacre, Michael Healy, Shizuhiko Nishisato, Roderick Mcdonald, and several anonymous referees.     

Generalized eigenvalue problems with positive semi-definite matrices

In a recent paper in this journal McDonald, Torii, and Nishisato show that generalized eigenvalue problems in which both matrices are singular can sometimes be solved by reducing them to similar problems of smaller order. In this paper a more extensive analysis of such problems is used to sharpen and clarify the results of McDonald, Torii, and Nishisato. Possible extensions are also indicated. The relevant mathematical literature is reviewed briefly.     

Regression estimation and post-stratification in factor analysis

Regression estimation and poststratification are methods used in survey sampling to estimate a population mean, when additional information is available for some auxiliary variables. The extension of these methods to factor analysis is considered. These methods may be used either to improve the efficiency of estimation or to adjust for the effects of nonrandom selection. The estimation procedure may be formulated in a LISREL framework.The author is grateful to the referees for their comments.     

An efficient algorithm for joint correspondence analysis

Joint correspondence analysis is a technique for constructing reduced-dimensional representations of pairwise relationships among categorical variables. The technique was proposed by Greenacre as an alternative to multiple correspondence analysis. Joint correspondence analysis differs from multiple correspondence analysis in that it focuses solely on between-variable relationships. Greenacre described one alternating least-squares algorithm for conducting joint correspondence analysis. Another alternating least-squares algorithm is described in this article. The algorithm is guaranteed to converge, and does so in fewer iterations than does the algorithm proposed by Greenacre. A modification of the algorithm for handling Heywood cases is described. The algorithm is illustrated on two data sets.     

Correspondence analysis used complementary to loglinear analysis

Loglinear analysis and correspondence analysis provide us with two different methods for the decomposition of contingency tables. In this paper we will show that there are cases in which these two techniques can be used complementary to each other. More specifically, we will show that often correspondence analysis can be viewed as providing a decomposition of the difference between two matrices, each following a specific loglinear model. Therefore, in these cases the correspondence analysis solution can be interpreted in terms of the difference between these loglinear models. A generalization of correspondence analysis, recently proposed by Escofier, will also be discussed. With this decomposition, which includes classical correspondence analysis as a special case, it is possible to use correspondence analysis complementary to loglinear analysis in more instances than those described for classical correspondence analysis. In this context correspondence analysis is used for the decomposition of the residuals of specific restricted loglinear models.     

Redundancy analysis for qualitative variables

Redundancy analysis (also called principal components analysis of instrumental variables) is a technique for two sets of variables, one set being dependent of the other. Its aim is maximization of the explained variance of the dependent variables by a linear combination of the explanatory variables. The technique is generalized to qualitative variables; it then gives implicitly a simultaneous optimal scaling of the dependent, qualitative variables. Examples are taken from the Dutch Life Situation Survey 1977, using Satisfaction with Life and Happiness as dependent variables. The analysis leads to one well-being scale, defined by the explanatory variables Marital status, Schooling, Income and Activity.The views expressed in this paper are those of the author and do not necessarily reflect the policies of the Netherlands Central Bureau of Statistics.     

A distance-based variety of nonlinear multivariate data analysis, including weights for objects and variables

In the distance approach to nonlinear multivariate data analysis the focus is on the optimal representation of the relationships between the objects in the analysis. In this paper two methods are presented for including weights in distance-based nonlinear multivariate data analysis. In the first method, weights are assigned to the objects while the second method is concerned with differential weighting of groups of variables. When each analysis variable defines a group the latter method becomes a variable weighting method. For objects the weights are assumed to be given; for groups of variables they may be given, or estimated. These weighting schemes can also be combined and have several important applications. For example, they make it possible to perform efficient analyses of large data sets, to use the distance-based variety of nonlinear multivariate data analysis as an addition to loglinear analysis of multiway contingency tables, and to do stability studies of the solutions by applying the bootstrap on the objects or the variables in the analysis. These and other applications are discussed, and an efficient algorithm is proposed to minimize the corresponding loss function.This study is funded by The Netherlands Organization for Scientific Research (NWO) by grant nr. 030-56403 for the PIONEER project Subject Oriented Multivariate Analysis to the third author.     

Effects on indscal of non-orthogonal perceptions of object space dimensions

A general question is raised concerning the possible consequences of employing the very popular INDSCAL multidimensional scaling model in cases where the assumptions of that model may be violated. Simulated data are generated which violate the INDSCAL assumption that all individuals perceive the dimensions of the common object space to be orthogonal. INDSCAL solutions for these various sets of data are found to exhibit extremely high goodness of fit, but systematically distorted object spaces and negative subject weights. The author advises use of Tucker's three-mode model for multidimensional scaling, which can account for non-orthogonal perceptions of the object space dimensions. It is shown that the INDSCAL model is a special case of the three-mode model.     

The integration of multidimensional scaling and multivariate analysis with optimal transformations

The recent history of multidimensional data analysis suggests two distinct traditions that have developed along quite different lines. In multidimensional scaling (MDS), the available data typically describe the relationships among a set of objects in terms of similarity/dissimilarity (or (pseudo-)distances). In multivariate analysis (MVA), data usually result from observation on a collection of variables over a common set of objects. This paper starts from a very general multidimensional scaling task, defined on distances between objects derived from one or more sets of multivariate data. Particular special cases of the general problem, following familiar notions from MVA, will be discussed that encompass a variety of analysis techniques, including the possible use of optimal variable transformation. Throughout, it will be noted how certain data analysis approaches are equivalent to familiar MVA solutions when particular problem specifications are combined with particular distance approximations.This research was supported by the Royal Netherlands Academy of Arts and Sciences (KNAW). An earlier version of this paper was written during a stay at McGill University in Montréal; this visit was supported by a travel grant from the Netherlands Organization for Scientific Research (NWO). I am grateful to Jim Ramsay and Willem Heiser for their encouragement and helpful suggestions, and to the Editor and referees for their constructive comments.     

Principal component analysis of three-mode data by means of alternating least squares algorithms

A new method to estimate the parameters of Tucker's three-mode principal component model is discussed, and the convergence properties of the alternating least squares algorithm to solve the estimation problem are considered. A special case of the general Tucker model, in which the principal component analysis is only performed over two of the three modes is briefly outlined as well. The Miller & Nicely data on the confusion of English consonants are used to illustrate the programs TUCKALS3 and TUCKALS2 which incorporate the algorithms for the two models described.     

The direct product model for the mtmm matrix parameterized as a second order factor analysis model

The composite direct product model for the multitrait-multimethod matrix is reparameterized as a second-order factor analysis model. This facilitates the use of widely available computer programs such as LISREL and LISCOMP for fitting the model.Bruce Bloxom. Paul Horst and Karl Jöreskog contributed helpful comments to an earlier version of this paper. Their suggestions are gratefully acknowledged.     

An alternative to the methodology for analysis of covariance

A general statistical model for simultaneous analysis of data from several groups is described. The model is primarily designed to be used for the analysis of covariance. The model can handle any number of covariates and criterion variables, and any number of treatment groups. Treatment effects may be assessed when the treatment groups are not randomized. In addition, the model allows for measurement errors in the criterion variables as well as in the covariates. A wide variety of hypotheses concerning the parameters of the model can be tested by means of a large sample likelihood ratio test. In particular, the usual assumptions of ANCOVA may be tested.Research reported in this paper has been partly supported by the Swedish Council for Social Science Research under project Statistical methods for analysis of longitudinal data, project director Karl G. Jöreskog, and partly by the Bank of Sweden Tercentenary Foundation under project Structural Equation Models in the Social Sciences, project director Karl G. Jöreskog.     

External analysis with three-mode principal component models

Through external analysis of two-mode data one attempts to map the elements of one mode (e.g., attributes) as vectors in a fixed space of the elements of the other mode (e.g., stimuli). This type of analysis is extended to three-mode data, for instance, when the ratings are made by more individuals. It is described how alternating least squares algorithms for three-mode principal component analysis (PCA) are adapted to enable external analysis, and it is demonstrated that these techniques are useful for exploring differences in the individuals' mappings of the attribute vectors in the fixed stimulus space. Conditions are described under which individual differences may be ignored. External three-mode PCA is illustrated with data from a person perception experiment, designed after two studies by Rosenberg and his associates whose results were used as external information.We gratefully acknowledge the assistance of Piet Brouwer in implementing the external analysis options in the TUCKALS programs.     

More on EM for ML factor analysis

We address several issues that are raised by Bentler and Tanaka's [1983] discussion of Rubin and Thayer [1982]. Our conclusions are: standard methods do not completely monitor the possible existence of multiple local maxima; summarizing inferential precision by the standard output based on second derivatives of the log likelihood at a maximum can be inappropriate, even if there exists a unique local maximum; EM and LISREL can be viewed as complementary, albeit not entirely adequate, tools for factor analysis.This work was partially supported by the Program Statistics Research Project at Educational Testing Service.     

An individual differences additive model: An alterating least squares method with optimal scaling features

An individual differences additive model is discussed which represents individual differences in additivity by differential weighting of additive factors. A procedure for estimating the model parameters for various data measurement characteristics is developed. The procedure is evaluated using both Monte Carlo and real data. The method is found to be very useful in describing certain types of developmental change in cognitive structure, as well as being numerically robust and efficient.The work reported here was partly supported by Grant A6394 to the first author by the Natural Sciences and Engineering Research Council of Canada.