Principal component analysis with external information on both subjects and variables

A method for structural analysis of multivariate data is proposed that combines features of regression analysis and principal component analysis. In this method, the original data are first decomposed into several components according to external information. The components are then subjected to principal component analysis to explore structures within the components. It is shown that this requires the generalized singular value decomposition of a matrix with certain metric matrices. The numerical method based on the QR decomposition is described, which simplifies the computation considerably. The proposed method includes a number of interesting special cases, whose relations to existing methods are discussed. Examples are given to demonstrate practical uses of the method.The work reported in this paper was supported by grant A6394 from the Natural Sciences and Engineering Research Council of Canada to the first author. Thanks are due to Jim Ramsay, Haruo Yanai, Henk Kiers, and Shizuhiko Nishisato for their insightful comments on earlier versions of this paper. Jim Ramsay, in particular, suggested the use of the QR decomposition, which simplified the presentation of the paper considerably.     

A simple multivariate probabilistic model for preferential and triadic choices

Multidimensional probabilistic models of behavior following similarity and choice judgements have proven to be useful in representing multidimensional percepts in Euclidean and non-Euclidean spaces. With few exceptions, these models are generally computationally intense because they often require numerical work with multiple integrals. This paper focuses attention on a particularly general triad and preferential choice model previously requiring the numerical evaluation of a 2n-fold integral, wheren is the number of elements in the vectors representing the psychological magnitudes. Transforming this model to an indefinite quadratic form leads to a single integral. The significance of this form to multidimensional scaling and computational efficiency is discussed.The authors would like to thank Jean-Claude Falmagne and Norman Johnson for suggestions and advice concerning quadratic forms.     

Effects on indscal of non-orthogonal perceptions of object space dimensions

A general question is raised concerning the possible consequences of employing the very popular INDSCAL multidimensional scaling model in cases where the assumptions of that model may be violated. Simulated data are generated which violate the INDSCAL assumption that all individuals perceive the dimensions of the common object space to be orthogonal. INDSCAL solutions for these various sets of data are found to exhibit extremely high goodness of fit, but systematically distorted object spaces and negative subject weights. The author advises use of Tucker's three-mode model for multidimensional scaling, which can account for non-orthogonal perceptions of the object space dimensions. It is shown that the INDSCAL model is a special case of the three-mode model.     

On the factor score controversy

A summary and interpretation of the recent literature on the indeterminacy of factor scores is given in simple terms. A good index of factor score determinacy is the squared multiple correlation of the factor with the observed variables.     

Some simple procedures for handling missing data in multivariate analysis

For analyses with missing data, some popular procedures delete cases with missing values, perform analysis with missing value correlation or covariance matrices, or estimate missing values by sample means. There are objections to each of these procedures. Several procedures are outlined here for replacing missing values by regression values obtained in various ways, and for adjusting coefficients (such as factor score coefficients) when data are missing. None of the procedures are complex or expensive.This research was supported by NIH Special Research Resources Grant RR-3. The author expresses his gratitude to Robert I. Jennrich and the referees for their suggestions.     

Determining the number of components from the matrix of partial correlations

A common problem for both principal component analysis and image component analysis is determining how many components to retain. A number of solutions have been proposed, none of which is totally satisfactory. An alternative solution which employs a matrix of partial correlations is considered. No components are extracted after the average squared partial correlation reaches a minimum. This approach gives an exact stopping point, has a direct operational interpretation, and can be applied to any type of component analysis. The method is most appropriate when component analysis is employed as an alternative to, or a first-stage solution for, factor analysis.     

Maximally correlated orthogonal composites and oblique factor analytic solutions

Kaiser presented a method for finding a set of derived orthogonal variables which correlate maximally with a set of original variables. A simpler, more complete derivation of Kaiser's result is given and compared to related types of transformations. The transformation derived here suggests a direct method for finding the orthogonal factor solution which is maximally similar to a given oblique solution.     

A note on the minres method

This paper discusses the advantages and problems related to factor analysis by minimizing residuals (minres). It is shown that this method fails if the starting point of iterations is not well chosen. A suitable starting point is suggested.     

On a metric,unidimensional unfolding model for attitudinal and developmental data

If stimulus responses are linearly related to squared distances between stimulus scale values and person scores along a latent continuum, (a) the stimulus × stimulus correlation matrix will display a simplex-like pattern, (b) the signs of first-order partial correlations can be specified in an empirically testable manner, and (c) the variables will have a semicircular, two-factor structure. Along the semicircle, variables will be ordered by their positions on the latent dimension. The above results suggest procedures for examining the appropriateness of the model and procedures for ordering the stimuli. Applications to developmental and attitudinal data are discussed. This research was supported by a grant from the Graduate School of the University of Minnesota to the author and by a grant from the U. S. Public Health Service (Grant No. 1-R01-MH27861-01) to Dr. James Rest, Principal Investigator.