Cluster differences scaling with a within-clusters loss component and a fuzzy successive approximation strategy to avoid local minima

Cluster differences scaling is a method for partitioning a set of objects into classes and simultaneously finding a low-dimensional spatial representation ofK cluster points, to model a given square table of dissimilarities amongn stimuli or objects. The least squares loss function of cluster differences scaling, originally defined only on the residuals of pairs of objects that are allocated to different clusters, is extended with a loss component for pairs that are allocated to the same cluster. It is shown that this extension makes the method equivalent to multidimensional scaling with cluster constraints on the coordinates. A decomposition of the sum of squared dissimilarities into contributions from several sources of variation is described, including the appropriate degrees of freedom for each source. After developing a convergent algorithm for fitting the cluster differences model, it is argued that the individual objects and the cluster locations can be jointly displayed in a configuration obtained as a by-product of the optimization. Finally, the paper introduces a fuzzy version of the loss function, which can be used in a successive approximation strategy for avoiding local minima. A simulation study demonstrates that this strategy significantly outperforms two other well-known initialization strategies, and that it has a success rate of 92 out of 100 in attaining the global minimum.     

Upper bounds for Kruskal's stress

In this paper the relationships between the two formulas for stress proposed by Kruskal in 1964 are studied. It is shown that stress formula one has a system of nontrivial upper bounds. It seems likely that minimization of this loss function will be liable to produce solutions for which this upper bound is small. These are regularly shaped configurations. Even though stress formula two yields less equivocal results, it seems to be expected that minimization of this loss function will tend to produce configurations in which the points are clumped. These results give no clue as to which of the two loss functions is to be preferred.This study has been supported by the Nederlandse Organisatie voor Zuiver-Wetenschappelijk Onderzoek (Netherlands Organization for the Advancement of Pure Research), under grant 56-146.Comments by Willem Heiser and Frank Critichley have been very helpful.The second author presently is employed by the Netherlands Central Bureau of Statistics, Voorburg.     

Confidence regions for INDSCAL using the jackknife and bootstrap techniques

Bootstrap and jackknife techniques are used to estimate ellipsoidal confidence regions of group stimulus points derived from INDSCAL. The validity of these estimates is assessed through Monte Carlo analysis. Asymptotic estimates of confidence regions based on a MULTISCALE solution are also evaluated. Our findings suggest that the bootstrap and jackknife techniques may be used to provide statements regarding the accuracy of the relative locations of points in space. Our findings also suggest that MULTISCALE asymptotic estimates of confidence regions based on small samples provide an optimistic view of the actual statistical reliability of the solution. The authors wish to thank Geert DeSoete, Richard A. Harshman, William Heiser, Jon Kettenring, Joseph B. Kruskal, Jacqueline Meulman, James O. Ramsay, John W. Tukey, Paul A. Tukey, and Mike Wish. Sharon L. Weinberg is a consultant at AT&T Bell Laboratories, Murray Hill, New Jersey 07974.     

Sequential estimation in multidimensional scaling

The concept of sequential estimation is introduced in multidimensional scaling (MDS). The sequential estimation method developed in this paper refers to continually updating estimates of a configuration as new observations are added. This method has a number of advantages, such as a locally optimal design of the experiment can be easily constructed, and dynamic experimentation is made possible. Using artificial data, the performance of our sequential method is illustrated.We are indebted to anonymous reviewers for their suggestions. In addition, we thank Dr. Frank Critchley for his helpful comments on our Q/S algorithm.     

Spatial versus tree representations of proximity data

In this paper we investigated two of the most common representations of proximities, two-dimensional euclidean planes and additive trees. Our purpose was to develop guidelines for comparing these representations, and to discover properties that could help diagnose which representation is more appropriate for a given set of data. In a simulation study, artificial data generated either by a plane or by a tree were scaled using procedures for fitting either a plane (KYST) or a tree (ADDTREE). As expected, the appropriate model fit the data better than the inappropriate model for all noise levels. Furthermore, the two models were roughly comparable: for all noise levels, KYST accounted for plane data about as well as ADDTREE accounted for tree data. Two properties of the data proved useful in distinguishing between the models: the skewness of the distribution of distances, and the proportion of elongated triangles, which measures departures from the ultrametric inequality, Applications of KYST and ADDTREE to some twenty sets of real data, collected by other investigators, showed that most of these data could be classified clearly as favoring either a tree or a two-dimensional representation.A portable PASCAL program implementing the Sattath and Tversky [1977] ADDTREE algorithm is available from J. Corter, Department of Psychology, Stanford University, Stanford, California 94305.     

Principal component analysis with external information on both subjects and variables

A method for structural analysis of multivariate data is proposed that combines features of regression analysis and principal component analysis. In this method, the original data are first decomposed into several components according to external information. The components are then subjected to principal component analysis to explore structures within the components. It is shown that this requires the generalized singular value decomposition of a matrix with certain metric matrices. The numerical method based on the QR decomposition is described, which simplifies the computation considerably. The proposed method includes a number of interesting special cases, whose relations to existing methods are discussed. Examples are given to demonstrate practical uses of the method.The work reported in this paper was supported by grant A6394 from the Natural Sciences and Engineering Research Council of Canada to the first author. Thanks are due to Jim Ramsay, Haruo Yanai, Henk Kiers, and Shizuhiko Nishisato for their insightful comments on earlier versions of this paper. Jim Ramsay, in particular, suggested the use of the QR decomposition, which simplified the presentation of the paper considerably.     

A general solution for a class of weakly constrained linear regression problems

This paper contains a globally optimal solution for a class of functions composed of a linear regression function and a penalty function for the sum of squared regression weights. Global optimality is obtained from inequalities rather than from partial derivatives of a Lagrangian function. Applications arise in multidimensional scaling of symmetric or rectangular matrices of squared distances, in Procrustes analysis, and in ridge regression analysis. The similarity of existing solutions for these applications is explained by considering them as special cases of the general class of functions addressed.The author is obliged to Henk Kiers and Willem Heiser for helpful comments.     

Was euclid an unnecessarily sophisticated psychologist?

A survey of the current state of multidimensional scaling using the city-block metric is presented. Topics include substantive and theoretical issues, recent algorithmic developments and their implications for seemingly straightforward analyses, isometries with other metrics, links to graph-theoretic models, and future prospects.Presented as the 1991 Psychometric Society Presidential Address. I am indebted to Doug Carroll, John Daws, Jan de Leeuw, Geert De Soete, Wayne DeSarbo, Eric Holman, Larry Hubert, Chingis Izmailov, Joe Kruskal, Rob Nosofsky, Akinori Okada, Roger Shepard, Auke Tellegen, and Wijbrandt van Schuur for many helpful comments on this research and to Yuko Minowa for bibliographic assistance. Doug Carroll's comments on an early draft were especially useful. Parts of this research were supported by a grant from AT&T to the University of Illinois. Some of the computational results reported here were obtained during the early 1970's at the Institute for Mathematical Studies in the Social Sciences at Stanford. Grateful acknowledgment is made to the Institute's staff and its Co-Directors, Richard C. Atkinson and Patrick Suppes.     

A simple multivariate probabilistic model for preferential and triadic choices

Multidimensional probabilistic models of behavior following similarity and choice judgements have proven to be useful in representing multidimensional percepts in Euclidean and non-Euclidean spaces. With few exceptions, these models are generally computationally intense because they often require numerical work with multiple integrals. This paper focuses attention on a particularly general triad and preferential choice model previously requiring the numerical evaluation of a 2n-fold integral, wheren is the number of elements in the vectors representing the psychological magnitudes. Transforming this model to an indefinite quadratic form leads to a single integral. The significance of this form to multidimensional scaling and computational efficiency is discussed.The authors would like to thank Jean-Claude Falmagne and Norman Johnson for suggestions and advice concerning quadratic forms.     

Effects on indscal of non-orthogonal perceptions of object space dimensions

A general question is raised concerning the possible consequences of employing the very popular INDSCAL multidimensional scaling model in cases where the assumptions of that model may be violated. Simulated data are generated which violate the INDSCAL assumption that all individuals perceive the dimensions of the common object space to be orthogonal. INDSCAL solutions for these various sets of data are found to exhibit extremely high goodness of fit, but systematically distorted object spaces and negative subject weights. The author advises use of Tucker's three-mode model for multidimensional scaling, which can account for non-orthogonal perceptions of the object space dimensions. It is shown that the INDSCAL model is a special case of the three-mode model.