A scalar product model for the multidimensional scaling of choice

A multidimensional scaling analysis is presented for replicated layouts of pairwise choice responses. In most applications the replicates will represent individuals who respond to all pairs in some set of objects. The replicates and the objects are scaled in a joint space by means of an inner product model which assigns weights to each of the dimensions of the space. Least squares estimates of the replicates' and objects' coordinates, and of unscalability parameters, are obtained through a manipulation of the error sum of squares for fitting the model. The solution involves the reduction of a three-way least squares problem to two subproblems, one trivial and the other solvable by classical least squares matrix factorization. The analytic technique is illustrated with political preference data and is contrasted with multidimensional unfolding in the domain of preferential choice.The present work was initiated at Oregon Research Institute under National Institute of Mental Health Grant MH 12972. It was reformulated and completed while the first author was a Visiting Research Fellow at Educational Testing Service.Presently at the Department of Mathematics, University of Toronto.     

Indclas: A three-way hierarchical classes model

A three-way three-mode extension of De Boeck and Rosenberg's (1988) two-way two-mode hierarchical classes model is presented for the analysis of individual differences in binary object × attribute arrays. In line with the two-way hierarchical classes model, the three-way extension represents both the association relation among the three modes and the set-theoretical relations among the elements of each model. An algorithm for fitting the model is presented and evaluated in a simulation study. The model is illustrated with data on psychiatric diagnosis. Finally, the relation between the model and extant models for three-way data is discussed.The research reported in this paper was partially supported by NATO (Grant CRG.921321 to Iven Van Mechelen and Seymour Rosenberg).     

Fitting the off-diagonal dedicom model in the least-squares sense by a generalization of the harman and jones minres procedure of factor analysis

Harshman's DEDICOM model providesa framework for analyzing square but asymmetric materices of directional relationships amongn objects or persons in terms of a small number of components. One version of DEDICOM ignores the diagonal entries of the matrices. A straight-forward computational solution for this model is offered in the present paper. The solution can be interpreted as a generalized Minres procedure suitable for handing asymmetric matrices.     

Mapclus: A mathematical programming approach to fitting the adclus model

We present a new algorithm, MAPCLUS (MAthematicalProgrammingCLUStering), for fitting the Shepard-Arabie ADCLUS (forADditiveCLUStering) model. MAPCLUS utilizes an alternating least squares method combined with a mathematical programming optimization procedure based on a penalty function approach, to impose discrete (0,1) constraints on parameters defining cluster membership. This procedure is supplemented by several other numerical techniques (notably a heuristically based combinatorial optimization procedure) to provide an efficient general-purpose computer implemented algorithm for obtaining ADCLUS representations. MAPCLUS is illustrated with an application to one of the examples given by Shepard and Arabie using the older ADCLUS procedure. The MAPCLUS solution uses half as many clusters to achieve nearly the same level of goodness-of-fit. Finally, we consider an extension of the present approach to fitting a three-way generalization of the ADCLUS model, called INDCLUS (INdividualDifferencesCLUStering).We are indebted to Scott A. Boorman, W. K. Estes, J. A. Hartigan, Lawrence J. Hubert, Carol L. Krumhansl, Joseph B. Kruskal, Sandra Pruzansky, Roger N. Shepard, Edward J. Shoben, Sigfrid D. Soli, and Amos Tversky for helpful discussions of this work, as well as the anonymous referees for their suggestions and corrections on an earlier version of this paper. We are also grateful to Pamela Baker and Dan C. Knutson for technical assistance. The research reported here was supported in part by LEAA Grant 78-NI-AX-0142 and NSF Grants SOC76-24512 and SOC76-24394.     

Weighted least squares fitting using ordinary least squares algorithms

A general approach for fitting a model to a data matrix by weighted least squares (WLS) is studied. This approach consists of iteratively performing (steps of) existing algorithms for ordinary least squares (OLS) fitting of the same model. The approach is based on minimizing a function that majorizes the WLS loss function. The generality of the approach implies that, for every model for which an OLS fitting algorithm is available, the present approach yields a WLS fitting algorithm. In the special case where the WLS weight matrix is binary, the approach reduces to missing data imputation.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author.     

Three-Mode Component Analysis with Crisp or Fuzzy Partition of Units

A new methodology is proposed for the simultaneous reduction of units, variables, and occasions of a three-mode data set. Units are partitioned into a reduced number of classes, while, simultaneously, components for variables and occasions accounting for the largest common information for the classification are identified. The model is a constrained three-mode factor analysis and it can be seen as a generalization of the REDKM model proposed by De Soete and Carroll for two-mode data. The least squares fitting problem is mathematically formalized as a constrained problem in continuous and discrete variables. An iterative alternating least squares algorithm is proposed to give an efficient solution to this minimization problem in the crisp and fuzzy classification context. The performances of the proposed methodology are investigated by a simulation study comparing our model with other competing methodologies. Different procedures for starting the proposed algorithm have also been tested. A discussion of some interesting differences in the results follows. Finally, an application to real data illustrates the ability of the proposed model to provide substantive insights into the data complexities.     

Analysis of asymmetry by a slide-vector

The slide-vector scaling model attempts to account for the asymmetry of a proximity matrix by a uniform shift in a fixed direction imposed on a symmetric Euclidean representation of the scaled objects. Although no method for fitting the slide-vector model seems available in the literature, the model can be viewed as a constrained version of the unfolding model, which does suggest one possible algorithm. The slide-vector model is generalized to handle three-way data, and two examples from market structure analysis are presented.     

A generalization of Takane's algorithm for dedicom

An algorithm is described for fitting the DEDICOM model for the analysis of asymmetric data matrices. This algorithm generalizes an algorithm suggested by Takane in that it uses a damping parameter in the iterative process. Takane's algorithm does not always converge monotonically. Based on the generalized algorithm, a modification of Takane's algorithm is suggested such that this modified algorithm converges monotonically. It is suggested to choose as starting configurations for the algorithm those configurations that yield closed-form solutions in some special cases. Finally, a sufficient condition is described for monotonic convergence of Takane's original algorithm.Financial Support by the Netherlands organization for scientific research (NWO) is gratefully acknowledged. The authors are obliged to Richard Harshman.     

Cluster differences scaling with a within-clusters loss component and a fuzzy successive approximation strategy to avoid local minima

Cluster differences scaling is a method for partitioning a set of objects into classes and simultaneously finding a low-dimensional spatial representation ofK cluster points, to model a given square table of dissimilarities amongn stimuli or objects. The least squares loss function of cluster differences scaling, originally defined only on the residuals of pairs of objects that are allocated to different clusters, is extended with a loss component for pairs that are allocated to the same cluster. It is shown that this extension makes the method equivalent to multidimensional scaling with cluster constraints on the coordinates. A decomposition of the sum of squared dissimilarities into contributions from several sources of variation is described, including the appropriate degrees of freedom for each source. After developing a convergent algorithm for fitting the cluster differences model, it is argued that the individual objects and the cluster locations can be jointly displayed in a configuration obtained as a by-product of the optimization. Finally, the paper introduces a fuzzy version of the loss function, which can be used in a successive approximation strategy for avoiding local minima. A simulation study demonstrates that this strategy significantly outperforms two other well-known initialization strategies, and that it has a success rate of 92 out of 100 in attaining the global minimum.     

Component models for three-way data: An alternating least squares algorithm with optimal scaling features

A review of the existing techniques for the analysis of three-way data revealed that none were appropriate to the wide variety of data usually encountered in psychological research, and few were capable of both isolating common information and systematically describing individual differences. An alternating least squares algorithm was proposed to fit both an individual difference model and a replications component model to three-way data which may be defined at the nominal, ordinal, interval, ratio, or mixed measurement level; which may be discrete or continuous; and which may be unconditional, matrix conditional, or row conditional. This algorithm was evaluated by a Monte Carlo study. Recovery of the original information was excellent when the correct measurement characteristics were assumed. Furthermore, the algorithm was robust to the presence of random error. In addition, the algorithm was used to fit the individual difference model to a real, binary, subject conditional data set. The findings from this application were consistent with previous research in the area of implicit personality theory and uncovered interesting systematic individual differences in the perception of political figures and roles.This paper is part of a Thesis performed by Richard Sands under the direction of Forrest Young at the L. L. Thurstone Psychometric Laboratory, University of North Carolina at Chapel Hill. Thanks are extended to Drs. Charles Schmidt and Andrea Sedlak for the use of their political role data set.     

A reformulation of the general Euclidean model for the external analysis of preference data

This paper discusses least squares methods for fitting a reformulation of the general Euclidean model for the external analysis of preference data. The reformulated subject weights refer to a common set of reference vectors for all subjects and hence are comparable across subjects. If the rotation of the stimulus space is fixed, the subject weight estimates in the model are uniquely determined. Weight estimates can be guaranteed nonnegative. While the reformulation is a metric model for single stimulus data, the paper briefly discusses extensions to nonmetric, pairwise, and logistic models. The reformulated model is less general than Carroll's earlier formulation.The author is grateful to Christopher J. Nachtsheim for his helpful suggestions.     

A general solution of the weighted orthonormal procrustes problem

A general solution for weighted orthonormal Procrustes problem is offered in terms of the least squares criterion. For the two-demensional case. this solution always gives the global minimum; for the general case, an algorithm is proposed that must converge, although not necessarily to the global minimum. In general, the algorithm yields a solution for the problem of how to fit one matrix to another under the condition that the dimensions of the latter matrix first are allowed to be transformed orthonormally and then weighted differentially, which is the task encountered in fitting analogues of the IDIOSCAL and INDSCAL models to a set of configurations.The authors are grateful to the Editor and the anonymous reviewers for their helpful comments on an earlier draft of this paper.     

Uniqueness proof for a family of models sharing features of Tucker's three-mode factor analysis and PARAFAC/candecomp

Some existing three-way factor analysis and MDS models incorporate Cattell's “Principle of Parallel Proportional Profiles”. These models can—with appropriate data—empirically determine a unique best fitting axis orientation without the need for a separate factor rotation stage, but they have not been general enough to deal with what Tucker has called “interactions” among dimensions. This article presents a proof of unique axis orientation for a considerably more general parallel profiles model which incorporates interacting dimensions. The model, X_k=A^AD_k H^BD_k B', does not assume symmetry in the data or in the interactions among factors. A second proof is presented for the symmetrically weighted case (i.e., where^AD_k=^BD_k). The generality of these models allows one to impose successive restrictions to obtain several useful special cases, including PARAFAC2 and three-way DEDICOM. We want to express appreciation for the contributions of several colleagues: Jos M. F. ten Berge and Henk A. L. Kiers carefully went through more than one version of this article, found an important error, and contributed many improvements. J. Douglas Carroll and Shizuhiko Nishisato acted with unusual editorial preserverance and flexibility, thereby making possible the successful completion of a difficult assessment and revision process. Joseph B. Kruskal has long provided crucial mathematical insights and inspiration to those working in this area, but this is particularly true for us. His contributions to this specific article include early discussion of basic questions and careful examination of some earlier attempted proofs, finding them to be invalid. The anonymous reviewers also made useful suggestions. Some portions of this work were supported in part by a grant from the Natural Sciences and Engineering Research Council of Canada.     

Indclus: An individual differences generalization of the adclus model and the mapclus algorithm

We present a new model and associated algorithm, INDCLUS, that generalizes the Shepard-Arabie ADCLUS (ADditive CLUStering) model and the MAPCLUS algorithm, so as to represent in a clustering solution individual differences among subjects or other sources of data. Like MAPCLUS, the INDCLUS generalization utilizes an alternating least squares method combined with a mathematical programming optimization procedure based on a penalty function approach to impose discrete (0,1) constraints on parameters defining cluster membership. All subjects in an INDCLUS analysis are assumed to have a common set of clusters, which are differentially weighted by subjects in order to portray individual differences. As such, INDCLUS provides a (discrete) clustering counterpart to the Carroll-Chang INDSCAL model for (continuous) spatial representations. Finally, we consider possible generalizations of the INDCLUS model and algorithm.We are indebted to Seymour Rosenberg for making available the data from Rosenberg and Kim [1975]. Also, this work has benefited from the observations of S. A. Boorman, W. S. DeSarbo, G. Furnas, P. E. Green, L. J. Hubert, L. E. Jones, J. B. Kruskal, S. Pruzansky, D. Schmittlein, E. J. Shoben, S. D. Soli, and anonymous referees.This research was supported in part by NSF Grant SES82 00441, LEAA Grant 78-NI-AX-0142, and NSF Grant SES80 04815.     

Three-way metric unfolding via alternating weighted least squares

Three-way unfolding was developed by DeSarbo (1978) and reported in DeSarbo and Carroll (1980, 1981) as a new model to accommodate the analysis of two-mode three-way data (e.g., nonsymmetric proximities for stimulus objects collected over time) and three-mode, three-way data (e.g., subjects rendering preference judgments for various stimuli in different usage occasions or situations). This paper presents a revised objective function and new algorithm which attempt to prevent the common type of degenerate solutions encountered in typical unfolding analysis. We begin with an introduction of the problem and a review of three-way unfolding. The three-way unfolding model, weighted objective function, and new algorithm are presented. Monte Carlo work via a fractional factorial experimental design is described investigating the effect of several data and model factors on overall algorithm performance. Finally, three applications of the methodology are reported illustrating the flexibility and robustness of the procedure.We wish to thank the editor and reviewers for their insightful comments.     

Order‐constrained linear optimization

Despite the fact that data and theories in the social, behavioural, and health sciences are often represented on an ordinal scale, there has been relatively little emphasis on modelling ordinal properties. The most common analytic framework used in psychological science is the general linear model, whose variants include ANOVA, MANOVA, and ordinary linear regression. While these methods are designed to provide the best fit to the metric properties of the data, they are not designed to maximally model ordinal properties. In this paper, we develop an order‐constrained linear least‐squares (OCLO) optimization algorithm that maximizes the linear least‐squares fit to the data conditional on maximizing the ordinal fit based on Kendall's τ. The algorithm builds on the maximum rank correlation estimator (Han, 1987, Journal of Econometrics, 35, 303) and the general monotone model (Dougherty & Thomas, 2012, Psychological Review, 119, 321). Analyses of simulated data indicate that when modelling data that adhere to the assumptions of ordinary least squares, OCLO shows minimal bias, little increase in variance, and almost no loss in out‐of‐sample predictive accuracy. In contrast, under conditions in which data include a small number of extreme scores (fat‐tailed distributions), OCLO shows less bias and variance, and substantially better out‐of‐sample predictive accuracy, even when the outliers are removed. We show that the advantages of OCLO over ordinary least squares in predicting new observations hold across a variety of scenarios in which researchers must decide to retain or eliminate extreme scores when fitting data.     

Nonlinear submodels of orthogonal linear models

A model containing linear and nonlinear parameters (e. g., a spatial multidimensional scaling model) is viewed as a linear model with free and constrained parameters. Since the rank deficiency of the design matrix for the linear model determines the number of side conditions needed to identify its parameters, the design matrix acts as a guide in identifying the parameters of the nonlinear model. Moreover, if the design matrix and the uniqueness conditions constitute anorthogonal linear model, then the associated error sum of squares may be expressed in a form which separates the free and constrained parameters. This immediately provides least squares estimates of the free parameters, while simplifying the least squares problem for those which are constrained. When the least squares estimates for a nonlinear model are obtained in this way,i.e. by conceptualizing it as a submodel, the final error sum of squares for the nonlinear model will be arestricted minimum whenever the side conditions of the model become real restrictions upon its submodel. In this case the design matrix for the embracing orthogonal model serves as a guide in introducing parameters into the nonlinear model as well as in identifying these parameters. The method of overwriting a nonlinear model with an orthogonal linear model is illustrated with two different spatial analyses of a three-way preference table.     

A program for fitting two-phase segmented-curve models with an unknown change point,with an application to the analysis of strategy shifts in a cognitive task

A program is described for fitting a regression model in which the relationship between the dependent and the independent variables is described by two regression equations, one for each of two mutually exclusive ranges of the independent variable. The point at which the change from one equation to the other occurs is often unknown, and thus must be estimated. In cognitive psychology, such models are relevant for studying the phenomenon of strategy shifts. The program uses a (weighted) least squares algorithm to estimate the regression parameters and the change point. The algorithm always finds the global minimum of the error sum of squares. The model is applied to data from a mental-rotation experiment. The program’s estimates of the point at which the strategy shift occurs are compared with estimates obtained from a nonlinear least squares minimization procedure in SPSSX.     

Scene perception: Detecting and judging objects undergoing relational violations

Five classes of relations between an object and its setting can characterize the organization of objects into real-world scenes. The relations are (1) Interposition (objects interrupt their background), (2) Support (objects tend to rest on surfaces), (3) Probability (objects tend to be found in some scenes but not others), (4) Position (given an object is probable in a scene, it often is found in some positions and not others), and (5) familiar Size (objects have a limited set of size relations with other objects). In two experiments subjects viewed brief (150 msec) presentations of slides of scenes in which an object in a cued location in the scene was either in a normal relation to its background or violated from one to three of the relations. Such objects appear to (1) have the background pass through them, (2) float in air, (3) be unlikely in that particular scene, (4) be in an inappropriate position, and (5) be too large or too small relative to the other objects in the scene. In Experiment I, subjects attempted to determine whether the cued object corresponded to a target object which had been specified in advance by name. With the exception of the Interposition violation, violation costs were incurred in that the detection of objects undergoing violations was less accurate and slower than when those same objects were in normal relations to their setting. However, the detection of objects in normal relations to their setting (innocent bystanders) was unaffected by the presence of another object undergoing a violation in that same setting. This indicates that the violation costs were incurred not because of an unsuccessful elicitation of a frame or schema for the scene but because properly formed frames interfered with (or did not facilitate) the perceptibility of objects undergoing violations. As the number of violations increased, target detectability generally decreased. Thus, the relations were accessed from the results of a single fixation and were available sufficiently early during the time course of scene perception to affect the perception of the objects in the scene. Contrary to expectations from a bottom-up account of scene perception, violations of the pervasive physical relations of Support and Interposition were not more disruptive on object detection than the semantic violations of Probability, Position and Size. These are termed semantic because they require access to the referential meaning of the object. In Experiment II, subjects attempted to detect the presence of the violations themselves. Violations of the semantic relations were detected more accurately than violations of Interposition and at least as accurately as violations of Support. As the number of violations increased, the detectability of the incongruities between an object and its setting increased. These results provide converging evidence that semantic relations can be accessed from the results of a single fixation. In both experiments information about Position was accessed at least as quickly as information on Probability. Thus in Experiment I, the interference that resulted from placing a fire hydrant in a kitchen was not greater than the interference from placing it on top of a mail ? in a street scene. Similarly, violations of Probability in Experiment II were not more detectable than violations of Position. Thus, the semantic relations which were accessed included information about the detailed interactions among the objects—information which is more specific than what can be inferred from the general setting. Access to the semantic relations among the entities in a scene is not deferred until the completion of spatial and depth processing and object identification. Instead, an object's semantic relations are accessed simultaneously with its physical relations as well as with its own identification.