Investigating the Performance of Alternate Regression Weights by Studying All Possible Criteria in Regression Models with a Fixed Set of Predictors

We describe methods for assessing all possible criteria (i.e., dependent variables) and subsets of criteria for regression models with a fixed set of predictors, x (where x is an n×1 vector of independent variables). Our methods build upon the geometry of regression coefficients (hereafter called regression weights) in n-dimensional space. For a full-rank predictor correlation matrix, R _xx, of order n, and for regression models with constant R ² (coefficient of determination), the OLS weight vectors for all possible criteria terminate on the surface of an n-dimensional ellipsoid. The population performance of alternate regression weights—such as equal weights, correlation weights, or rounded weights—can be modeled as a function of the Cartesian coordinates of the ellipsoid. These geometrical notions can be easily extended to assess the sampling performance of alternate regression weights in models with either fixed or random predictors and for models with any value of R ². To illustrate these ideas, we describe algorithms and R (R Development Core Team, 2009) code for: (1) generating points that are uniformly distributed on the surface of an n-dimensional ellipsoid, (2) populating the set of regression (weight) vectors that define an elliptical arc in ℝ ⁿ , and (3) populating the set of regression vectors that have constant cosine with a target vector in ℝ ⁿ . Each algorithm is illustrated with real data. The examples demonstrate the usefulness of studying all possible criteria when evaluating alternate regression weights in regression models with a fixed set of predictors.     

On a shortened method of estimation of mental factors by regression

A shortened method of finding the regression for estimation of the scores on mental factors is presented. If the correlations amongn tests are accounted for byr factors, the regression equation involves the inverse of anr ×r matrix instead of the inverse of ann ×n matrix. Whenr is much less thann, there is considerable saving in computational labor. A numerical example is presented to illustrate the method.     

Enumeration of paths in digraphs

The solution of the problem of enumeration of then-paths in a digraph has so far been attempted through an indirect approach of enumerating the redundant chains. The approach has yielded an algorithm for determination of the general formula for the matrix of redundantn-chains and also a partial recurrence formula for the same. This paper presents a direct approach to the problem. It gives a recurrence relation expressing the matrix ofn-paths of a digraph in terms of the matrices of (n – 1)-paths of its first-order subgraphs. The result is exploited to give an algorithm for computing the matrix ofn-paths. The algorithm is illustrated with a 6 × 6 matrix.The writer wishes to express his indebtedness to Dr. A. K. Gayen of Indian Institute of Technology, Kharagpur, for constant guidance and encouragement. Thanks are also due to Dr. S. N. N. Pandit of the Indian Institute of Technology, Kharagpur, and Shri M. T. Subrahmanya of Indian Statistical Institute, Calcutta, for helpful suggestions.     

Generating fixed-length sequences satisfying any givennth-order transition probability matrix

An experimental design involving sequences ofm distinct events can be conceptualized as annthorder transition probability matrix specifying the probabilities with which each of themdistinct events is to follow certainn-grams. This paper describes a general method for constructing sequences of shortest possible length that satisfy any such matrix and presents a computer program that randomly generates such sequences.     

A simplification of a result by zellini on the maximal rank of symmetric three-way arrays

Zellini (1979, Theorem 3.1) has shown how to decompose an arbitrary symmetric matrix of ordern ×n as a linear combination of 1/2n(n+1) fixed rank one matrices, thus constructing an explicit tensor basis for the set of symmetricn ×n matrices. Zellini's decomposition is based on properties of persymmetric matrices. In the present paper, a simplified tensor basis is given, by showing that a symmetric matrix can also be decomposed in terms of 1/2n(n+1) fixed binary matrices of rank one. The decomposition implies that ann ×n ×p array consisting ofp symmetricn ×n slabs has maximal rank 1/2n(n+1). Likewise, an unconstrained INDSCAL (symmetric CANDECOMP/PARAFAC) decomposition of such an array will yield a perfect fit in 1/2n(n+1) dimensions. When the fitting only pertains to the off-diagonal elements of the symmetric matrices, as is the case in a version of PARAFAC where communalities are involved, the maximal number of dimensions can be further reduced to 1/2n(n–1). However, when the saliences in INDSCAL are constrained to be nonnegative, the tensor basis result does not apply. In fact, it is shown that in this case the number of dimensions needed can be as large asp, the number of matrices analyzed.     

The number of paths and cycles in a digraph

An algorithm is presented for constructing from the adjacency matrix of a digraph the matrix of its simplen-sequences. In this matrix, thei, j entry,i j, gives the number of paths of lengthn from a pointv _i to a pointv _j ; the diagonal entryi, i gives the number of cycles of lengthn containingv _i . The method is then generalized to networks—that is, digraphs in which some value is assigned to each line. With this generalized algorithm it is possible, for a variety of value systems, to calculate the values of the paths and cycles of lengthn in a network and to construct its value matrix of simplen-sequences. The procedures for obtaining the two algorithms make use of properties of a line digraph—that is, a derived digraph whose points and lines represent the lines and adjacency of lines of the given digraph.The research reported here was supported by Grant NSF-G-17771 from the National Science Foundation. We wish to thank Professor Frank Harary for suggesting certain ways of improving an earlier draft of this paper.     

Matrix correlation

A correlational measure for ann byp matrixX and ann byq matrixY assesses their relation without specifying either as a fixed target. This paper discusses a number of useful measures of correlation, with emphasis on measures which are invariant with respect to rotations or changes in singular values of either matrix. The maximization of matrix correlation with respect to transformationsXL andYM is discussed where one or both transformations are constrained to be orthogonal. Special attention is focussed on transformations which causeXL andYM to ben bys, wheres may be any number between 1 and min (p, q). An efficient algorithm is described for maximizing the correlation betweenXL andYM where analytic solutions do not exist. A factor analytic example is presented illustrating the advantages of various coefficients and of varying the number of columns of the transformed matrices.This research was supported by grant APA 0320 from the Natural Sciences and Engineering Research Council of Canada. The authors wish to acknowledge valuable discussion of this problem with Jan de Leeuw, University of Leiden.     

Connectivity and generalized cliques in sociometric group structure

By using the concepts of antimetry andn-chain it is possible to define and to investigate some properties of connectivity in a sociometric group. It is shown that the number of elements in a group, the number of antimetries, and the degree of connectivity must satisfy certain inequalities. Using the ideas of connectivity, a generalized concept of clique, called ann-clique, is introduced.n-cliques are shown to have a very close relationship to the existence of cliques in an artificial structure defined on the same set of elements, thus permitting the determination ofn-cliques by means of the same simple matrix procedures used to obtain the clique structures. The presence of two or morem-cliques, wherem is the number of elements in the group, is proved to mean an almost complete splitting of the group.     

Approximating a symmetric matrix

We examine the least squares approximationC to a symmetric matrixB, when all diagonal elements get weightw relative to all nondiagonal elements. WhenB has positivityp andC is constrained to be positive semi-definite, our main result states that, whenw1/2, then the rank ofC is never greater thanp, and whenw1/2 then the rank ofC is at leastp. For the problem of approximating a givenn×n matrix with a zero diagonal by a squared-distance matrix, it is shown that the sstress criterion leads to a similar weighted least squares solution withw=(n+2)/4; the main result remains true. Other related problems and algorithmic consequences are briefly discussed.     

Determining a simple structure when loadings for certain tests are known

A rigorous and an approximate solution are found for the problem: Given a primary trait matrix forn tests andr ₁ traits, and a matrix for the samen tests andr ₂ reference axes, to discover the transformation which will transform the second matrix into the first, or primary trait matrix. Formulas for determining the limits of the effect of using the approximate solution are presented. The method is applied to a set of twenty hypothetical tests, defined by their loadings on four orthogonal primary traits. After factoring the inter-correlations of these variables by Thurstone's centroid method, approximating the diagonals, the original hypothetical matrix is reproduced with a root mean square discrepancy of .014 by assuming as known the primary trait loadings of only the first eight tests. The method is applied to the results of factoring two batteries of 14 tests, having 8 tests in common, to give the factor loadings of the two batteries on the same reference axes. The method provides a means of comparing directly and quantitatively the results of two different factor studies, provided they have tests in common, and of testing the stability of simple structure under changes in the battery. The relations of the method here developed to certain problems in multiple correlation are shown.     

A note on the estimation of the Pareto efficient set for multiobjective matrix permutation problems

There are a number of important problems in quantitative psychology that require the identification of a permutation of the n rows and columns of an n × n proximity matrix. These problems encompass applications such as unidimensional scaling, paired‐comparison ranking, and anti‐Robinson forms. The importance of simultaneously incorporating multiple objective criteria in matrix permutation applications is well recognized in the literature; however, to date, there has been a reliance on weighted‐sum approaches that transform the multiobjective problem into a single‐objective optimization problem. Although exact solutions to these single‐objective problems produce supported Pareto efficient solutions to the multiobjective problem, many interesting unsupported Pareto efficient solutions may be missed. We illustrate the limitation of the weighted‐sum approach with an example from the psychological literature and devise an effective heuristic algorithm for estimating both the supported and unsupported solutions of the Pareto efficient set.     

A multistage ranking model

In this paper, we propose a (n–1)² parameter, multistage ranking model, which represents a generalization of Luce's model. We propose then×n item-rank relative frequency matrix (p-matrix) as a device for summarizing a set of rankings. As an alternative to the traditional maximum likelihood estimation, for the proposed model we suggest a method which estimates the parameters from thep-matrix. An illustrative numerical example is given. The proposed model and its differences from Luce's model are briefly discussed. We also show some specialp-matrix patterns possessed by the Thurstonian models and distance-based models.     

Some properties of the communality in multiple factor theory

Several theorems concerning properties of the communaltiy of a test in the Thurstone multiple factor theory are established. The following theorems are applicable to a battery ofn tests which are describable in terms ofr common factors, with orthogonal reference vectors.1. The communality of a testj is equal to the square of the multiple correlation of testj with ther reference vectors.2. The communality of a testj is equal to the square of the multiple correlation of testj with ther reference vectors and then—1 remaining tests. Corollary: The square of the multiple correlation of a testj with then—1 remaining tests is equal to or less than the communality of testj. It cannot exceed the communality.3. The square of the multiple correlation of a testj with then—1 remaining tests equals the communality of testj if the group of tests containsr statistically independent ests teach with a communality of unity.4. With correlation coefficients corrected for attenuation, when the number of tests increases indefinitely while the rank of the correlational matrix remains unchanged, the communality of a testj equals the square of the multiple correlation of testj with then—1 remaining tests.5. With raw correlation coefficients, it is shown in a special case that the square of the multiple correlation of a testj with then—1 remaining tests approaches the communality of testj as a limit when the number of tests increases indefinitely while the rank of correlational matrix remains the same. This has not yet been proved for the general case.The author wishes to express his appreciation of the encouragement and assistance given him by Dr. L. L. Thurstone.     

Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: A search procedure to locate the greatest lower bound

Finding the greatest lower bound for the reliability of the total score on a test comprisingn non-homogenous items with dispersion matrix Σ _x is equivalent to maximizing the trace of a diagonal matrix Σ _E with elements θ _I , subject to Σ _E and Σ _T =Σ _x − Σ _E being non-negative definite. The casesn=2 andn=3 are solved explicity. A computer search in the space of the θ _i is developed for the general case. When Guttman's λ₄ (maximum split-half coefficient alpha) is not the g.l.b., the maximizing set of θ _i makes the rank of Σ _T less thann − 1. Numerical examples of various bounds are given. Present affiliation of the first author: St. Hild's College of Education, Durham City, England.     

Increased training complexity reduces the effectiveness of brief working memory training: evidence from short-term single and dual n-back training interventions

N-back training has recently come under intense scientific scrutiny due to reports of training-related improvements in general fluid intelligence. As of yet, relatively little is known about the effects of short-term n-back training interventions, however. In a pretest-training-posttest design, we compared brief dual and single n-back training regimen in terms of training gains and transfer effects relative to a passive control group. Transfer effects indicated that, in the short-term, single n-back training may be the more effective training task: At the short training duration we employed, neither training group showed far transfer to specific task switch costs, Stroop inhibition costs or matrix reasoning indexing fluid intelligence. Yet, both types of training resulted in a reduction of general task switch costs indicating improved cognitive control during the sustained maintenance of competing task sets. Single but not dual n-back training additionally yielded near transfer to an untrained working memory updating task.     

Novices,Apprentices, and Mnemonists: Acquiring Expertise with the Phonetic Mnemonic

Three kinds of participants can be identified in research with the phonetic mnemonic—novices, apprentices, and mnemonists, and most research has involved novices. Two experiments investigated whether ordinary college students using the phonetic mnemonic could duplicate two feats of Luria's memorist identified as S: memorizing a 20-digit matrix in 40 seconds, and memorizing a 50-digit matrix in 3 minutes. Experiment 1 was a large-n study in which novices were provided with phonetic keywords along with the matrices; nearly half of the students successfully memorized each matrix with no errors in recall. Experiment 2 was a small-n study in which apprentices were not given the keywords with the matrices; three of the four students also successfully memorized each matrix with perfect recall. The results suggest that ordinary college students are capable of acquiring a level of skilled memory that rivals the abilities that appear to be innate or idiosyncratic in memorists and mnemonists, and that additional research with mnemonic apprentices might be beneficial. © 1997 by John Wiley & Sons, Ltd.     

Back to the basics: Rethinking partial correlation network methodology

The Gaussian graphical model (GGM) is an increasingly popular technique used in psychology to characterize relationships among observed variables. These relationships are represented as elements in the precision matrix. Standardizing the precision matrix and reversing the sign yields corresponding partial correlations that imply pairwise dependencies in which the effects of all other variables have been controlled for. The graphical lasso (glasso) has emerged as the default estimation method, which uses ℓ₁-based regularization. The glasso was developed and optimized for high-dimensional settings where the number of variables (p) exceeds the number of observations (n), which is uncommon in psychological applications. Here we propose to go ‘back to the basics’, wherein the precision matrix is first estimated with non-regularized maximum likelihood and then Fisher Z transformed confidence intervals are used to determine non-zero relationships. We first show the exact correspondence between the confidence level and specificity, which is due to 1 minus specificity denoting the false positive rate (i.e., α). With simulations in low-dimensional settings (p ≪ n), we then demonstrate superior performance compared to the glasso for detecting the non-zero effects. Further, our results indicate that the glasso is inconsistent for the purpose of model selection and does not control the false discovery rate, whereas the proposed method converges on the true model and directly controls error rates. We end by discussing implications for estimating GGMs in psychology.     

Communality of a variable: Formulation by cluster analysis

The communality of a variable represents the degree of its generality acrossn – 1 behaviors. Domain-sampling principles provide a fundamental conception and definition of the communality. This definition may be alternatively stated in eight different ways. Three definitions lead to precise formulas that determine thetrue value of the communality: (i) from thek necessary and sufficient dimensions derived by iterated factoring, (ii) from then – 1 remaining variable-domains, and (iii) fromk' multiple clusters of then variables. Seven definitions provide approximation formulas: (i) one from thek dimensions as initially factored, (ii) one from then – 1 remaining variables, and (iii) five from a single cluster. Rank of the matrix is not a desiratum in some definitions. Using an example designed by Guilford to illustrate multiple-factor analysis, applications of the formulas based on the three precise definitions recover the true communalities, and five approximation formulas each gives values closer than thead hoc estimates usually employed in factor analysis.The writer wishes to express his indebtedness to C. F. Wrigley and H. Kaiser for their many helpful constructive criticisms.     

Matrix reorganization and dynamic programming: Applications to paired comparisons and unidimensional seriation

A recursive dynamic programming strategy is discussed for optimally reorganizing the rows and simultaneously the columns of ann ×n proximity matrix when the objective function measuring the adequacy of a reorganization has a fairly simple additive structure. A number of possible objective functions are mentioned along with several numerical examples using Thurstone's paired comparison data on the relative seriousness of crime. Finally, the optimization tasks we propose to attack with dynamic programming are placed in a broader theoretical context of what is typically referred to as the quadratic assignment problem and its extension to cubic assignment.Partial support for this research was provided by NIJ Grant 80-IJ-CX-0061.