A general solution for a class of weakly constrained linear regression problems

This paper contains a globally optimal solution for a class of functions composed of a linear regression function and a penalty function for the sum of squared regression weights. Global optimality is obtained from inequalities rather than from partial derivatives of a Lagrangian function. Applications arise in multidimensional scaling of symmetric or rectangular matrices of squared distances, in Procrustes analysis, and in ridge regression analysis. The similarity of existing solutions for these applications is explained by considering them as special cases of the general class of functions addressed.The author is obliged to Henk Kiers and Willem Heiser for helpful comments.     

A Comparative Investigation of Confidence Intervals for IndependentVariables in Linear Regression

In linear regression, the most appropriate standardized effect size for individual independent variables having an arbitrary metric remains open to debate, despite researchers typically reporting a standardized regression coefficient. Alternative standardized measures include the semipartial correlation, the improvement in the squared multiple correlation, and the squared partial correlation. No arguments based on either theoretical or statistical grounds for preferring one of these standardized measures have been mounted in the literature. Using a Monte Carlo simulation, the performance of interval estimators for these effect-size measures was compared in a 5-way factorial design. Formal statistical design methods assessed both the accuracy and robustness of the four interval estimators. The coverage probability of a large-sample confidence interval for the semipartial correlation coefficient derived from Aloe and Becker was highly accurate and robust in 98% of instances. It was better in small samples than the Yuan-Chan large-sample confidence interval for a standardized regression coefficient. It was also consistently better than both a bootstrap confidence interval for the improvement in the squared multiple correlation and a noncentral interval for the squared partial correlation.     

Model‐related factor score predictors for confirmatory factor analysis

The present paper introduces model‐related (MR) factor score predictors, which reflect specific aspects of confirmatory factor models. The development is mainly based on Schönemann and Steiger's regression score components, but it can also be applied to the factor score coefficients. It is shown that the rotation of factor score predictors has no impact on the covariance matrix reproduced from the corresponding regression component patterns. Thus, regression score components or factor score coefficients can be rotated in order to obtain the required properties. This idea is the basis for MR factor score predictors, which are computed by means of a partial Procrustes rotation towards a target pattern representing the interesting properties of a confirmatory factor model. Two examples demonstrate the construction of MR factor score predictors reflecting specific constraints of a factor model.     

A modified Aitken Pivotal Condensation Method for partial regression and multiple correlation

In this modification of the Aitken Pivotal Condensation Method for obtaining partial regression weights and multiple correlation coefficients, the unit matrices used by Aitken are eliminated, with a resultant decrease in the computational work. Through a simplified arrangement of successive matrices, each one is reduced by the same simple rule. Other modifications also contribute to ease of computation and facilitate checking.     

Regularized Partial and/or Constrained Redundancy Analysis

Methods of incorporating a ridge type of regularization into partial redundancy analysis (PRA), constrained redundancy analysis (CRA), and partial and constrained redundancy analysis (PCRA) were discussed. The usefulness of ridge estimation in reducing mean square error (MSE) has been recognized in multiple regression analysis for some time, especially when predictor variables are nearly collinear, and the ordinary least squares estimator is poorly determined. The ridge estimation method was extended to PRA, CRA, and PCRA, where the reduced rank ridge estimates of regression coefficients were obtained by minimizing the ridge least squares criterion. It was shown that in all cases they could be obtained in closed form for a fixed value of ridge parameter. An optimal value of the ridge parameter is found by G-fold cross validation. Illustrative examples were given to demonstrate the usefulness of the method in practical data analysis situations. We thank Jim Ramsay for his insightful comments on an earlier draft of this paper. The work reported in this paper is supported by Grants 10630 from the Natural Sciences and Engineering Research Council of Canada to the first author.     

Replies to comments on "Which method delivers greater signal-to-noise ratio: Structural equation modelling or regression analysis with weighted composites?" by Yuan and Fang (2023)

Observational data typically contain measurement errors. Covariance-based structural equation modelling (CB-SEM) is capable of modelling measurement errors and yields consistent parameter estimates. In contrast, methods of regression analysis using weighted composites as well as a partial least squares approach to SEM facilitate the prediction and diagnosis of individuals/participants. But regression analysis with weighted composites has been known to yield attenuated regression coefficients when predictors contain errors. Contrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the signal-to-noise ratio (SNR). In particular, the SNR for the regression coefficient via the least squares (LS) method with equally weighted composites is mathematically greater than that by CB-SEM if the items for each factor are parallel, even when the SEM model is correctly specified and estimated by an efficient method. Analytical, numerical and empirical results also show that LS regression using weighted composites performs as well as or better than the normal maximum likelihood method for CB-SEM under many conditions even when the population distribution is multivariate normal. Results also show that the LS regression coefficients become more efficient when considering the sampling errors in the weights of composites than those that are conditional on weights.     

Matrix formulas for part and partial correlation

General formulas for part and partial correlation of any order are derived in terms of multiple correlation coefficients, standard partial regression weights, and validities. The relationship between part correlation and the independent contribution of a predictor is discussed.     

An individual differences model for multiple regression

A model for multiple regression was developed which allows individual differences to emerge empirically. The model encompasses as special cases several of the previous attempts to improve psychological prediction by deviating from the usual linear multiple regression model. The model is tested with both artificial and real data. The results indicate that the model effectively reduces the variance of the error of prediction, and that the weights obtained are stable over different samples, and, to some extent, over different sets of predictors.This article is based upon a thesis submitted in partial fulfillment of the requirements for the doctoral degree at the University of Illinois. The author thanks Professor Ledyard R Tucker who served as committee chairman and offered considerable support and assistance.     

The evaluation of multiple and partial correlation coefficients from the factorial matrix

This paper shows how to compute multiple correlation coefficients, partial correlation coefficients, and regression coefficients from the factorial matrix. Special emphasis is given to computation technique and to approximation formulas. The method is extremely flexible in application since it may be applied to any subset of the original set of observed variables. It is also extremely useful when many of these coefficients are desired.     

Correlates of retrospective early life experience with personality in young Japanese women

To examine the influence of early experiences on the development of personality. We used the Temperament and Character Inventory to assess 98 young women who had first entered a company. Different early experiences were linked, albeit weakly, to test scores. Both partial correlations and multiple regression analyses indicated that Self-directedness was higher if women reported more care of parents. Partial correlation, but not multiple regression analyses, showed that Cooperativeness was greater if women reported more care of parents and less frequent abuse. Reports of early parental loss or negative or positive early life events showed no correlation with scores on any of the Temperament and Character Inventory subscales.     

Dispositional mindfulness as a positive predictor of psychological well-being and the role of the private self-consciousness insight factor

The current study explored relationships among dispositional mindfulness, the private self-consciousness (PrSC) insight factor, and psychological well-being. Several mindfulness studies indicate that dispositional mindfulness is a positive predictor of psychological well-being. In a distinctly different area of consciousness research, Grant, Franklin, and Langford’s (2002) PrSC insight factor shows similar predictive results. Here it is hypothesized that these two seemingly independent dispositional consciousness constructs have overlapping variance and that insight can serve as a partial mediator for dispositional mindfulness when it predicts psychological well-being. Participants were 184 university students who were administered a self-report measure of dispositional mindfulness, insight, and psychological well-being. Correlational analyses revealed that mindfulness and insight were significantly and positively correlated with each other and with psychological well-being. Bootstrap regression analyses supported the model of insight as a partial mediator of the mindfulness–psychological well-being predictive relationship.     

Subvocalization rate versus other predictors of the memory span

Two experiments were performed to determine how accurately the immediate memory span may be predicted from the subject's subvocalization rate, as compared with other subject and stimulus variables. In the first, span and free-recall measures were obtained for 24 subjects, each tested with four types of spoken material (nonsense syllables, random words, fourth-order approximations to English, and normal prose). Silent subvocalization rate, whispered subvocalization rate, speaking rate, and silent reading rate were also measured for each subject. The span (3.9-14 items) showed the highest zero-order correlation with silent subvocalization rate: r = .80. Multiple regression analysis confirmed this as the span's best predictor, partial r = .31. Similar results were found for free-recall scores. In Exp. 2 the digit span was correlated against measures of age, intelligence, subvocalization rate, perceptual speed, and memory search rate, for 40 subjects aged 7 to 17 yr. The best zero-order predictor was age (.62), followed by subvocalization rate (.57) and intelligence (.39). Multiple regression analysis indicated that the span was best predicted by age, partial r = .37, and subvocalization rate, partial r = .29. The span appears more closely related to subjects' internal speech rate than to other cognitive functions.     

A default Bayesian hypothesis test for correlations and partial correlations

We propose a default Bayesian hypothesis test for the presence of a correlation or a partial correlation. The test is a direct application of Bayesian techniques for variable selection in regression models. The test is easy to apply and yields practical advantages that the standard frequentist tests lack; in particular, the Bayesian test can quantify evidence in favor of the null hypothesis and allows researchers to monitor the test results as the data come in. We illustrate the use of the Bayesian correlation test with three examples from the psychological literature. Computer code and example data are provided in the journal archives.     

A generalization of Verhelst's solution for a constrained regression problem in alscal and related MDS-algorithms

Verhelst derived a solution for a constrained regression problem which occurs in the interval measurement application of ALSCAL and related MDS-algorithms. In the present paper it is shown that Verhelst's solution is based on an implicit nonsingularity assumption. A general solution, which contains Verhelst's solution as a special case, is derived by a simple completing-the-squares type approach instead of partial differentiation with a Lagrange multiplier. In addition, this approach permits the identification of a small interval which uniquely contains the optimal value of a parameter needed to solve the special case where Verhelst's solution is valid.The author is obliged to Dirk Knol and Klaas Nevels for helpful comments.     

La Justice Procédurale comme Variable Prévisionnelle de la Motivation Intrinsèque au Travail

Although the importance of intrinsic motivation at work is already known, the relationship between organizational justice and employees’ motivation remains unexplored. Consequently, the purpose of the present study was to examine the predictive role of organizational justice on intrinsic motivation and the mediation effect of fundamental needs satisfaction in the study of organizational justice and employees’ intrinsic motivation. Key variables have been measured with a sample of 273 workers coming from numerous fields of work. Correlational analyses and multiples regression analyses have been conducted and have shown a significant positive relationship between the variables as well as a partial mediation effect of the basic psychological needs’ satisfaction in the relationship between procedural justice and intrinsic motivation. Results show significant positive relations between the three variables of interest and a partial mediation of the basic needs in the relation existing between procedural justice and intrinsic motivation. The role of justice for the development of intrinsic motivation at work is discussed.     

反应风格的测量与统计控制

反应风格是共同方法偏差的主要来源之一。本文首先讨论反应风格的定义和类型,梳理其危害,认为反应风格能使测验分数出现偏差,影响测验信效度分析和变量关系分析,有必要控制其危害。然后介绍了常用的反应风格测量方法,包括计数法和模型法两大类,对测量方法的选择给出了建议,在此基础上,就如何结合反应风格的测量方法与残差回归法、偏相关法来控制反应风格危害给出建议。     

反应风格的测量与统计控制

反应风格是共同方法偏差的主要来源之一。本文首先讨论反应风格的定义和类型,梳理其危害,认为反应风格能使测验分数出现偏差,影响测验信效度分析和变量关系分析,有必要控制其危害。然后介绍了常用的反应风格测量方法,包括计数法和模型法两大类,对测量方法的选择给出了建议,在此基础上,就如何结合反应风格的测量方法与残差回归法、偏相关法来控制反应风格危害给出建议。     

How Many Subjects Does It Take To Do A Regression Analysis

Numerous rules-of-thumb have been suggested for determining the minimum number of subjects required to conduct multiple regression analyses. These rules-of-thumb are evaluated by comparing their results against those based on power analyses for tests of hypotheses of multiple and partial correlations. The results did not support the use of rules-of-thumb that simply specify some constant (e.g., 100 subjects) as the minimum number of subjects or a minimum ratio of number of subjects (N) to number of predictors (m). Some support was obtained for a rule-of-thumb that N ≥ 50 + 8 m for the multiple correlation and N ≥104 + m for the partial correlation. However, the rule-of-thumb for the multiple correlation yields values too large for N when m ≥ 7, and both rules-of-thumb assume all studies have a medium-size relationship between criterion and predictors. Accordingly, a slightly more complex rule-of thumb is introduced that estimates minimum sample size as function of effect size as well as the number of predictors. It is argued that researchers should use methods to determine sample size that incorporate effect size.     

Correlates of organizational commitment: Differences in the public and private sector

Private and public sector managers were compared regarding their job characteristics and organizational commitment. We hypothesized that job characteristics would be positively related to commitment and that sector would moderate that relationship. Moderated regression analyses revealed partial support for both hypotheses. The existence of clarity and challenge were positively related to commitment. Job characteristics demonstrated a stronger relationship with commitment among private sector managers. Explanations for these findings and directions for future research are discussed.     

CHull: A generic convex-hull-based model selection method

When analyzing data, researchers are often confronted with a model selection problem (e.g., determining the number of components/factors in principal components analysis [PCA]/factor analysis or identifying the most important predictors in a regression analysis). To tackle such a problem, researchers may apply some objective procedure, like parallel analysis in PCA/factor analysis or stepwise selection methods in regression analysis. A drawback of these procedures is that they can only be applied to the model selection problem at hand. An interesting alternative is the CHull model selection procedure, which was originally developed for multiway analysis (e.g., multimode partitioning). However, the key idea behind the CHull procedure—identifying a model that optimally balances model goodness of fit/misfit and model complexity—is quite generic. Therefore, the procedure may also be used when applying many other analysis techniques. The aim of this article is twofold. First, we demonstrate the wide applicability of the CHull method by showing how it can be used to solve various model selection problems in the context of PCA, reduced K-means, best-subset regression, and partial least squares regression. Moreover, a comparison of CHull with standard model selection methods for these problems is performed. Second, we present the CHULL software, which may be downloaded from http://ppw.kuleuven.be/okp/software/CHULL/, to assist the user in applying the CHull procedure.