The equivalence of regression models using difference scores and models using separate scores for each informant: implications for the study of informant discrepancies

Research on informant discrepancies has increasingly utilized difference scores. This article demonstrates the statistical equivalence of regression models using difference scores (raw or standardized) and regression models using separate scores for each informant to show that interpretations should be consistent with both models. First, regression equations were used to demonstrate that difference score models are equivalent to models using separate scores for each informant. Second, a hypothesis-driven empirical example (218 mother-child dyads, mean age = 11.5 years, 49% female participants, 49% White, 47% African American) was used to provide an illustration of the equivalence of the 2 models. Implications of the equivalence of models using difference scores and models using separate scores for each informant are discussed in terms of the growing prevalence of an interpretation in the literature of difference score analyses that is inconsistent with results from equivalent separate informant analyses. Differences in the separate predictive ability of informants should be acknowledged as an alternative interpretation of the difference score regression coefficient.     

Using Raw VAR Regression Coefficients to Build Networks can be Misleading

Many questions in the behavioral sciences focus on the causal interplay of a number of variables across time. To reveal the dynamic relations between the variables, their (auto- or cross-) regressive effects across time may be inspected by fitting a lag-one vector autoregressive, or VAR(1), model and visualizing the resulting regression coefficients as the edges of a weighted directed network. Usually, the raw VAR(1) regression coefficients are drawn, but we argue that this may yield misleading network figures and characteristics because of two problems. First, the raw regression coefficients are sensitive to scale and variance differences among the variables and therefore may lack comparability, which is needed if one wants to calculate, for example, centrality measures. Second, they only represent the unique direct effects of the variables, which may give a distorted picture when variables correlate strongly. To deal with these problems, we propose to use other VAR(1)-based measures as edges. Specifically, to solve the comparability issue, the standardized VAR(1) regression coefficients can be displayed. Furthermore, relative importance metrics can be computed to include direct as well as shared and indirect effects into the network.     

Biases and Standard Errors of Standardized Regression Coefficients

The paper obtains consistent standard errors (SE) and biases of order O(1/n) for the sample standardized regression coefficients with both random and given predictors. Analytical results indicate that the formulas for SEs given in popular text books are consistent only when the population value of the regression coefficient is zero. The sample standardized regression coefficients are also biased in general, although it should not be a concern in practice when the sample size is not too small. Monte Carlo results imply that, for both standardized and unstandardized sample regression coefficients, SE estimates based on asymptotics tend to under-predict the empirical ones at smaller sample sizes.     

A Comparison of Imputation Strategies for Ordinal Missing Data on Likert Scale Variables

This article compares a variety of imputation strategies for ordinal missing data on Likert scale variables (number of categories = 2, 3, 5, or 7) in recovering reliability coefficients, mean scale scores, and regression coefficients of predicting one scale score from another. The examined strategies include imputing using normal data models with naïve rounding/without rounding, using latent variable models, and using categorical data models such as discriminant analysis and binary logistic regression (for dichotomous data only), multinomial and proportional odds logistic regression (for polytomous data only). The result suggests that both the normal model approach without rounding and the latent variable model approach perform well for either dichotomous or polytomous data regardless of sample size, missing data proportion, and asymmetry of item distributions. The discriminant analysis approach also performs well for dichotomous data. Naïvely rounding normal imputations or using logistic regression models to impute ordinal data are not recommended as they can potentially lead to substantial bias in all or some of the parameters.     

Feature extraction of event-related potentials using wavelets: an application to human performance monitoring

This report describes the development and evaluation of mathematical models for predicting human performance from discrete wavelet transforms (DWT) of event-related potentials (ERP) elicited by task-relevant stimuli. The DWT was compared to principal components analysis (PCA) for representation of ERPs in linear regression and neural network models developed to predict a composite measure of human signal detection performance. Linear regression models based on coefficients of the decimated DWT predicted signal detection performance with half as many free parameters as comparable models based on PCA scores. In addition, the DWT-based models were more resistant to model degradation due to over-fitting than PCA-based models. Feed-forward neural networks were trained using the backpropagation algorithm to predict signal detection performance based on raw ERPs, PCA scores, or high-power coefficients of the DWT. Neural networks based on high-power DWT coefficients trained with fewer iterations, generalized to new data better, and were more resistant to overfitting than networks based on raw ERPs. Networks based on PCA scores did not generalize to new data as well as either the DWT network or the raw ERP network. The results show that wavelet expansions represent the ERP efficiently and extract behaviorally important features for use in linear regression or neural network models of human performance. The efficiency of the DWT is discussed in terms of its decorrelation and energy compaction properties. In addition, the DWT models provided evidence that a pattern of low-frequency activity (1 to 3.5 Hz) occurring at specific times and scalp locations is a reliable correlate of human signal detection performance.     

SPSS and SAS programs for comparing Pearson correlations and OLS regression coefficients

Several procedures that use summary data to test hypotheses about Pearson correlations and ordinary least squares regression coefficients have been described in various books and articles. To our knowledge, however, no single resource describes all of the most common tests. Furthermore, many of these tests have not yet been implemented in popular statistical software packages such as SPSS and SAS. In this article, we describe all of the most common tests and provide SPSS and SAS programs to perform them. When they are applicable, our code also computes 100 × (1 ? α)% confidence intervals corresponding to the tests. For testing hypotheses about independent regression coefficients, we demonstrate one method that uses summary data and another that uses raw data (i.e., Potthoff analysis). When the raw data are available, the latter method is preferred, because use of summary data entails some loss of precision due to rounding.     

Comparing standardized coefficients in structural equation modeling: a model reparameterization approach

We propose a two-stage method for comparing standardized coefficients in structural equation modeling (SEM). At stage 1, we transform the original model of interest into the standardized model by model reparameterization, so that the model parameters appearing in the standardized model are equivalent to the standardized parameters of the original model. At stage 2, we impose appropriate linear equality constraints on the standardized model and use a likelihood ratio test to make statistical inferences about the equality of standardized coefficients. Unlike other existing methods for comparing standardized coefficients, the proposed method does not require specific modeling features (e.g., specification of nonlinear constraints), which are available only in certain SEM software programs. Moreover, this method allows researchers to compare two or more standardized coefficients simultaneously in a standard and convenient way. Three real examples are given to illustrate the proposed method, using EQS, a popular SEM software program. Results show that the proposed method performs satisfactorily for testing the equality of standardized coefficients.     

Sample size for multiple regression: obtaining regression coefficients that are accurate, not simply significant

An approach to sample size planning for multiple regression is presented that emphasizes accuracy in parameter estimation (AIPE). The AIPE approach yields precise estimates of population parameters by providing necessary sample sizes in order for the likely widths of confidence intervals to be sufficiently narrow. One AIPE method yields a sample size such that the expected width of the confidence interval around the standardized population regression coefficient is equal to the width specified. An enhanced formulation ensures, with some stipulated probability, that the width of the confidence interval will be no larger than the width specified. Issues involving standardized regression coefficients and random predictors are discussed, as are the philosophical differences between AIPE and the power analytic approaches to sample size planning.     

Statistical inference based on latent ability estimates

The quality of approximations to first and second order moments (e.g., statistics like means, variances, regression coefficients) based on latent ability estimates is being discussed. The ability estimates are obtained using either the Rasch, or the two-parameter logistic model. Straightforward use of such statistics to make inferences with respect to true latent ability is not recommended, unless we account for the fact that the basic quantities are estimates. In this paper true score theory is used to account for the latter; the counterpart of observed/true score being estimated/true latent ability. It is shown that statistics based on the true score theory are virtually unbiased if the number of items presented to each examinee is larger than fifteen. Three types of estimators are compared: maximum likelihood, weighted maximum likelihood, and Bayes modal. Furthermore, the (dis)advantages of the true score method and direct modeling of latent ability is discussed.     

Norming clinical questionnaires with multiple regression: the Pain Cognition List

Questionnaires for measuring patients' feelings or beliefs are commonly used in clinical settings for diagnostic purposes, clinical decision making, or treatment evaluation. Raw scores of a patient can be evaluated by comparing them with norms based on a reference population. Using the Pain Cognition List (PCL-2003) as an example, this article shows how clinical questionnaires can be normed with multiple regression of raw scores on demographic and other patient variables. Compared with traditional norm tables for subgroups based on age or gender, this approach offers 2 advantages. First, multiple regression allows determination of which patient variables are relevant to the norming and which are not (validity). Second, by using information from the entire sample, multiple regression leads to continuous and more stable norms for any subgroup defined in terms of prognostic variables (reliability).     

Method Variance in Organizational Behavior and Human Resources Research: Effects on Correlations,Path Coefficients,and Hypothesis Testing

This research simulates the effects of method variance on correlations, standardized regression (path) coefficients, and squared multiple correlation coefficients. The results show that method variance can have extreme effects on these measures of association, depending on assumptions made about the nature of the method factors. The analysis also indicated that method variance can have strong effects on the probability of obtaining significant findings in the absence of true relationships. The link between findings of the present study and current developments on method variance in organizational behavior and human resources management research is discussed.     

Bias and error in multiple-choice tests

A formula for estimating real scores on a multiple-choice test from a knowledge of raw scores is derived. This formula does not involve the assumption of a binomial distribution of real scores as does the Calandra formula. Other important formulas derived show: the variance of real scores in terms of the variance of raw scores and the correlation between real scores and raw scores. If the variance of real scores (or of raw scores also) is binomial, the regression of real scores on raw scores is linear; but, otherwise the regression is curvilinear. Yet the linear estimating formula is a close approximation to the curvilinear relationship. Factors affecting the regression of real scores on raw scores and the correlation coefficient are: (1) the number of choices per question; (2) the number of questions answered; (3) the ratio of the average group raw score to the variance of raw scores.     

结构方程模型中调节效应的标准化估计

回归分析和结构方程分析中,标准化估计对解释模型和比较效应大小有重要作用。对于调节效应模型（或交互效应模型）,通常的标准化估计没有意义。虽然显变量的调节效应模型标准化估计问题已经解决,但潜变量的调节效应模型标准化估计问题复杂得多。本文先介绍回归分析中显变量调节效应模型的标准化估计,然后提出了一种通过参数的原始估计和通常标准化估计来计算潜变量调节效应模型的“标准化”估计的方法,得到的“标准化”估计是尺度不变的,说明可以用“标准化”估计来解释和比较主效应和调节效应     

Augmenting Data With Published Results in Bayesian Linear Regression

In most research, linear regression analyses are performed without taking into account published results (i.e., reported summary statistics) of similar previous studies. Although the prior density in Bayesian linear regression could accommodate such prior knowledge, formal models for doing so are absent from the literature. The goal of this article is therefore to develop a Bayesian model in which a linear regression analysis on current data is augmented with the reported regression coefficients (and standard errors) of previous studies. Two versions of this model are presented. The first version incorporates previous studies through the prior density and is applicable when the current and all previous studies are exchangeable. The second version models all studies in a hierarchical structure and is applicable when studies are not exchangeable. Both versions of the model are assessed using simulation studies. Performance for each in estimating the regression coefficients is consistently superior to using current data alone and is close to that of an equivalent model that uses the data from previous studies rather than reported regression coefficients. Overall the results show that augmenting data with results from previous studies is viable and yields significant improvements in the parameter estimation.     

On the use of beta coefficients in meta-analysis

This research reports an investigation of the use of standardized regression (beta) coefficients in meta-analyses that use correlation coefficients as the effect-size metric. The investigation consisted of analyzing more than 1,700 corresponding beta coefficients and correlation coefficients harvested from published studies. Results indicate that, under certain conditions, using knowledge of corresponding beta coefficients to input missing correlations (effect sizes) generally produces relatively accurate and precise population effect-size estimates. Potential benefits from applying this knowledge include smaller sampling errors because of increased numbers of effect sizes and smaller non-sampling errors because of the inclusion of a broader array of research designs.     

Model‐related factor score predictors for confirmatory factor analysis

The present paper introduces model‐related (MR) factor score predictors, which reflect specific aspects of confirmatory factor models. The development is mainly based on Schönemann and Steiger's regression score components, but it can also be applied to the factor score coefficients. It is shown that the rotation of factor score predictors has no impact on the covariance matrix reproduced from the corresponding regression component patterns. Thus, regression score components or factor score coefficients can be rotated in order to obtain the required properties. This idea is the basis for MR factor score predictors, which are computed by means of a partial Procrustes rotation towards a target pattern representing the interesting properties of a confirmatory factor model. Two examples demonstrate the construction of MR factor score predictors reflecting specific constraints of a factor model.     

Using reliability generalization methods to explore measurement error: an illustration using the MMPI-2 PSY-5 scales

Reliability generalization (RG) is a meta-analytic technique that allows for the systematic examination of variation in score reliability for different samples of test takers; this procedure is based on the recognition that reliability is not a stable property of a test but is sample dependent. As a demonstration of an RG analysis, I obtained 63 reliability coefficients for each of the MMPI-2 (Butcher et al., 2001) Personality Psychopathology 5 (Harkness, McNulty, & Ben-Porath, 1995) scales. The overall variability of alpha coefficients supports the argument that reliability is sample dependent and underscores the need for researchers to calculate reliability estimates based on their research samples rather than simply citing published alpha coefficients as evidence of score reliability. I observed statistically significant mean reliability differences for scores across the 5 scales, with the highest level of reliability observed for scores on the measure of Negative Emotionality and the lowest levels of reliability observed for scores on the measures of Aggression and Disconstraint. There was no evidence that the sex-composition of a sample was systematically related to score reliability, and there were no statistically significant differences in reliability between scores obtained with the English version of the test and those obtained with translated forms. However, reliability was consistently lower for scores on some scales when the data were obtained in nonclinical settings as opposed to clinical ones. Sample size was not significantly correlated with reliability estimates. RG methods have the potential for deepening the level of understanding about the role of reliability in the evaluation and use of personality tests.     

MMPI-215中F量表的特性及其影响因素

使用MMPI—215在全国18个省市对7073名应征青年进行测试,探讨F量表分数的相关和影响因素。结果：1．F分在正常人群与异常人群中有各自不同分布特点;其应答率存在有显著差异;2．F分与其它分量表呈显著性相关;3F分受年龄、文化程度、生长地等多因素影响,低学历,生长地靠北方和农村、年龄偏小的应征青年F分数较高。     

Relations of body mass index and coronary risk as estimated by the Framingham Risk Score

The Framingham Risk Score is considered to predict 10-yr. risk of developing coronary heart disease. Other risk factors, such as a family history of coronary heart disease, sedentary lifestyle, and obesity should also be considered when estimating the risk of development of coronary heart disease. The last two factors can be modified by therapeutic lifestyle alterations. This cross-sectional coronary risk assessment of 16,871 Japanese subjects was estimated by the Framingham Risk Score. Sex and age were included in the calculation of the Framingham Risk Score. When multiple regression analysis was conducted controlling for age, regular physical exercise, smoking, and drinking, the body mass index was significantly associated with the Framingham Risk Score. The standardized regression coefficients for body mass index were .271 in men (p < .001) and .211 in women (p < .001), respectively. The significant association of body mass index with coronary heart disease risk, as estimated by the Framingham Risk Score, confirmed prior work.