Standard errors and confidence intervals for correlations corrected for indirect range restriction: A simulation study comparing analytic and bootstrap methods

A frequent topic of psychological research is the estimation of the correlation between two variables from a sample that underwent a selection process based on a third variable. Due to indirect range restriction, the sample correlation is a biased estimator of the population correlation, and a correction formula is used. In the past, bootstrap standard error and confidence intervals for the corrected correlations were examined with normal data. The present study proposes a large-sample estimate (an analytic method) for the standard error, and a corresponding confidence interval for the corrected correlation. Monte Carlo simulation studies involving both normal and non-normal data were conducted to examine the empirical performance of the bootstrap and analytic methods. Results indicated that with both normal and non-normal data, the bootstrap standard error and confidence interval were generally accurate across simulation conditions (restricted sample size, selection ratio, and population correlations) and outperformed estimates of the analytic method. However, with certain combinations of distribution type and model conditions, the analytic method has an advantage, offering reasonable estimates of the standard error and confidence interval without resorting to the bootstrap procedure's computer-intensive approach. We provide SAS code for the simulation studies.     

On approximate confidence intervals for measures of concordance

The use ofU-statistics based on rank correlation coefficients in estimating the strength of concordance among a group of rankers is examined for cases where the null hypothesis of random rankings is not tenable. The studentizedU-statistics is asymptotically distribution-free, and the Student-t approximation is used for small and moderate sized samples. An approximate confidence interval is constructed for the strength of concordance. Monte Carlo results indicate that the Student-t approximation can be improved by estimating the degrees of freedom.Research partially supported on ONR Contract N00014-82-K-0207.     

Four bootstrap confidence intervals for the binomial-error model

Confidence intervals for the mean function of the true proportion score ( _x ), where andx respectively denote the true proportion and observed test scores, can be approximated by the Efron, Bayesian, and parametric empirical Bayes (PEB) bootstrap procedures. The similarity of results yielded by all the bootstrap methods suggests the following: the unidentifiability problem of the prior distributiong() can be bypassed with respect to the construction of confidence intervals for the mean function, and a beta distribution forg() is a reasonable assumption for the test scores in compliance with a negative hypergeometric distribution. The PEB bootstrap, which reflects the construction of Morris intervals, is introduced for computing predictive confidence bands for x. It is noted that the effect of test reliability on the precision of interval estimates varies with the two types of confidence statements concerned.The Authors are indebted to the Editor and anonymous reviewers for constructive suggestions and comments. The authors wish to thank Min-Te Chao and Cheng-Der Fuh for some useful suggestions at earlier stages of writing this paper.     

认知诊断模型的标准误与置信区间估计：并行自助法

认知诊断模型的标准误(Standard Error, SE; 或方差—协方差矩阵)与置信区间(Confidence Interval, CI)在模型参数估计不确定性的度量、项目功能差异检验、项目水平上的模型比较、Q矩阵检验以及探索属性层级关系等领域有重要的理论与实践价值。本研究提出了两种新的SE和CI计算方法：并行参数化自助法和并行非参数化自助法。模拟研究发现：模型完全正确设定时, 在高质量及中等质量项目条件下, 这两种方法在计算模型参数的SE和CI时均有好的表现; 模型参数存在冗余时, 在高质量及中等质量项目条件下, 对于大部分允许存在的模型参数而言, 其SE和CI有好的表现。通过实证数据展示了新方法的价值及计算效率提升效果。     

On the sampling interpretation of confidence intervals and hypothesis tests in the context of conditional maximum likelihood estimation

In the context ofconditional maximum likelihood (CML) estimation, confidence intervals can be interpreted in three different ways, depending on the sampling distribution under which these confidence intervals contain the true parameter value with a certain probability. These sampling distributions are (a) the distribution of the data given theincidental parameters, (b) the marginal distribution of the data (i.e., with the incidental parameters integrated out), and (c) the conditional distribution of the data given the sufficient statistics for the incidental parameters. Results on the asymptotic distribution of CML estimates under sampling scheme (c) can be used to construct asymptotic confidence intervals using only the CML estimates. This is not possible for the results on the asymptotic distribution under sampling schemes (a) and (b). However, it is shown that theconditional asymptotic confidence intervals are also valid under the other two sampling schemes. I am indebted to Theo Eggen, Norman Verhelst and one of Psychometrika's reviewers for their helpful comments.     

Fisher transformation based confidence intervals of correlations in fixed- and random-effects meta-analysis

Meta-analyses of correlation coefficients are an important technique to integrate results from many cross-sectional and longitudinal research designs. Uncertainty in pooled estimates is typically assessed with the help of confidence intervals, which can double as hypothesis tests for two-sided hypotheses about the underlying correlation. A standard approach to construct confidence intervals for the main effect is the Hedges-Olkin-Vevea Fisher-z (HOVz) approach, which is based on the Fisher-z transformation. Results from previous studies (Field, 2005, Psychol. Meth., 10, 444; Hafdahl and Williams, 2009, Psychol. Meth., 14, 24), however, indicate that in random-effects models the performance of the HOVz confidence interval can be unsatisfactory. To this end, we propose improvements of the HOVz approach, which are based on enhanced variance estimators for the main effect estimate. In order to study the coverage of the new confidence intervals in both fixed- and random-effects meta-analysis models, we perform an extensive simulation study, comparing them to established approaches. Data were generated via a truncated normal and beta distribution model. The results show that our newly proposed confidence intervals based on a Knapp-Hartung-type variance estimator or robust heteroscedasticity consistent sandwich estimators in combination with the integral z-to-r transformation (Hafdahl, 2009, Br. J. Math. Stat. Psychol., 62, 233) provide more accurate coverage than existing approaches in most scenarios, especially in the more appropriate beta distribution simulation model.     

A model-based approach to multivariate principal component regression: Selecting principal components and estimating standard errors for unstandardized regression coefficients

Principal component regression (PCR) is a popular technique in data analysis and machine learning. However, the technique has two limitations. First, the principal components (PCs) with the largest variances may not be relevant to the outcome variables. Second, the lack of standard error estimates for the unstandardized regression coefficients makes it hard to interpret the results. To address these two limitations, we propose a model-based approach that includes two mean and covariance structure models defined for multivariate PCR. By estimating the defined models, we can obtain inferential information that will allow us to test the explanatory power of individual PCs and compute the standard error estimates for the unstandardized regression coefficients. A real example is used to illustrate our approach, and simulation studies under normality and nonnormality conditions are presented to validate the standard error estimates for the unstandardized regression coefficients. Finally, future research topics are discussed.     

Bootstrap standard error and confidence intervals for the correlations corrected for indirect range restriction

The standard Pearson correlation coefficient, r, is a biased estimator of the population correlation coefficient, ρ(XY) , when predictor X and criterion Y are indirectly range-restricted by a third variable Z (or S). Two correction algorithms, Thorndike's (1949) Case III, and Schmidt, Oh, and Le's (2006) Case IV, have been proposed to correct for the bias. However, to our knowledge, the two algorithms did not provide a procedure to estimate the associated standard error and confidence intervals. This paper suggests using the bootstrap procedure as an alternative. Two Monte Carlo simulations were conducted to systematically evaluate the empirical performance of the proposed bootstrap procedure. The results indicated that the bootstrap standard error and confidence intervals were generally accurate across simulation conditions (e.g., selection ratio, sample size). The proposed bootstrap procedure can provide a useful alternative for the estimation of the standard error and confidence intervals for the correlation corrected for indirect range restriction.     

Asymptotic confidence intervals for the Pearson correlation via skewness and kurtosis

When bivariate normality is violated, the default confidence interval of the Pearson correlation can be inaccurate. Two new methods were developed based on the asymptotic sampling distribution of Fisher's z′ under the general case where bivariate normality need not be assumed. In Monte Carlo simulations, the most successful of these methods relied on the (Vale & Maurelli, 1983, Psychometrika, 48, 465) family to approximate a distribution via the marginal skewness and kurtosis of the sample data. In Simulation 1, this method provided more accurate confidence intervals of the correlation in non-normal data, at least as compared to no adjustment of the Fisher z′ interval, or to adjustment via the sample joint moments. In Simulation 2, this approximate distribution method performed favourably relative to common non-parametric bootstrap methods, but its performance was mixed relative to an observed imposed bootstrap and two other robust methods (PM1 and HC4). No method was completely satisfactory. An advantage of the approximate distribution method, though, is that it can be implemented even without access to raw data if sample skewness and kurtosis are reported, making the method particularly useful for meta-analysis. Supporting information includes R code.     

bootstrap法在合成分数信度区间估计中的应用

在介绍bootstrap法原理的基础上,本文以一个同质测量模式的模拟数据为例,对结构方程模型下使用bootstrap法对合成分数信度的区间估计的应用中进行了演示。     

Improved bounds for optimal intervals in weakly constrained regression problems

The solution of weakly constrained regression problems typically requires the iterative search, in a given interval, of a point where a certain function has a zero derivative. This note deals with improved bounds for the interval to be searched.     

Bootstrap standard error and confidence intervals for the correlation corrected for range restriction: a simulation study

The standard Pearson correlation coefficient is a biased estimator of the true population correlation, rho, when the predictor and the criterion are range restricted. To correct the bias, the correlation corrected for range restriction, rc, has been recommended, and a standard formula based on asymptotic results for estimating its standard error is also available. In the present study, the bootstrap standard-error estimate is proposed as an alternative. Monte Carlo simulation studies involving both normal and nonnormal data were conducted to examine the empirical performance of the proposed procedure under different levels of rho, selection ratio, sample size, and truncation types. Results indicated that, with normal data, the bootstrap standard-error estimate is more accurate than the traditional estimate, particularly with small sample size. With nonnormal data, performance of both estimates depends critically on the distribution type. Furthermore, the bootstrap bias-corrected and accelerated interval consistently provided the most accurate coverage probability for rho.     

The statistical recommendations of the American Psychological Association Publication Manual: Effect sizes,confidence intervals,and meta‐analysis

Estimation based on effect sizes, confidence intervals, and meta‐analysis usually provides a more informative analysis of empirical results than does statistical significance testing, which has long been the conventional choice in psychology. The sixth edition of the American Psychological Association Publication Manual now recommends that psychologists should, wherever possible, use estimation and base their interpretation of research results on point and interval estimates. We outline the Manual's recommendations and suggest how they can be put into practice: adopt an estimation framework, starting with the formulation of research aims as ‘How much?’ or ‘To what extent?’ questions. Calculate from your data effect size estimates and confidence intervals to answer those questions, then interpret. Wherever appropriate, use meta‐analysis to integrate evidence over studies. The Manual's recommendations can assist psychologists improve they way they do their statistics and help build a more quantitative and cumulative discipline.     

Probability-of-coverage tolerance intervals for the beta binomial

True score tolerance intervals, which are designed to cover a chosen proportion of the conditional distribution of true scores given an observed score, are suggested as alternatives to true score confidence intervals. Using large sample theory, a tolerance interval estimator for the beta binomial is derived. An example indicates that with moderate sample sizes, tolerance intervals with high probability of coverage will not be much wider than when the two beta true score parameters are known.The author acknowledges valuable comments from Richard Sawyer.Most work was completed while the author was at the American College Testing Program.     

Estimating confidence intervals for principal component loadings: A comparison between the bootstrap and asymptotic results

Confidence intervals (CIs) in principal component analysis (PCA) can be based on asymptotic standard errors and on the bootstrap methodology. The present paper offers an overview of possible strategies for bootstrapping in PCA. A motivating example shows that CI estimates for the component loadings using different methods may diverge. We explain that this results from both differences in quality and in perspective on the rotational freedom of the population loadings. A comparative simulation study examines the quality of various estimated component loading CIs. The bootstrap approach is more flexible and generally yields better CIs than the asymptotic approach. However, in the case of a clear simple structure of varimax rotated loadings, one can be confident that the asymptotic estimates are reasonable as well.     

Assessing the stability of principal components using regression

This paper presents an analysis, based on simulation, of the stability of principal components. Stability is measured by the expectation of the absolute inner product of the sample principal component with the corresponding population component. A multiple regression model to predict stability is devised, calibrated, and tested using simulated Normal data. Results show that the model can provide useful predictions of individual principal component stability when working with correlation matrices. Further, the predictive validity of the model is tested against data simulated from three non-Normal distributions. The model predicted very well even when the data departed from normality, thus giving robustness to the proposed measure. Used in conjunction with other existing rules this measure will help the user in determining interpretability of principal components.The authors would like to thank the four anonymous reviewers and the two editors for their valuable comments. Atanu R. Sinha gratefully acknowledges the research support received from the Marketing Studies Center, AGSM, UCLA. Send requests for reprints to Atanu R. Sinha, B418 Gold Hall, 110 Westwood Plaza, Los Angeles, CA 90095.     

The regression trunk approach to discover treatment covariate interaction

The regression trunk approach (RTA) is an integration of regression trees and multiple linear regression analysis. In this paper RTA is used to discover treatment covariate interactions, in the regression of one continuous variable on a treatment variable withmultiple covariates. The performance of RTA is compared to the classical method of forward stepwise regression. The results of two simulation studies, in which the true interactions are modeled as threshold interactions, show that RTA detects the interactions in a higher number of cases (82.3% in the first simulation study, and 52.3% in the second) than stepwise regression (56.5% and 20.5%). In a real data example the final RTA model has a higher cross-validated variance-accounted-for (29.8%) than the stepwise regression model (12.5%). All of these results indicate that RTA is a promising alternative method for demonstrating differential effectiveness of treatments. Supported by The Netherlands Organization for Scientific Research (NWO), grant 030-56403 to Jacqueline Meulman for the Pioneer project “Subject-Oriented Multivariate Analysis,” and grant 451-02-058 to Elise Dusseldorp for the Veni project “Modeling interaction effects as small trees in regression and classification.” We thank Bram Bakker for making the data of his doctoral dissertation available, and David Hand, Jerome Friedman, Bart Jan van Os, Philip Spinhoven, and the anonymous reviewers for their helpful suggestions. Special thanks are due to Lawrence Hubert.     

Constructing bootstrap confidence intervals for principal component loadings in the presence of missing data: a multiple-imputation approach

Earlier research has shown that bootstrap confidence intervals from principal component loadings give a good coverage of the population loadings. However, this only applies to complete data. When data are incomplete, missing data have to be handled before analysing the data. Multiple imputation may be used for this purpose. The question is how bootstrap confidence intervals for principal component loadings should be corrected for multiply imputed data. In this paper, several solutions are proposed. Simulations show that the proposed corrections for multiply imputed data give a good coverage of the population loadings in various situations.     

Approximate confidence intervals for a robust scale parameter

A recent paper by Wainer and Thissen has renewed the interest in Gini's mean difference,G, by pointing out its robust characteristics. This note presents distribution-free asymptotic confidence intervals for its population value,γ, in the one sample case and for the difference Δ=(γ ₁?γ ₂) in the two sample situations. Both procedures are based on a technique of jackknifingU-statistics developed by Arvesen.     

Incremental criterion prediction of personality facets over factors: Obtaining unbiased estimates and confidence intervals

Many researchers have argued that higher order models of personality such as the Five Factor Model are insufficient, and that facet-level analysis is required to better understand criteria such as well-being, job performance, and personality disorders. However, common methods in the extant literature used to estimate the incremental prediction of facets over factors have several shortcomings. This paper delineates these shortcomings by evaluating alternative methods using statistical theory, simulation, and an empirical example. We recommend using differences between Olkin–Pratt adjusted r-squared for factor versus facet regression models to estimate the incremental prediction of facets and present a method for obtaining confidence intervals for such estimates using double adjusted-r-squared bootstrapping. We also provide an R package that implements the proposed methods.