心理研究中统计方法应用的元分析——以《心理学报》《心理科学》(1998-2008)统计方法应用为例

统计方法在心理研究中得到了广泛的应用。统计方法的应用,除了具备一定的统计方法知识外,还应当了解统计方法应用的必要前提条件。通过调查、分析国内最具代表性的两种杂志《心理学报》、《心理科学》(1998 2008)应用了统计方法的论文(或报告),主要了解哪些统计方法得到最多的应用以及发现应用中存在的一些问题,并对此作了评述。目的是使心理学研究者在心理研究中能更好地应用统计方法分析问题,解决问题。     

Inverse effectiveness, multisensory integration, and the bodily self: Some statistical considerations

A recent report in Consciousness and Cognition provided evidence from a study of the rubber hand illusion (RHI) that supports the multisensory principle of inverse effectiveness (PoIE). I describe two methods of assessing the principle of inverse effectiveness (‘a priori’ and ‘post-hoc’), and discuss how the post-hoc method is affected by the statistical artefact of ‘regression towards the mean’. I identify several cases where this artefact may have affected particular conclusions about the PoIE, and relate these to the historical origins of ‘regression towards the mean’. Although the conclusions of the recent report may not have been grossly affected, some of the inferential statistics were almost certainly biased by the methods used. I conclude that, unless such artefacts are fully dealt with in the future, and unless the statistical methods for assessing the PoIE evolve, strong evidence in support of the PoIE will remain lacking.     

Neuropsychological testing for assessment of treatment effects: methodologic issues

Neuropsychological (NP) testing is now recognized as an important method for evaluating treatment effects. However, there are limitations to how these tests are currently used in most drug treatment protocols. Changes in cognition are typically defined in statistical terms, with little knowledge as to whether the observed differences are meaningful in any other sense. Methods for assessing changes in test scores need to account for test-retest reliability, practice effects, regression to the mean, and the impact of initial performance. All of these factors may vary according to the individual characteristics of the subject. This article reviews two methods from the NP literature that have attempted to account for these sources of test-retest bias. The reliable change index provides a confidence interval for predicted change by taking into account the test-retest reliability of the measure. Standardized regression-based measures use a more sophisticated statistical approach that enables them to better account for other potential sources of confound. Use of this methodology has been limited to studies of epilepsy surgery and sports-related concussion. These methods have the potential for enhancing the interpretation of NP test data in drug treatment protocols by providing an empirically based definition of clinically meaningful change.     

FACTOR: A computer program to fit the exploratory factor analysis model

Exploratory factor analysis (EFA) is one of the most widely used statistical procedures in psychological research. It is a classic technique, but statistical research into EFA is still quite active, and various new developments and methods have been presented in recent years. The authors of the most popular statistical packages, however, do not seem very interested in incorporating these new advances. We present the program FACTOR, which was designed as a general, user-friendly program for computing EFA. It implements traditional procedures and indices and incorporates the benefits of some more recent developments. Two of the traditional procedures implemented are polychoric correlations and parallel analysis, the latter of which is considered to be one of the best methods for determining the number of factors or components to be retained. Good examples of the most recent developments implemented in our program are (1) minimum rank factor analysis, which is the only factor method that allows one to compute the proportion of variance explained by each factor, and (2) the simplimax rotation method, which has proved to be the most powerful rotation method available. Of these methods, only polychoric correlations are available in some commercial programs. A copy of the software, a demo, and a short manual can be obtained free of charge from the first author.     

An objective theory of statistical testing

Theories of statistical testing may be seen as attempts to provide systematic means for evaluating scientific conjectures on the basis of incomplete or inaccurate observational data. The Neyman-Pearson Theory of Testing (NPT) has purported to provide an objective means for testing statistical hypotheses corresponding to scientific claims. Despite their widespread use in science, methods of NPT have themselves been accused of failing to be objective; and the purported objectivity of scientific claims based upon NPT has been called into question. The purpose of this paper is first to clarify this question by examining the conceptions of (I) the function served by NPT in science, and (II) the requirements of an objective theory of statistics upon which attacks on NPT's objectivity are based. Our grounds for rejecting these conceptions suggest altered conceptions of (I) and (II) that might avoid such attacks. Second, we propose a reformulation of NPT, denoted by NPT*, based on these altered conceptions, and argue that it provides an objective theory of statistics. The crux of our argument is that by being able to objectively control error frequencies NPT* is able to objectively evaluate what has or has not been learned from the result of a statistical test.     

On testing for stochastic dissociations

Methods for examining stochastic relationships have been proposed as powerful ways to dissociate different underlying psychological processes, but a number of problems have undermined conclusions based upon these methods. These testing methods and criticisms are reviewed, and the statistical methods for the testing of stochastic dependence are examined using computer simulation. With each of the methods examined, there were difficulties dealing with certain situations that are likely to occur in experiments examining dependence between successive tests. Examination also showed that the sample sizes of some previous studies were insufficient for findings of moderate amounts of dependence, calling some conclusions of stochastic independence into question. The results of the studies presented here suggest that testing for statistical dependence is a statistically perilous technique, but they also suggest several ways in which dedicated users of this form of testing can strengthen its application.     

New Methods for Comparing Groups

Abstract— A commonly used method for comparing groups of individuals is the analysis of variance (ANOVA) F test. When the assumptions underlying the derivation of this test are true, its power, meaning its probability of detecting true differences among the groups, competes well with all other methods that might be used. But when these assumptions are false, its power can be relatively low. Many new statistical methods have been proposed—ones that are aimed at achieving about the same amount of power when the assumptions of the F test are true but which have the potential of high power in situations where the F test performs poorly. A brief summary of some relevant issues and recent developments is provided. Some related issues are discussed and implications for future research are described.     

The Data Envelopment Analysis and Equal Weights/Minimax Methods of Composite Social Indicator Construction: a Methodological Study of Data Sensitivity and Robustness

In the construction of composite or summary social indicators/indices, a recurrent methodological issue pertains to how to weight each of the quality-of-life/well-being components of the indices. Two methods of composite index construction that have been widely applied empirically in recent decades are Data Envelopment Analysis (DEA), which is based on an optimization principle, and the equal weights/minimax (EW/MM) method, which has been shown to have minimax statistical properties in the sense that it minimizes maximum possible disagreements among individuals on weights. This paper applies both of these methods to two empirical datasets of social indicators: 1) data on 25 well-being indicators used in the construction of state-level Child and Youth Well-being Indices for each of the 50 U.S. states, and 2) data on indicators of life expectancy, educational attainment, and income used in the construction of the United Nations Human Development Programme’s Human Development Index (HDI) for 188 countries. In these empirical contexts, we study issues of measurement sensitivity of the EW/MM and DEA methods to the numbers of indictors used in the construction of the composite indices and corresponding issues of robustness. We find that the DEA method is more sensitive to the numbers of component indicators than the EW/MM method. In addition, the composite indicators formed by the EW/MM and DEA methods become more similar as the numbers of indicators in the composites decreases. We also apply Chance-Constrained DEA method to reclassify countries in the HDI dataset by levels of human development. The resulting human development groupings of the DEA composite indices have a large overlap with those of the HDI in the Human Development Reports, which are based on fixed cut-off points derived from the quartiles of distributions of the HDI component indicators.

     

An Alternate Method of Deriving Priorities and Related Inferences for Group Decision Making in Analytic Hierarchy Process

Some work has been carried out in the past on statistically deriving priorities in Analytic Hierarchy Process (AHP). In AHP, the aggregated worths of the alternatives, when compared with respect to several criteria, are estimated in a hierarchical comparisons model introduced by Saaty. In this setup, statistical models are used for Saaty's method of scaling in paired comparisons experiments in any level of the hierarchy. At the end, the final priority weights of the alternatives and related inferences are developed with appropriate statistical methods. Existing statistical methods in the literature assume independence of the entries of the paired comparison matrix. However, these entries are highly dependent among themselves. In this article, we propose a statistical method that allows for the dependence among the entries of the pairwise comparisons matrix. The proposed method is then illustrated with a numerical example. Copyright © 2011 John Wiley & Sons, Ltd.     

The Change and Development of Statistical Methods Used in Research Articles in Child Development 1930–2010

The evolution of developmental psychology has been characterized by the use of different quantitative and qualitative methods and procedures. But how does the use of methods and procedures change over time? This study explores the change and development of statistical methods used in articles published in Child Development from 1930 to 2010. The methods used in every article in the first issue of every volume were categorized into four categories. Until 1980 relatively simple statistical methods were used. During the last 30 years there has been an explosive use of more advanced statistical methods employed. The absence of statistical methods or use of simple methods had been eliminated.     

Three-state models of memory development: A review of advances in statistical methodology

A certain family of three-state Markov processes has been widely used in experiments on children's memory. Although the statistical machinery for these models is well developed in most respects, two shortcomings have been noted in recent literature reviews. First, the parameters of the models are not identifiable. Second, goodness-of-fit tests are laborious to execute and their sampling distributions are unknown. New procedures for dealing with these difficulties are reported.     

Meehl's contribution to clinical versus statistical prediction

Paul E. Meehl's work on the clinical versus statistical prediction controversy is reviewed. His contributions included the following: putting the controversy center stage in applied psychology; clarifying concepts underpinning the debate (especially his crucial distinction between ways of gathering data and ways of combining them) as well as establishing that the controversy was real and not concocted, analyzing clinical inference from both theoretical and probabilistic points of view, and reviewing studies that compared the accuracy of these 2 methods of data combination. Meehl's (1954/1996) conclusion that statistical prediction consistently outperforms clinical judgment has stood up extremely well for half a century. His conceptual analyses have not been significantly improved since he published them in the 1950s and 1960s. His work in this area contains several citation classics, which are part of the working knowledge of all competent applied psychologists today.     

A Bayesian Way to Make Stopping Rules Matter

Disputes between advocates of Bayesians and more orthodox approaches to statistical inference presuppose that Bayesians must regard must regard stopping rules, which play an important role in orthodox statistical methods, as evidentially irrelevant.In this essay, I show that this is not the case and that the stopping rule is evidentially relevant given some Bayesian confirmation measures that have been seriously proposed. However, I show that accepting a confirmation measure of this sort comes at the cost of rejecting two useful ancillaryBayesian principles.     

It has been said that, “There are three degrees of falsehoods: Lies,damn lies,and statistics”

Applied behavior analysis (ABA) researchers have historically eschewed population-based, inferential statistics, preferring to conduct and analyze repeated observations of each participant's responding under carefully controlled and manipulated experimental conditions using single-case designs (SCDs). In addition, early attempts to adapt traditional statistical procedures for use with SCDs often involved trade-offs between experimental and statistical control that most ABA researchers have found undesirable. The statistical methods recommended for use with SCDs in the current special issue represent a welcome departure from such prior suggestions in that the current authors are proposing that SCD researchers add statistical methods to their current practices in ways that do not alter traditional SCD practices. Further refinement and use of such methods would (a) facilitate the inclusion of research using SCDs in meta-analyses and (b) aid in the development and planning of grant-funded research using SCD methods. Collaboration between SCD researchers and statisticians, particularly on research that demonstrates the benefit of these methods, may help promote their acceptance and use in ABA.     

Loglinear Rasch model tests

Existing statistical tests for the fit of the Rasch model have been criticized, because they are only sensitive to specific violations of its assumptions. Contingency table methods using loglinear models have been used to test various psychometric models. In this paper, the assumptions of the Rasch model are discussed and the Rasch model is reformulated as a quasi-independence model. The model is a quasi-loglinear model for the incomplete subgroup × score × item 1 × item 2 × ... × itemk contingency table. Using ordinary contingency table methods the Rasch model can be tested generally or against less restrictive quasi-loglinear models to investigate specific violations of its assumptions.     

Clinical significance methods: a comparison of statistical techniques

Clinically significant change refers to meaningful change in individual patient functioning during psychotherapy. Following the operational definition of clinically significant change offered by Jacobson, Follette, and Revenstorf (1984), several alternatives have been proposed because they were thought to be either more accurate or more sensitive to detecting meaningful change. In this study, we compared five methods using a sample of 386 outpatients who underwent treatment in routine clinical practice. Differences were found between methods, suggesting that the statistical method used to calculate clinical significance has an effect on estimates of meaningful change. The Jacobson method (Jacobson & Truax, 1991) provided a moderate estimate of treatment effects and was recommended for use in outcome studies and research on clinically significant change, but future research is needed to validate this statistical method.     

Using analysis of covariance (ANCOVA) with fallible covariates

Analysis of covariance (ANCOVA) is used widely in psychological research implementing nonexperimental designs. However, when covariates are fallible (i.e., measured with error), which is the norm, researchers must choose from among 3 inadequate courses of action: (a) know that the assumption that covariates are perfectly reliable is violated but use ANCOVA anyway (and, most likely, report misleading results); (b) attempt to employ 1 of several measurement error models with the understanding that no research has examined their relative performance and with the added practical difficulty that several of these models are not available in commonly used statistical software; or (c) not use ANCOVA at all. First, we discuss analytic evidence to explain why using ANCOVA with fallible covariates produces bias and a systematic inflation of Type I error rates that may lead to the incorrect conclusion that treatment effects exist. Second, to provide a solution for this problem, we conduct 2 Monte Carlo studies to compare 4 existing approaches for adjusting treatment effects in the presence of covariate measurement error: errors-in-variables (EIV; Warren, White, & Fuller, 1974), Lord's (1960) method, Raaijmakers and Pieters's (1987) method (R&P), and structural equation modeling methods proposed by S?rbom (1978) and Hayduk (1996). Results show that EIV models are superior in terms of parameter accuracy, statistical power, and keeping Type I error close to the nominal value. Finally, we offer a program written in R that performs all needed computations for implementing EIV models so that ANCOVA can be used to obtain accurate results even when covariates are measured with error.     

On the Treatment of Outliners in Cognitive and Psychomotor Test Data

Authors of many statistical texts and review articles have pointed to the possi- ble adverse effects that outliers can have on the calculation of sample statistics and have suggested several methods for detecting and treating outliers. We investigated two different methods-data censoring and transformation-for treating outliers in aptitude test data at the item level and total-score level and their effects on the internal consistency and predictive validity of six computer- ized tests being evaluated by the U.S. Air Force. Results from our sample of more than 2,000 pilot training candidates indicated that neither outlier treat- ment method at either level of analysis had significant effects on the tests' internal consistencies or predictive validities. Possible reasons for these findings include the frequency with which outliers occur and the robustness of linear modeling methods.     

A Note on Recurring Misconceptions When Fitting Nonlinear Mixed Models

Nonlinear mixed-effects (NLME) models are used when analyzing continuous repeated measures data taken on each of a number of individuals where the focus is on characteristics of complex, nonlinear individual change. Challenges with fitting NLME models and interpreting analytic results have been well documented in the statistical literature. However, parameter estimates as well as fitted functions from NLME analyses in recent articles have been misinterpreted, suggesting the need for clarification of these issues before these misconceptions become fact. These misconceptions arise from the choice of popular estimation algorithms, namely, the first-order linearization method (FO) and Gaussian-Hermite quadrature (GHQ) methods, and how these choices necessarily lead to population-average (PA) or subject-specific (SS) interpretations of model parameters, respectively. These estimation approaches also affect the fitted function for the typical individual, the lack-of-fit of individuals’ predicted trajectories, and vice versa.