A Critique of Miller and Rohling's Statistical Interpretive Method for Neuropsychological Test Data

A critical review of the 24-step procedure of Miller and Rohling's (in press) proposed standardization of clinician's use of neuropsychological assessment batteries is presented. Each step is examined for statistical sources of invalidity. It was concluded that parts of the procedure are quite vulnerable to between-battery variability that cannot be easily estimated or controlled, leading to significant errors in analysis and classification. A second fatal flaw is the failure to distinguish in the procedures between standard error measurement and standard error of the estimate in calculations in several steps. The purpose of the process remains viable, however, and is an important contribution toward the improvement of clinical diagnosis.     

A Test by Any Other Name: P Values,Bayes Factors,and Statistical Inference

Procedures used for statistical inference are receiving increased scrutiny as the scientific community studies the factors associated with insuring reproducible research. This note addresses recent negative attention directed at p values, the relationship of confidence intervals and tests, and the role of Bayesian inference and Bayes factors, with an eye toward better understanding these different strategies for statistical inference. We argue that researchers and data analysts too often resort to binary decisions (e.g., whether to reject or accept the null hypothesis) in settings where this may not be required.     

平均数差异显著性检验统计检验力和效果大小的估计原理与方法

该文以平均数差异显著性检验为例,对实验数据进行假设检验后,继续对其统计检验力和效果大小进行估计的基本原理和方法作一介绍。     

The Effect of Researchers' Focus on Interpretation of Diversity Data

Abstract

The tendency for intercultural researchers to focus primarily on cultural differences instead of both differences and similarities may reflect the emphasis of current statistical methodology toward cultural distance instead of cultural overlap. The authors proposed the cultural similarity index as a way of assessing the extent of communalities between 2 groups. The authors (a) analyzed research (T. Cox, S. Lobel, & P. L. McLeod, 1991; Y. F. Niemann & J. Dovidio, 1998) that placed a primary emphasis on differences, (b) presented alternative insights gained from a focus on similarities, and (c) explored the implications of a research focus on both cultural differences and similarities.     

共同方法偏差的统计检验与控制方法

共同方法偏差是心理学研究中一个越来越被关注的课题,但国内心理学界对其的介绍、研究还比较少,导致研究中对共同方法偏差的控制还有所欠缺。共同方法偏差的控制方法分为程序控制和统计控制,该文介绍了共同方法偏差的多种统计控制法,如因素分析法、偏相关法、潜在的误差变量控制法、结构方程模型法等,分析了其各自的优缺点,并提出了如何选择采用这些方法的建议.     

差数显著性t检验与元分析的对比研究

利用计算机构造被试总体、模拟实验研究程序进行抽样研究,探讨显著性ｔ检验方法与元分析方法在检验实验结果数据方面的差异。在模拟实验过程中,ｔ验受到显著性水平、样本容量和总体效果大小的影响,因此最终影响了统计推断的可靠性,建议：在进行显著性检验过程中,应对统计检验能力进行估计;元分析方法以样本为元素对总体进行推断,因此具有很高的准确性和可靠性,它将很有可能成为今后心理学研究的重要统计工具。     

与零假设检验有关的统计分析流程

人们熟知的零假设显著性检验,受到一次次质疑与辩护,地位并未动摇,报告检验结果仍然是统计分析的习惯做法。不过,其局限性促使研究者探寻更多的统计方法如区间估计、效应量分析、检验力分析等。本文先介绍假设检验与置信区间的关系;然后讨论检验力与两类错误率和效应量的关系;最后在理顺上述统计方法的基础上,提供一个可操作的统计分析流程。     

方差分析的统计检验力和效果大小的常用方法比较

本文对用方差分析统计检验力和效果大小进行估计的几种不同方法作了简要的介绍和比较。     

Australian Norms and Retest Data for the Rey Auditory and Verbal Learning Test

This article presents normative data for the Rey Auditory and Verbal Learning Test (RAVLT). A representative sample of 390 healthy young adults aged between 18 and 34 living within the Sydney metropolitan area, Australia, completed Form AB of the RAVLT as part of the Macquarie University Neurological Normative Study. Retest data were collected from a subsample of 98 participants after an interval of 1 year. Norms were derived for commonly used measures of the RAVLT and are presented for the whole sample as well as separately for males and females with different levels of education. Age was not found to impact significantly on test performance for this group of young adults, and therefore age‐adjusted norms are not provided. An excel program to calculate RAVLT standard scores (mean of 10 and standard deviation of 3) can be downloaded from http://www.psy.mq.edu.au/RAVLT . Poor test–retest reliability raises concerns about the use of the RAVLT in clinical diagnosis.     

差数显著性t检验与元分析方法的模拟对比

利用计算机程序构造被试总体、模拟实验研究程序进行抽样研究 ,探讨差数显著性t检验方法与元分析方法在检验实验结果数据和进行实际应用方面的差异。模拟实验结果表明 ,差数显著性t检验与总体效果大小和样本容量有明显关系 ,但与随机抽样分布样本数目基本无关 ;抽样分布样本效果大小的平均值可以作为总体效果大小的估计值 ,它与样本容量和抽样样本数目有密切关系 ;本研究提供的实际应用数据结果可以作为研究者进行元分析方法实验研究的参考依据     

共同方法变异的影响及其统计控制途径的模型分析

共同方法变异(common method variance, CMV)指两个变量之间变异的重叠是因为使用同类测量工具而导致, 而不是代表潜在构念之间的真实关系。虽然以往研究显示CMV不一定导致研究结果的偏差, 在实际研究中应当加以考量。特别是在使用测量方法的研究中, 如果数据来源越单一, 测量方法越类似, CMV效应使研究结果产生偏差的可能性越大。CMV效应的控制方法包括过程控制法和统计控制法。在统计控制法的选择和使用上, 需要重点考虑该方法是否分离了三大变异(特质变异、方法变异和误差变异), CMV效应是在测量构念层面还是题目层面, CMV效应是加法效应还是乘法效应。控制潜在方法因子途径是统计控制方法中最重要的一类方法, 理解其模型是正确使用这类方法的前提。未来研究应当关注多个研究的CMV效应和侧重评估某个理论研究中CMV所引起的潜在的效度威胁。     

A taxonomy of latent structure assumptions for probability matrix decomposition models

A taxonomy of latent structure assumptions (LSAs) for probability matrix decomposition (PMD) models is proposed which includes the original PMD model (Maris, De Boeck, & Van Mechelen, 1996) as well as a three-way extension of the multiple classification latent class model (Maris, 1999). It is shown that PMD models involving different LSAs are actually restricted latent class models with latent variables that depend on some external variables. For parameter estimation a combined approach is proposed that uses both a mode-finding algorithm (EM) and a sampling-based approach (Gibbs sampling). A simulation study is conducted to investigate the extent to which information criteria, specific model checks, and checks for global goodness of fit may help to specify the basic assumptions of the different PMD models. Finally, an application is described with models involving different latent structure assumptions for data on hostile behavior in frustrating situations.Note: The research reported in this paper was partially supported by the Fund for Scientific Research-Flanders (Belgium) (project G.0207.97 awarded to Paul De Boeck and Iven Van Mechelen), and the Research Fund of K.U. Leuven (F/96/6 fellowship to Andrew Gelman, OT/96/10 project awarded to Iven Van Mechelen and GOA/2000/02 awarded to Paul De Boeck and Iven Van Mechelen). We thank Marcel Croon and Kristof Vansteelandt for commenting on an earlier draft of this paper.     

Norms for Parental Ratings on Conners' Abbreviated Parent-Teacher Questionnaire: Implications for the Design of Behavioral Rating Inventories and Analyses of Data Derived from Them

Despite the wide usage of Conners' 10-item Abbreviated Parent-Teacher Questionnaire (APTQ), normative data for parental ratings on the APTQ are rare. Moreover, its psychometric properties as a stand alone behavioral rating instrument have yet to be examined in detail. Using data from parental ratings for 6,841 children aged 5 to 14 years, this paper reports the psychometric properties of the APTQ and provides normative and prevalence estimates for four age cohorts (5 to 6, 7 to 8, 9 to 11, and 12 to 14 years) by gender. In presenting the findings, the paper highlights key methodological issues endemic to the design of behavioral rating inventories like the APTQ and the analysis of data derived from them. These issues are illustrated and discussed in terms of their implications for the measurement and determination of prevalence estimates of problem behaviors in child psychology and child psychiatry. The need to revise the design, content, and response formats of child behavior rating inventories is stressed.     

随机截距因子分析模型在控制条目表述效应中的应用

本研究用中文修订版罗森博格自尊量表(RSES-R)考察随机截距因子分析模型在控制条目表述效应时的表现。用RSES-R和过分宣称问卷组成的量表调查621名中学生。结果表明,随机截距模型在建模时,拟合指数良好、因子方差与负荷合理,自尊因子分与RSES-R总分有极高相关,表明该模型能有效分离RSES-R得分的特质与表述效应。分离的表述效应因子分与受测者的自我提升水平具有显著但较弱的相关,表明表述效应与自受测者的社会赞许性有共同的成分。     

The CHIC Model: A Global Model for Coupled Binary Data

Often problems result in the collection of coupled data, which consist of different N-way N-mode data blocks that have one or more modes in common. To reveal the structure underlying such data, an integrated modeling strategy, with a single set of parameters for the common mode(s), that is estimated based on the information in all data blocks, may be most appropriate. Such a strategy implies a global model, consisting of different N-way N-mode submodels, and a global loss function that is a (weighted) sum of the partial loss functions associated with the different submodels. In this paper, such a global model for an integrated analysis of a three-way three-mode binary data array and a two-way two-mode binary data matrix that have one mode in common is presented. A simulated annealing algorithm to estimate the model parameters is described and evaluated in a simulation study. An application of the model to real psychological data is discussed. T. Wilderjans is a Research Assistant of the Fund for Scientific Research—Flanders (Belgium). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/2005/04). We are grateful to Kristof Vansteelandt for providing us with an interesting data set. We also thank three anonymous reviewers for their useful comments.     

Disclosure of Tests and Raw Test Data to the Courts: A Need for Reform

Psychologists' current practices with respect to the disclosure of tests and raw test data to courts are damaging their reputations as scientists, undermining their credibility as honest professionals acting in good faith, and contrary to the best interests of consumers. The profession is conducting itself in a contradictory fashion that deserves reform. There is need for an interdisciplinary panel drawn from the American Psychological Association and American Bar Association to develop reasonable procedures for disclosure of tests and test data in legal proceedings. These procedures should be developed with input from experienced psychological expert witnesses, attorneys, and judges from a variety of state and federal jurisdictions, diverse geographical areas, and different types of legal proceedings. This expert panel should also address the issue of attorney coaching of clients in preparation for assessment by psychologists.     

Sample Size Planning for Detecting Mediation Effects: A Power Analysis Procedure Considering Uncertainty in Effect Size Estimates

Abstract

When planning mediation studies, researchers are often interested in the sample size needed to achieve adequate power for testing mediation. Power depends on population effect sizes, which are unknown in practice. In conventional power analysis, effect size estimates, however, are often used as population values, which could result in underpowered studies. Uncertainty in effect size estimates has been considered in other sample size planning contexts (e.g., t-test, ANOVA), but has not been handled properly for planning mediation studies. In the current study, we proposed an easy-to-use sample size planning method for testing mediation with uncertainty in effect size estimates considered. We conducted simulation studies to demonstrate the impact of uncertainty in effect size estimates on power of testing mediation, and to provide sample size suggestions under different levels of uncertainty. Empirical examples were provided to illustrate the application of our method. R functions and a web application were developed to facilitate implementation.     

A Test of Order-Constrained Hypotheses for Circular Data With Applications to Human Movement Science

Researchers studying the movements of the human body often encounter data measured in angles (e.g., angular displacements of joints). The evaluation of these circular data requires special statistical methods. The authors introduce a new test for the analysis of order-constrained hypotheses for circular data. Through this test, researchers can evaluate their expectations regarding the outcome of an experiment directly by representing their ideas in the form of a hypothesis containing inequality constraints. The resulting data analysis is generally more powerful than one using standard null hypothesis testing. Two examples of circular data from human movement science are presented to illustrate the use of the test. Results from a simulation study show that the test performs well.     

A Data Analysis Method for Using Longitudinal Binary Outcome Data from a SMART to Compare Adaptive Interventions

Sequential multiple assignment randomized trials (SMARTs) are a useful and increasingly popular approach for gathering information to inform the construction of adaptive interventions to treat psychological and behavioral health conditions. Until recently, analysis methods for data from SMART designs considered only a single measurement of the outcome of interest when comparing the efficacy of adaptive interventions. Lu et al. proposed a method for considering repeated outcome measurements to incorporate information about the longitudinal trajectory of change. While their proposed method can be applied to many kinds of outcome variables, they focused mainly on linear models for normally distributed outcomes. Practical guidelines and extensions are required to implement this methodology with other types of repeated outcome measures common in behavioral research. In this article, we discuss implementation of this method with repeated binary outcomes. We explain how to compare adaptive interventions in terms of various summaries of repeated binary outcome measures, including average outcome (area under the curve) and delayed effects. The method is illustrated using an empirical example from a SMART study to develop an adaptive intervention for engaging alcohol- and cocaine-dependent patients in treatment. Monte Carlo simulations are provided to demonstrate the good performance of the proposed technique.