Effect of the violation of normality and sphericity in the linear mixed model in split-plot designs

This study aimed to evaluate the robustness of the linear mixed model, with the Kenward-Roger correction for degrees of freedom, when implemented in SAS PROC MIXED, using split-plot designs with small sample sizes. A Monte Carlo simulation design involving three groups and four repeated measures was used, assuming an unstructured covariance matrix to generate the data. The study variables were: sphericity, with epsilon values of 0.75 and 0.57; group sizes, equal or unequal; and shape of the distribution. As regards the latter, non-normal distributions were introduced, combining different values of kurtosis in each group. In the case of unbalanced designs, the effect of pairing (positive or negative) the degree of kurtosis with group size was also analysed. The results show that the Kenward-Roger procedure is liberal, particularly for the interaction effect, under certain conditions in which normality is violated. The relationship between the values of kurtosis in the groups and the pairing of kurtosis with group size are found to be relevant variables to take into account when applying this procedure.     

Working with the Holocaust victims psychologically: Some vital cautions

This article traces the evolution of a major psychological organization's involvement with the Holocaust as a fully justified and legitimate topic of concern and of an ongoing commitment to its study and memory. A number of important cautions and requirements are presented for mental health workers who wish to work in the areas encompassed by the Holocaust, including working therapeutically with Survivors of the Holocaust and their children. The special danger of diminishing or trivializing the Holocaust in the course of writing about aspects or elements of it in psychological articles or writings, as has been the case in other kinds of literature, is brought into focus.     

Statistical power in complex experimental designs

We show that power and sample size tables developed by Cohen (1988, pp. 289–354, 381–389) produce incorrect estimates for factorial designs: power is underestimated, and sample size is overestimated. The source of this bias is shrinkage in the implied value of the noncentrality parameter, λ, caused by using Cohen’s adjustment ton for factorial designs (pp. 365 and 396). The adjustment was intended to compensate for differences in the actual versus presumed (by the tables) error degrees of freedom; however, more accurate estimates are obtained if the tables are used without adjustment. The problems with Cohen’s procedure were discovered while testing subroutines in DATASIM 1.2 for computing power and sample size in completely randomized, randomized-blocks, and split-plot factorial designs. The subroutines give the user the ability to generate power and sample size tables that are as easy to use as Cohen’s, but that eliminate the conservative bias of his tables. We also implemented several improvements relative to “manual” use of Cohen’s tables: (1) Since the user can control the specific values of 1- β,n, andf used on the rows and columns of the table, interpolation is never required; (2) exact as opposed to approximate solutions for the noncentralF distribution are employed; (3) solutions for factorial designs, including those with repeated measures factors, take into account the actual error degrees of freedom for the effect being tested; and (4) provision is made for the computation of power for applications involving the doubly noncentralF distribution.     

Regression-based techniques for statistical decision making in single-case designs

The present study evaluates the performance of four methods for estimating regression coefficients used to make statistical decisions about intervention effectiveness in single-case designs. Ordinary least square estimation is compared to two correction techniques dealing with general trend and a procedure that eliminates autocorrelation whenever it is present. Type I error rates and statistical power are studied for experimental conditions defined by the presence or absence of treatment effect (change in level or in slope), general trend, and serial dependence. The results show that empirical Type I error rates do not approach the nominal ones in the presence of autocorrelation or general trend when ordinary and generalized least squares are applied. The techniques controlling trend show lower false alarm rates, but prove to be insufficiently sensitive to existing treatment effects. Consequently, the use of the statistical significance of the regression coefficients for detecting treatment effects is not recommended for short data series.     

Comparing single-pool and multiple-pool designs regarding test security in computerized testing

This article compares the use of single- and multiple-item pools with respect to test security against item sharing among some examinees in computerized testing. A simulation study was conducted to make a comparison among different pool designs using the item selection method of maximum item information with the Sympson-Hetter exposure control and content balance. The results from the simulation study indicate that two-pool designs have a better degree of resistance to item sharing than do the single-pool design in terms of measurement precision in ability estimation. This article further characterizes the conditions under which employing a multiple-pool design is better than using a single, whole pool in terms of minimizing the number of compromised items encountered by examinees under a randomized item selection method. Although no current computerized testing program endorses the randomized item selection method, the results derived in this study can shed some light on item pool designs regarding test security for all item selection algorithms, especially those that try to equalize or balance item exposure rates by employing a randomized item selection method locally, such as the a-stratified-with-b-blocking method.     

Some statistical considerations in multidimensional scaling

Some shortcomings of current methods of estimating the magnitude of perceived difference are considered. A statistical model for perceived difference is derived which avoids these difficulties and employs judgments of ratios of differences as data. Three estimators of squared difference are developed.This study was conducted while the author was a Psychometric Fellow at Princeton University and Educational Testing Service and is part of a dissertation presented in candidacy for the degree of doctor of philosophy. This research was supported by Office of Naval Research Contract Nonr 1858 and by National Science Foundation Grant GB3402. Extensive use was made of the computing facilities of Princeton University supported in part by National Science Foundation Grant NSF-GP579. The author wishes to express his appreciation to Prof. H. Gulliksen, Prof. F. Geldard, Dr. C. Helm, and Dr. F. Lord for their comments and encouragement.     

How meta-analysis increases statistical power

One of the most frequently cited reasons for conducting a meta-analysis is the increase in statistical power that it affords a reviewer. This article demonstrates that fixed-effects meta-analysis increases statistical power by reducing the standard error of the weighted average effect size (T.) and, in so doing, shrinks the confidence interval around T.. Small confidence intervals make it more likely for reviewers to detect nonzero population effects, thereby increasing statistical power. Smaller confidence intervals also represent increased precision of the estimated population effect size. Computational examples are provided for 3 effect-size indices: d (standardized mean difference), Pearson's r, and odds ratios. Random-effects meta-analyses also may show increased statistical power and a smaller standard error of the weighted average effect size. However, the authors demonstrate that increasing the number of studies in a random-effects meta-analysis does not always increase statistical power.     

The power of statistical tests in meta-analysis

Calculations of the power of statistical tests are important in planning research studies (including meta-analyses) and in interpreting situations in which a result has not proven to be statistically significant. The authors describe procedures to compute statistical power of fixed- and random-effects tests of the mean effect size, tests for heterogeneity (or variation) of effect size parameters across studies, and tests for contrasts among effect sizes of different studies. Examples are given using 2 published meta-analyses. The examples illustrate that statistical power is not always high in meta-analysis.     

Some approximate tests for repeated measurement designs

Four approximate tests are considered for repeated measurement designs in which observations are multivariate normal with arbitrary covariance matrices. In these tests traditional within-subject mean square ratios are compared with critical values derived fromF distributions with adjusted degrees of freedom. Two of them—the approximate and the improved general approximate (IGA) tests—behave adequately in terms of Type I error. Generally, the IGA test functions better than the approximate test, however the latter involves less computations. In regards to power, the IGA test may compete with one multivariate procedure when the assumptions of the latter are tenable.The author wishes to thank Garrett K. Mandeville for his careful reading of the final version of the paper.     

Some evidence regarding development of cerebral lateralization

Frequencies of significant rs for right- and left-hemisphere scales of Elementary School and Youth forms of Style of Learning and Thinking increase from K through Grade 8 (Ns = 1360) as would be expected if cerebral lateralization begins with acquisition of language and is completed in puberty.     

A comparison of single-subject and traditional statistical designs in steady-state evoked potential research

An experiment was conducted to assess the relationship between level of auditory noise and a measure derived from the steady-state evoked potential (SSEP) called relative transmission time (RTF). The visual stimulus consisted of a 150-fL light that was modulated at 45, 48, or 51 Hz to a depth of 30%. The auditory stimulus was bandpass-limited white noise at 65, 75, or 85 dB, The outcomes of single-subject and group statistical analyses were compared. Neither statistic indicated a significant effect for cross-modal white noise on the SSEP. Whereas the more traditional group approach indicated only large between-subject variability, the single-subject approach clearly indicated deviant data of one of the subjects.     

Implications of statistical power for confidence intervals

The statistical power of a hypothesis test is closely related to the precision of the accompanying confidence interval. In the case of a z-test, the width of the confidence interval is a function of statistical power for the planned study. If minimum effect size is used in power analysis, the width of the confidence interval is the minimum effect size times a multiplicative factor φ. The index φ, or the precision-to-effect ratio, is a function of the computed statistical power. In the case of a t-test, statistical power affects the probability of achieving a certain width of confidence interval, which is equivalent to the probability of obtaining a certain value of φ. To consider estimate precision in conjunction with statistical power, we can choose a sample size to obtain a desired probability of achieving a short width conditional on the rejection of the null hypothesis.     

A SAS macro for statistical power calculations in meta-analysis

Although statistical power is often considered in the design of primary research studies, it is rarely considered in meta-analysis. Background and guidelines are provided for conducting power analysis in meta-analysis, followed by the presentation of a SAS macro that calculates power using the methods described by Hedges and Pigott (2001, 2004). Several detailed examples are given, including input statements and output. Practical issues in the application of power analysis to meta-analysis are discussed. The macro and examples may be downloaded as supplemental materials for this article from brm.psychonomic-journals.org/content/supplemental.     

The power of statistical tests for moderators in meta-analysis

Calculation of the statistical power of statistical tests is important in planning and interpreting the results of research studies, including meta-analyses. It is particularly important in moderator analyses in meta-analysis, which are often used as sensitivity analyses to rule out moderator effects but also may have low statistical power. This article describes how to compute statistical power of both fixed- and mixed-effects moderator tests in meta-analysis that are analogous to the analysis of variance and multiple regression analysis for effect sizes. It also shows how to compute power of tests for goodness of fit associated with these models. Examples from a published meta-analysis demonstrate that power of moderator tests and goodness-of-fit tests is not always high.