Problematic standard errors and confidence intervals for skewness and kurtosis

Many statistics packages print skewness and kurtosis statistics with estimates of their standard errors. The function most often used for the standard errors (e.g., in SPSS) assumes that the data are drawn from a normal distribution, an unlikely situation. Some textbooks suggest that if the statistic is more than about 2 standard errors from the hypothesized value (i.e., an approximate value for the critical value from the t distribution for moderate or large sample sizes when α = 5%), the hypothesized value can be rejected. This is an inappropriate practice unless the standard error estimate is accurate and the sampling distribution is approximately normal. We show distributions where the traditional standard errors provided by the function underestimate the actual values, often being 5 times too small, and distributions where the function overestimates the true values. Bootstrap standard errors and confidence intervals are more accurate than the traditional approach, although still imperfect. The reasons for this are discussed. We recommend that if you are using skewness and kurtosis statistics based on the 3^rd and 4^th moments, bootstrapping should be used to calculate standard errors and confidence intervals, rather than using the traditional standard. Software in the freeware R for this article provides these estimates.     

Univariate and multivariate skewness and kurtosis for measuring nonnormality: Prevalence,influence and estimation

Nonnormality of univariate data has been extensively examined previously (Blanca et al., Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 9(2), 78–84, 2013; Miceeri, Psychological Bulletin, 105(1), 156, 1989). However, less is known of the potential nonnormality of multivariate data although multivariate analysis is commonly used in psychological and educational research. Using univariate and multivariate skewness and kurtosis as measures of nonnormality, this study examined 1,567 univariate distriubtions and 254 multivariate distributions collected from authors of articles published in Psychological Science and the American Education Research Journal. We found that 74 % of univariate distributions and 68 % multivariate distributions deviated from normal distributions. In a simulation study using typical values of skewness and kurtosis that we collected, we found that the resulting type I error rates were 17 % in a t-test and 30 % in a factor analysis under some conditions. Hence, we argue that it is time to routinely report skewness and kurtosis along with other summary statistics such as means and variances. To facilitate future report of skewness and kurtosis, we provide a tutorial on how to compute univariate and multivariate skewness and kurtosis by SAS, SPSS, R and a newly developed Web application.     

The effect of skewness and kurtosis on the robustness of linear mixed models

This study analyzes the robustness of the linear mixed model (LMM) with the Kenward–Roger (KR) procedure to violations of normality and sphericity when used in split-plot designs with small sample sizes. Specifically, it explores the independent effect of skewness and kurtosis on KR robustness for the values of skewness and kurtosis coefficients that are most frequently found in psychological and educational research data. To this end, a Monte Carlo simulation study was designed, considering a split-plot design with three levels of the between-subjects grouping factor and four levels of the within-subjects factor. Robustness is assessed in terms of the probability of type I error. The results showed that (1) the robustness of the KR procedure does not differ as a function of the violation or satisfaction of the sphericity assumption when small samples are used; (2) the LMM with KR can be a good option for analyzing total sample sizes of 45 or larger when their distributions are normal, slightly or moderately skewed, and with different degrees of kurtosis violation; (3) the effect of skewness on the robustness of the LMM with KR is greater than the corresponding effect of kurtosis for common values; and (4) when data are not normal and the total sample size is 30, the procedure is not robust. Alternative analyses should be performed when the total sample size is 30.     

The effects of skewness on the interpretation of ITPA scaled scores

Studied the degree to which skewed score distributions can affect the interpretation of Illinois Test of Psycholinguistic Abilities (ITPA) (Kirk, McCarthy, & Kirk, 1968) subtest scores. Indices of skewness were determined for the 10 main ITPA subtests for each of the eight age groups which comprised the normative sample (Paraskevopoulos & Kirk, 1969). The ITPA normative sample was drawn from children, both male and female, having abbreviated Stanford-Binet IQs between 84 and 116 and ranging in age from 2 years 7 months to 10 years 1 month. The results indicate that the following subtests were most seriously affected by scale limitations: auditory reception, auditory association, visual reception, manual expression, and grammatic closure. The results suggest that indices of score variability such as average deviation and standard scores must be interpreted with extreme caution when skewness is a significant factor.     

Effects of skewness and kurtosis on normal-theory based maximum likelihood test statistic in multilevel structural equation modeling

A simulation study investigated the effects of skewness and kurtosis on level-specific maximum likelihood (ML) test statistics based on normal theory in multilevel structural equation models. The levels of skewness and kurtosis at each level were manipulated in multilevel data, and the effects of skewness and kurtosis on level-specific ML test statistics were examined. When the assumption of multivariate normality was violated, the level-specific ML test statistics were inflated, resulting in Type I error rates that were higher than the nominal level for the correctly specified model. Q-Q plots of the test statistics against a theoretical chi-square distribution showed that skewness led to a thicker upper tail and kurtosis led to a longer upper tail of the observed distribution of the level-specific ML test statistic for the correctly specified model.     

Psychology without p values. Data analysis at the turn of the 19th century

Although the fledgling psychology of 100 years ago was assertively empirical, there were no inferential statistics to guide psychologists' data analyses. However, 19th-century developments had left psychology with a rich array of techniques for analyzing and presenting data, some of which remain underutilized today. These include comparisons across replications, within-subject designs, reanalysis of data, analyses of factorial designs, and especially the use of tables and graphs. As the merits of hypothesis-testing statistics are debated at the turn of the 21st century, the history of data-handling practices can remind psychologists that there are many ways to overcome the current uniformity of statistical practice.     

Asymptotic log-linear analysis: some cautions concerning sparse frequency tables

Traditional asymptotic probability values resulting from log-linear analyses of sparse frequency tables are often much too large. Asymptotic probability values for chi-squared and likelihood-ratio statistics are compared to nonasymptotic and exact probability values for selected log-linear models. The asymptotic probability values are all too often substantially larger than the exact probability values for the analysis of sparse frequency tables. An exact nondirectional permutation method is presented to analyze combined independent multinomial distributions. Exact nondirectional permutation methods to analyze hypergeometric distributions associated with r-way frequency tables are confined to r = 2.     

Asymptotic confidence intervals for the Pearson correlation via skewness and kurtosis

When bivariate normality is violated, the default confidence interval of the Pearson correlation can be inaccurate. Two new methods were developed based on the asymptotic sampling distribution of Fisher's z′ under the general case where bivariate normality need not be assumed. In Monte Carlo simulations, the most successful of these methods relied on the (Vale & Maurelli, 1983, Psychometrika, 48, 465) family to approximate a distribution via the marginal skewness and kurtosis of the sample data. In Simulation 1, this method provided more accurate confidence intervals of the correlation in non-normal data, at least as compared to no adjustment of the Fisher z′ interval, or to adjustment via the sample joint moments. In Simulation 2, this approximate distribution method performed favourably relative to common non-parametric bootstrap methods, but its performance was mixed relative to an observed imposed bootstrap and two other robust methods (PM1 and HC4). No method was completely satisfactory. An advantage of the approximate distribution method, though, is that it can be implemented even without access to raw data if sample skewness and kurtosis are reported, making the method particularly useful for meta-analysis. Supporting information includes R code.     

On determining the reliability and significance of a tetrachoric coefficient of correlation

In this note are presented facilitating tables for the estimation of the standard error of a tetrachoric and also tables providing significant and very significant tetrachoric coefficients for various sizes of samples and various combinations of proportions in the dichotomized distributions.The task of computing the values in the accompanying tables should be credited to Mr. Lyons.     

Calculation of signal detection theory measures

Signal detection theory (SDT) may be applied to any area of psychology in which two different types of stimuli must be discriminated. We describe several of these areas and the advantages that can be realized through the application of SDT. Three of the most popular tasks used to study discriminability are then discussed, together with the measures that SDT prescribes for quantifying performance in these tasks. Mathematical formulae for the measures are presented, as are methods for calculating the measures with lookup tables, computer software specifically developed for SDT applications, and general purpose computer software (including spreadsheets and statistical analysis software).     

Comparative Optimism and Event Skewness

Comparative judgment biases—wherein a majority of people report being above‐ or below‐average in their abilities, traits, or future events—are a robust phenomenon in psychology. A recent explanation for these biases has focused on people's awareness that many comparative judgment domains form skewed distributions, and, hence, a majority of people can feasibly be above or below average. Indeed, this prior research found that comparative biases for abilities emerged more for skewed (vs. normal) distributions. In the current research, we attempted to (i) conceptually replicate this finding in a comparative likelihood context and (ii) provide evidence of an alternative explanation for the prior results. Replicating prior research, three correlational studies and one experimental study found that event skewness was related to direct comparative likelihood judgments for health events, such that comparative optimism emerged more for events judged or manipulated to come from positively skewed distributions than from negatively skewed distributions. However, event skewness was unrelated to indirect comparisons (absolute self minus absolute other). Moreover, consistent with an egocentric‐processes account, absolute self‐judgments were more predictive of direct comparisons than were absolute other judgments and showed the same association with event skewness as direct comparisons. Implications for explaining and interpreting comparative judgment biases are discussed. Copyright © 2016 John Wiley & Sons, Ltd.     

Should we rely on the Kenward–Roger approximation when using linear mixed models if the groups have different distributions?

The study explores the robustness to violations of normality and sphericity of linear mixed models when they are used with the Kenward–Roger procedure (KR) in split‐plot designs in which the groups have different distributions and sample sizes are small. The focus is on examining the effect of skewness and kurtosis. To this end, a Monte Carlo simulation study was carried out, involving a split‐plot design with three levels of the between‐subjects grouping factor and four levels of the within‐subjects factor. The results show that: (1) the violation of the sphericity assumption did not affect KR robustness when the assumption of normality was not fulfilled; (2) the robustness of the KR procedure decreased as skewness in the distributions increased, there being no strong effect of kurtosis; and (3) the type of pairing between kurtosis and group size was shown to be a relevant variable to consider when using this procedure, especially when pairing is positive (i.e., when the largest group is associated with the largest value of the kurtosis coefficient and the smallest group with its smallest value). The KR procedure can be a good option for analysing repeated‐measures data when the groups have different distributions, provided the total sample sizes are 45 or larger and the data are not highly or extremely skewed.     

Percentage points of Wilks'L mvc andL vc criteria

Likelihood ratio tests have been proposed by Wilks for testing the hypothesis of equal means, variances, and covariances (H _mvc) and the hypothesis of equal variances and covariances (H _vc) in ap-variate normal distribution. Using exact distributions of the appropriate likelihood ratio statistics, tables of the .05 and .01 points of these distributions are constructed forp = 4, 5, 6, 7 and sample sizen = 25 (5) 60 (10) 100. A correction factor is recommended for largern. Two numerical examples illustrate use of the tables. A nonparametric test is proposed forH _mvc when the multivariate parent population is known to be non-normal.This research was supported partly by the Office of Naval Research under Contract No. Nonr-855(06) and partly by the United States Air Force through the Air Force Office of Scientific Research of the Air Research and Development Command, under Contract No. 18(600)-83. Reproduction in whole or in part for any purpose of the United States Government is permitted.     

Test skewness and kurtosis as functions of item parameters

Indexes of skewness and kurtosis for a test-score distribution are expressed in terms of item parameters. Both are shown to depend, in part, on item means, variances, and covariances. The index of skewness depends also on trivariances. A trivariance is a product moment involving first powers of deviation scores for three items. The index of kurtosis depends on quadrivariances, as well as trivariances. A quadrivariance is a product moment involving first powers of deviation scores for four items. Empirical data are presented for responses of groups of subjects to 25 triads and 25 tetrads of items from five tests.Certain parts of this article represent the results of doctoral research conducted by Hundleby and Goldstein under the direction of Ray in the Department of Psychology at Pennsylvania State University. The authors are indebted to Professor Lester Guest and Professor William Lepley for their supervisory assistance in the final stages of the two dissertations during the absence of the senior author.     

Accurate tests of statistical significance for r(WG) and average deviation interrater agreement indexes

The authors demonstrated that the most common statistical significance test used with r(WG)-type interrater agreement indexes in applied psychology, based on the chi-square distribution, is flawed and inaccurate. The chi-square test is shown to be extremely conservative even for modest, standard significance levels (e.g., .05). The authors present an alternative statistical significance test, based on Monte Carlo procedures, that produces the equivalent of an approximate randomization test for the null hypothesis that the actual distribution of responding is rectangular and demonstrate its superiority to the chi-square test. Finally, the authors provide tables of critical values and offer downloadable software to implement the approximate randomization test for r(WG)-type and for average deviation (AD)-type interrater agreement indexes. The implications of these results for studying a broad range of interrater agreement problems in applied psychology are discussed.     

A Cathode Ray Oscillograph Apparatus for the Psychological Laboratory

A computer simulation which generated various sample distributions was performed. The equal interval assumption and degree of skewness were systematically manipulated to examine the sensitivity of the t and Mann-Whitney tests to varying degrees of violations of these t test assumptions. Results show that of the 348 sets (2436 t tests), 94 (27%) contained shifts from significant to nonsignificant t test values under mild violations of the equal-interval and normality assumption and would have led to decision errors. Of these, 55% would have been Type I and 45% would have been Type II. The implications of these results are discussed relative to Types I and II errors, to the use of parametric and nonparametric statistics, and to the likelihood of encountering such problems in sampled distributions.     

Four improved statistics for contrasting means by correcting skewness and kurtosis

This paper is concerned with removing the influence of non‐normality in the classical t‐statistic for contrasting means. Using higher‐order expansion to quantify the effect of non‐normality, four corrected statistics are provided. Two aim to correct the mean bias and two to correct the overall distribution. The classical t‐statistic is also robust against non‐normality when the observed variables satisfy certain structures. A special case is when the marginal distributions of the contrast are independent and identically distributed.     

MMPI-2 Assessment of Positive Attributes: A Methodological Note

T-score tables for the Minnesota Multiphasic Personality Inventory-2 (MMPI2; Butcher, Dahlstrom, Graham, Tetlegen, & Kaemmer, 1989) provide no values lower than 30. Use. of these tables with measures of positive psychological attributes in clinical samples can produce T-score distributions that are markedly truncated at the low end, Data presented in this article show that the statistical characteristics of several MMP1-2 scales are affected.