Using confidence intervals for graphically based data interpretation.

As a potential alternative to standard null hypothesis significance testing, we describe methods for graphical presentation of data--particularly condition means and their corresponding confidence intervals--for a wide range of factorial designs used in experimental psychology. We describe and illustrate confidence intervals specifically appropriate for between-subject versus within-subject factors. For designs involving more than two levels of a factor, we describe the use of contrasts for graphical illustration of theoretically meaningful components of main effects and interactions. These graphical techniques lend themselves to a natural and straightforward assessment of statistical power.     

Confidence intervals for standardized linear contrasts of means

Most psychology journals now require authors to report a sample value of effect size along with hypothesis testing results. The sample effect size value can be misleading because it contains sampling error. Authors often incorrectly interpret the sample effect size as if it were the population effect size. A simple solution to this problem is to report a confidence interval for the population value of the effect size. Standardized linear contrasts of means are useful measures of effect size in a wide variety of research applications. New confidence intervals for standardized linear contrasts of means are developed and may be applied to between-subjects designs, within-subjects designs, or mixed designs. The proposed confidence interval methods are easy to compute, do not require equal population variances, and perform better than the currently available methods when the population variances are not equal.     

Point-biserial correlation: Interval estimation,hypothesis testing,meta-analysis,and sample size determination

The point-biserial correlation is a commonly used measure of effect size in two-group designs. New estimators of point-biserial correlation are derived from different forms of a standardized mean difference. Point-biserial correlations are defined for designs with either fixed or random group sample sizes and can accommodate unequal variances. Confidence intervals and standard errors for the point-biserial correlation estimators are derived from the sampling distributions for pooled-variance and separate-variance versions of a standardized mean difference. The proposed point-biserial confidence intervals can be used to conduct directional two-sided tests, equivalence tests, directional non-equivalence tests, and non-inferiority tests. A confidence interval for an average point-biserial correlation in meta-analysis applications performs substantially better than the currently used methods. Sample size formulas for estimating a point-biserial correlation with desired precision and testing a point-biserial correlation with desired power are proposed. R functions are provided that can be used to compute the proposed confidence intervals and sample size formulas.     

Testing the multiplicative hypothesis of intensifiers

The way in which intensifiers affect the meaning of the phrases to which they are applied is investigated. Previous research demonstrates that they often have little impact in between-subject designs. Here we present research which demonstrates that their effects in within-subject designs do not conform to the multiplicative hypothesis. Both the psychological mechanisms employed and the implications for survey methodologists are discussed.     

The generalisability of confidence-accuracy studies in eyewitnessing

Abstract

This paper discusses the evidence on the confidence-accuracy relationship in eyewitness research. It is pointed out that the conclusion often drawn on the basis of such research, that there is little or no relationship between eyewitness confidence and accuracy, is an unwarranted generalisation based on the use of experimental paradigms that are limited in terms of their generalisability to courtroom situations. In particular, almost all studies involve between-subject rather than within-subject designs, thereby limiting the generalisability of findings. A within-subjects analysis examines whether, within an individual, more confident responses are associated with greater accuracy than are less confident responses. A between-subjects analysis examines whether a more confident individual is likely to be more accurate than a less confident individual. A further limitation on the generalisability to real life situations of studies conducted to date is that experiments must involve making errors in identification in order to allow correlational analysis to take place. This means that findings cannot be generalised to those real life situations where all subjects are likely to be completely accurate and confident.     

Fisher transformation based confidence intervals of correlations in fixed- and random-effects meta-analysis

Meta-analyses of correlation coefficients are an important technique to integrate results from many cross-sectional and longitudinal research designs. Uncertainty in pooled estimates is typically assessed with the help of confidence intervals, which can double as hypothesis tests for two-sided hypotheses about the underlying correlation. A standard approach to construct confidence intervals for the main effect is the Hedges-Olkin-Vevea Fisher-z (HOVz) approach, which is based on the Fisher-z transformation. Results from previous studies (Field, 2005, Psychol. Meth., 10, 444; Hafdahl and Williams, 2009, Psychol. Meth., 14, 24), however, indicate that in random-effects models the performance of the HOVz confidence interval can be unsatisfactory. To this end, we propose improvements of the HOVz approach, which are based on enhanced variance estimators for the main effect estimate. In order to study the coverage of the new confidence intervals in both fixed- and random-effects meta-analysis models, we perform an extensive simulation study, comparing them to established approaches. Data were generated via a truncated normal and beta distribution model. The results show that our newly proposed confidence intervals based on a Knapp-Hartung-type variance estimator or robust heteroscedasticity consistent sandwich estimators in combination with the integral z-to-r transformation (Hafdahl, 2009, Br. J. Math. Stat. Psychol., 62, 233) provide more accurate coverage than existing approaches in most scenarios, especially in the more appropriate beta distribution simulation model.     

基于概化理论的方差分量变异量估计

概化理论广泛应用于心理与教育测量实践中, 方差分量估计是进行概化理论分析的关键。方差分量估计受限于抽样, 需要对其变异量进行探讨。采用蒙特卡洛(Monte Carlo)数据模拟技术, 在正态分布下讨论不同方法对基于概化理论的方差分量变异量估计的影响。结果表明: Jackknife方法在方差分量变异量估计上不足取; 不采取Bootstrap方法的“分而治之”策略, 从总体上看, Traditional方法和有先验信息的MCMC方法在标准误及置信区间这两个变异量估计上优势明显。     

An alternative to Cohen's standardized mean difference effect size: a robust parameter and confidence interval in the two independent groups case

The authors argue that a robust version of Cohen's effect size constructed by replacing population means with 20% trimmed means and the population standard deviation with the square root of a 20% Winsorized variance is a better measure of population separation than is Cohen's effect size. The authors investigated coverage probability for confidence intervals for the new effect size measure. The confidence intervals were constructed by using the noncentral t distribution and the percentile bootstrap. Over the range of distributions and effect sizes investigated in the study, coverage probability was better for the percentile bootstrap confidence interval.     

A Policy-Capturing Approach to Individuals′ Decisions to Be Absent

This study provided a within-subjects assessment of the factors associated with an individual′s decision to be absent and examined whether there were differences between individuals in their decisions. A sample of maintenance and clerical employees at a large Midwest university responded to hypothetical scenarios describing factors that might contribute to their decisions to be absent on a particular day. Illness explained mare variance than any other factor in individual′s absence decisions. Several other within-subject and between-subject influences were identified. Results from a cluster analysis suggested that the importance of these factors differed between subgroups of individuals, lending support to Johns and Nicholson′s (1982) argument that the meaning of absence is not the same for all individuals.     

Calculating and graphing within-subject confidence intervals for ANOVA

The psychological and statistical literature contains several proposals for calculating and plotting confidence intervals (CIs) for within-subjects (repeated measures) ANOVA designs. A key distinction is between intervals supporting inference about patterns of means (and differences between pairs of means, in particular) and those supporting inferences about individual means. In this report, it is argued that CIs for the former are best accomplished by adapting intervals proposed by Cousineau (Tutorials in Quantitative Methods for Psychology, 1, 42–45, 2005) and Morey (Tutorials in Quantitative Methods for Psychology, 4, 61–64, 2008) so that nonoverlapping CIs for individual means correspond to a confidence for their difference that does not include zero. CIs for the latter can be accomplished by fitting a multilevel model. In situations in which both types of inference are of interest, the use of a two-tiered CI is recommended. Free, open-source, cross-platform software for such interval estimates and plots (and for some common alternatives) is provided in the form of R functions for one-way within-subjects and two-way mixed ANOVA designs. These functions provide an easy-to-use solution to the difficult problem of calculating and displaying within-subjects CIs.     

Center-of-pressure regularity as a marker for attentional investment in postural control: A comparison between sitting and standing postures

Postural control is a highly automatized basic activity that requires limited attentional investments. These investments have been shown to increase from balancing experts to controls, and from controls to persons with impaired postural control. Such between-subject comparisons led to a proposed direct relation between the regularity of center-of-pressure (COP) fluctuations and the amount of attention invested in posture. This study aims to expand this relation to a within-subject comparison of conditions that differ in balance demands. Specifically, more regular COP fluctuations were expected for standing than sitting, as stimulus-response reaction-time studies showed that the required attentional demands are lower for sitting than standing. COP registrations were made for fifteen healthy young adults in seated and standing postures. COP regularity was quantified with sample entropy. As expected, COP fluctuations were found to be more regular for standing than sitting, as evidenced by significantly lower sample entropy values. These findings expand the relation between COP regularity and the amount of attention invested in posture to postural tasks that vary in balance demands. An assessment of COP regularity may thus not only be instrumental in the examination of attentional investment in posture in between-subject designs, but also for different postures in within-subjects designs.     

Sample size planning for composite reliability coefficients: accuracy in parameter estimation via narrow confidence intervals

Composite measures play an important role in psychology and related disciplines. Composite measures almost always have error. Correspondingly, it is important to understand the reliability of the scores from any particular composite measure. However, the point estimates of the reliability of composite measures are fallible and thus all such point estimates should be accompanied by a confidence interval. When confidence intervals are wide, there is much uncertainty in the population value of the reliability coefficient. Given the importance of reporting confidence intervals for estimates of reliability, coupled with the undesirability of wide confidence intervals, we develop methods that allow researchers to plan sample size in order to obtain narrow confidence intervals for population reliability coefficients. We first discuss composite reliability coefficients and then provide a discussion on confidence interval formation for the corresponding population value. Using the accuracy in parameter estimation approach, we develop two methods to obtain accurate estimates of reliability by planning sample size. The first method provides a way to plan sample size so that the expected confidence interval width for the population reliability coefficient is sufficiently narrow. The second method ensures that the confidence interval width will be sufficiently narrow with some desired degree of assurance (e.g., 99% assurance that the 95% confidence interval for the population reliability coefficient will be less than W units wide). The effectiveness of our methods was verified with Monte Carlo simulation studies. We demonstrate how to easily implement the methods with easy-to-use and freely available software.     

Accuracy in parameter estimation for targeted effects in structural equation modeling: sample size planning for narrow confidence intervals

In addition to evaluating a structural equation model (SEM) as a whole, often the model parameters are of interest and confidence intervals for those parameters are formed. Given a model with a good overall fit, it is entirely possible for the targeted effects of interest to have very wide confidence intervals, thus giving little information about the magnitude of the population targeted effects. With the goal of obtaining sufficiently narrow confidence intervals for the model parameters of interest, sample size planning methods for SEM are developed from the accuracy in parameter estimation approach. One method plans for the sample size so that the expected confidence interval width is sufficiently narrow. An extended procedure ensures that the obtained confidence interval will be no wider than desired, with some specified degree of assurance. A Monte Carlo simulation study was conducted that verified the effectiveness of the procedures in realistic situations. The methods developed have been implemented in the MBESS package in R so that they can be easily applied by researchers.     

Confidence interval-based sample size determination formulas and some mathematical properties for hierarchical data

The use of hierarchical data (also called multilevel data or clustered data) is common in behavioural and psychological research when data of lower-level units (e.g., students, clients, repeated measures) are nested within clusters or higher-level units (e.g., classes, hospitals, individuals). Over the past 25 years we have seen great advances in methods for computing the sample sizes needed to obtain the desired statistical properties for such data in experimental evaluations. The present research provides closed-form and iterative formulas for sample size determination that can be used to ensure the desired width of confidence intervals for hierarchical data. Formulas are provided for a four-level hierarchical linear model that assumes slope variances and inclusion of covariates under both balanced and unbalanced designs. In addition, we address several mathematical properties relating to sample size determination for hierarchical data via the standard errors of experimental effect estimates. These include the relative impact of several indices (e.g., random intercept or slope variance at each level) on standard errors, asymptotic standard errors, minimum required values at the highest level, and generalized expressions of standard errors for designs with any-level randomization under any number of levels. In particular, information on the minimum required values will help researchers to minimize the risk of conducting experiments that are statistically unlikely to show the presence of an experimental effect.     

The unreliable change of reliable change indices

The classic method for assessment of reliable change, in 1991 re-introduced as Jacobson's RC, can be characterized as a confidence interval method. In recent years, several RC indices have been proposed using Kelley's (1947) (Kelley, T. L. (1947). Fundamentals of statistics. Cambridge: Harvard University Press) formula for estimating true change. In these proposals, interval estimation and confidence intervals are mixed up, which leads to unjustified probability statements. When Kelley's estimate is correctly expanded into a normal distributed statistic, the classic approach reveals itself as a large sample approximation of a properly constructed RCI based on Kelley's formula. Researchers should continue using the classic approach for the determination of reliable change.     

Confidence intervals and sample size calculations for the weighted eta-squared effect sizes in one-way heteroscedastic ANOVA

Effect size reporting and interpreting practices have been extensively recommended in academic journals when primary outcomes of all empirical studies have been analyzed. This article presents an alternative approach to constructing confidence intervals of the weighted eta-squared effect size within the context of one-way heteroscedastic ANOVA models. It is shown that the proposed interval procedure has advantages over an existing method in its theoretical justification, computational simplicity, and numerical performance. For design planning, the corresponding sample size procedures for precise interval estimation of the weighted eta-squared association measure are also delineated. Specifically, the developed formulas compute the necessary sample sizes with respect to the considerations of expected confidence interval width and tolerance probability of interval width within a designated value. Supplementary computer programs are provided to aid the implementation of the suggested techniques in practical applications of ANOVA designs when the assumption of homogeneous variances is not tenable.     

A score method of constructing asymmetric confidence intervals for the mean of a rating scale item

This article presents a generalization of the Score method of constructing confidence intervals for the population proportion (E. B. Wilson, 1927) to the case of the population mean of a rating scale item. A simulation study was conducted to assess the properties of the Score confidence interval in relation to the traditional Wald (A. Wald, 1943) confidence interval under a variety of conditions, including sample size, number of response options, extremeness of the population mean, and kurtosis of the response distribution. The results of the simulation study indicated that the Score interval usually outperformed the Wald interval, suggesting that the Score interval is a viable method of constructing confidence intervals for the population mean of a rating scale item.     

Will the real relationship between facial expression and affective experience please stand up: The case of exhilaration

Abstract

It is hypothesised that the empirical correlation between facial expression and affective experience varies as a function of the correlational design used to compute the coefficients. Predictions about the rank order of five designs were derived based on two assumptions. Female subjects were placed into one of three alcohol conditions (no ethanol, low dose, high dose) and were exposed to 30 slides containing jokes or cartoons. The degree of rated funniness and overt behaviour were intercorrelated using five different designs to analyse the same set of data. The results show that within-subject analyses yielded higher coefficients than between-subjects analyses. Aggregation of data increased the coefficients for within-subject analyses, but not for between-subject analyses. A cheerful mood was associated with hyper-expressiveness, i.e. the occurrence of smiling and laughter at relatively low levels of perceived funniness. It was demonstrated that low correlations between facial expression and affective experience may be based on several method artefacts.     

SIMSTAT: Bootstrap computer simulation and statistical program for IBM personal computers

SIMSTAT is a computer program that provides bootstrap confidence intervals and estimates distribution parameters for 27 univariate and bivariate statistics. Because it permits variation of the bootstrap sample size, it is a convenient tool for research planning, allowing comparison of power estimates and estimator precision for various estimators and/or sample sizes. SIMSTAT also is a useful tool for studying the robustness of various statistics on real-world data and may be helpful in teaching statistical principles.