Determining the power of multiple regression analyses both with and without repeated measures

Power analysis guides researchers in planning how much data to collect. This article describes BWPower, a computer program for the Windows 95 environment that performs power analyses for research designs that may or may not include both between- and within-subjects factors. We discuss how BWPower easily accommodates both between- and within-subjects factors and provide examples of BWPower’s use in performing power analyses on designs with only between-subjects factors, designs with only repeated measures, and with mixed between- and within-subjects designs. We highlight the major features of BWPower’s user interface, such as the ability to iteratively increment or decrement the number of subjects and the automatic recalculation of power when the number of subjects or effect sizes is changed.     

Null hypothesis significance testing and Type I error: The domain problem

Although many common uses of p-values for making statistical inferences in contemporary scientific research have been shown to be invalid, no one, to our knowledge, has adequately assessed the main original justification for their use, which is that they can help to control the Type I error rate (Neyman & Pearson, 1928, 1933). We address this issue head-on by asking a specific question: Across what domain, specifically, do we wish to control the Type I error rate? For example, do we wish to control it across all of science, across all of a specific discipline such as psychology, across a researcher's active lifetime, across a substantive research area, across an experiment, or across a set of hypotheses? In attempting to answer these questions, we show that each one leads to troubling dilemmas wherein controlling the Type I error rate turns out to be inconsistent with other scientific desiderata. This inconsistency implies that we must make a choice. In our view, the other scientific desiderata are much more valuable than controlling the Type I error rate and so it is the latter, rather than the former, with which we must dispense. But by doing so—that is, by eliminating the Type I error justification for computing and using p-values—there is even less reason to believe that p is useful for validly rejecting null hypotheses than previous critics have suggested.     

The power and sensitivity of four core driver workload measures for benchmarking the distraction potential of new driver vehicle interfaces

This study evaluated the power and sensitivity of several core driver workload measures in order to better understand their use as a component of future driver distraction potential evaluation procedures of the in-vehicle human machine interface (HMI). Driving is a task that requires visual, manual and cognitive resources to perform. Secondary tasks, such as mobile phone use and interaction with in-built navigation, which load onto any of these three processing resources increase driver workload and can lead to impaired driving. Because workload and distraction potential are interrelated, a comprehensive method to assess driver workload that produces valid and predictive results is needed to advance the science of distraction potential evaluation. It is also needed to incorporate into New Car Assessment Program (NCAP) testing regimes. Workload measures of cognitive (DRT [Detection Response Task] Reaction Time), visual (DRT Miss Rate), subjective (NASA-TLX [driver workload questionnaire]), and temporal demand (Task Interaction Time) were collected as participants drove one of 40 vehicles while completing a variety of secondary tasks with varying interaction requirements. Of the evaluated measures, variance and power analyses demonstrated that Task Interaction Time is the most sensitive in detecting differences in driver workload between different in-vehicle HMIs, followed by DRT Miss Rate, NASA-TLX and finally DRT Reaction Time. There were relatively weak correlations between each of the four measures. These results suggest that Task Interaction Time, coupled with a reliable visual demand metric such as DRT Miss Rate, eye glance coding, or visual occlusion, more efficiently detect differences in driver workload between different HMIs compared to DRT Reaction Time and the NASA-TLX questionnaire. These results can be used to improve the understanding of the utility of each of these core driver workload measures in assessing driver distraction potential.     

Variable error: variance-covariance heterogeneity, block size and type L error rates

The degree to which blocked (VE) data satisfies the assumptions of compound symmetry required for a repeated measures ANOVA was studied. Monte Carlo procedures were used to study the effect of violation of this assumption, under varying block sizes, on the Type l error rate. Populations of 10,000 subjects for each of two groups, the underlying variance-covariance matrices reflecting a specific condition of violation of the homogeneity of covariance assumptions, was generated based on each of three actual experimental data sets. The data were blocked in various ways, VE calculated, and subsequently analyzed by a repeated measures ANOVA. The complete process was replicated for four covariance homogeneity conditions for each of the three data sets, resulting in a total of 22,000 simulated experiments. Results indicated that the Type l error rate increases as the degree of heterogeneity within the variance-covariance matrices increases when raw (unblocked) data are analyzed. With VE, the effects of within-matrix heterogeneity on the Type l error rate are inconclusive. However, block size does seem to affect the probability of obtaining a significant interaction, but the nature of this relationship is not clear as there does not appear to be any consistent relationship between the size of the block and the probability of obtaining significance. For both raw and VE data there was no inflation in the number of Type l errors when the covariances within a given matrix were homogeneous, regardless of the differences between the group variance-covariance matrices.     

Reaction times and error rates for judging nominal identity of letter strings

Most theories of “same”-“different” judgments predict that “same” responses should be at least as slow as “different” responses. However, the contrary has often been found. To explain this, a two-processor model has been proposed. In this model, a fast processor and a slow processor operate simultaneously. “Same” responses are initiated by whichever processor first indicates that the stimuli are “same,” whereas “different” responses are initiated only by the slow processor. In the experiment reported here, Ss judged whether two successively presented letter strings were nominally “same” or “different.” It was expected that the fast processor would be incapable of making nominal identity judgments. Thus, both “same” and “different” responses would be initiated by the slow processor. Consequently, “same” responses should have been slower than “different” responses. However, this did not occur. This finding casts doubt upon, but does not disprove, the two-processor model.     

Manual reaction times and error rates in stutterers.

In this study 18 stutterers and 18 nonstutterers were presented trials on which they should press a button as fast as possible, intermixed with trials which required no responding. Stutterers had slightly faster reaction times but also made slightly more errors, that is, they tended to press the button when they should not have done so. As neither difference was significant, it was concluded that stutterers did not differ from normal speakers in manual reaction speed, nor did they choose a different speed-accuracy trade-off criterion for the given task.     

Statistical power for the two-factor repeated measures ANOVA

Determining a priori power for univariate repeated measures (RM) ANOVA designs with two or more within-subjects factors that have different correlational patterns between the factors is currently difficult due to the unavailability of accurate methods to estimate the error variances used in power calculations. The main objective of this study was to determine the effect of the correlation between the levels in one RM factor on the power of the other RM factor. Monte Carlo simulation procedures were used to estimate power for the A, B, and AB tests of a 2×3, a 2×6, a 2×9, a 3×3, a 3×6, and a 3×9 design under varying experimental conditions of effect size (small, medium, and large), average correlation (.4 and .8), alpha (.01 and .05), and sample size (n = 5, 10 ,15, 20, 25, and 30). Results indicated that the greater the magnitude of the differences between the average correlation among the levels of Factor A and the average correlation in the AB matrix, the lower the power for Factor B (and vice versa). Equations for estimating the error variance of each test of the two-way model were constructed by examining power and mean square error trends across different correlation matrices. Support for the accuracy of these formulae is given, thus allowing for direct analytic power calculations in future studies.     

Controlling the Type I error rate by using the nonparametric bootstrap when comparing means

Of the several tests for comparing population means, the best known are the ANOVA, Welch, Brown–Forsythe, and James tests. Each performs appropriately only in certain conditions, and none performs well in every setting. Researchers, therefore, have to select the appropriate procedure and run the risk of making a bad selection and, consequently, of erroneous conclusions. It would be desirable to have a test that performs well in any situation and so obviate preliminary analysis of data. We assess and compare several tests for equality of means in a simulation study, including non‐parametric bootstrap techniques, finding that the bootstrap ANOVA and bootstrap Brown–Forsythe tests exhibit a similar and exceptionally good behaviour.     

Statistical inference for false positive and false negative error rates in mastery testing

This paper describes an asymptotic inferential procedure for the estimates of the false positive and false negative error rates. Formulas and tables are described for the computations of the standard errors. A simulation study indicates that the asymptotic standard errors may be used even with samples of 25 cases as long as the Kuder-Richardson Formula 21 reliability is reasonably large. Otherwise, a large sample would be required.This work was performed pursuant to Grant No NIE-G-78-0087 with the National Institute of Education, Department of Health, Education and Welfare, Huynh Huynh, Principal Investigator. Points of view or opinions stated do not necessarily reflect NIE position or policy and no official endorsement should be inferred. The editorial assistance of Joseph C. Saunders is gratefully acknowledged.     

Collective decision making reduces metacognitive control and increases error rates,particularly for overconfident individuals

This research aimed to investigate the changes in judgment accuracy, confidence, control thresholds, and decision outcomes when people act in two-person groups (dyads) compared with acting individually. First, we used interacting dyads to determine the metacognitive and behavioral outcomes of collective decision making and compared them with those of individuals. Second, we examined whether these changes were related to the trait-confidence and bias of individuals working together. Using a within-person design, undergraduate psychology students (N = 116) completed a General-knowledge Test individually, then together as a dyad. Each question was accompanied by a confidence rating and a decision to bet $10 on the answer. Dyads had significantly higher confidence and lower control thresholds than individuals. They were also significantly more decisive (made more bets) and reckless (lost a higher rate of bets) than when working alone. Thus, we observed a higher rate of decision errors for groups than individuals. The results also demonstrated the important role of individual differences: Overconfident individuals became even more confident, decisive, and reckless when working together compared with less confident or underconfident individuals working together. These findings have important theoretical and applied implications for collective decision making; metacognitive bias and potentially control thresholds may be targeted to alleviate the larger error rates and guide the formation of more effective groups.     

Impaired word naming in right-brain-damaged right-handers: error types and time-course analyses

The purpose of this study is to search for cues that could account for the fact that right-hemisphere lesions affect word naming when the criterion for naming is semantic. The authors analyzed the errors as well as the time course of productions of 35 vascular right-brain-damaged subjects (RBD) and 20 control subjects (C) in a semantic-based word-naming task. Results show that there are no differences between groups in terms of (a) the number of errors produced and, (b) the patterns of error types. However, results also show a reduction of verbal fluency for semantic criteria in RBD-subjects present only after the first 30 seconds of production. These results indicate that reduction of word production in RBD-subjects is not the consequence of nonspecific factors such as perseveration or spontaneity, but reflects problems with the less automatized processes permitting exploration of semantic organization, either because scanning process are affected, or because the presence of a discreet semantic impairment prevents scanning from being as efficient. Given that efficient exploration of semantic knowledge is important for lexico-semantic functioning, the present results suggest that the right hemisphere contributes to some aspects of lexico-semantic processing necessary for language production.     

糖尿病的免疫抑制治疗

淡化糖尿病的分型为我们提供了一个全新的思维空间.即传统的1型和至少部分2型糖尿病可能都是一种自身免疫性疾病,而目前流行的炎症学说可能是对认可免疫损伤是糖尿病发病理论的一个过渡,我们应该重新审视糖尿病的早期治疗和研究其慢性并发症的发病机理.我们则采用小剂量、长时间的治疗理念,早期接受这种免疫治疗的患者出现低血糖的症状,遂减少胰岛素的剂量;再次出现低血糖,我们就继续减少胰岛素的荆量.结果,有部分患者完全摆脱了胰岛素.我们对糖尿病的再认识是建立在对多脏器活检的基础之上.我们的活检结果表明,糖尿病肾病存在着免疫损伤.糖尿病眼底病变是使患者致盲的主要病因,糖尿病患者的眼底病变是一种自身免疫损伤的结果.我们尝试用小剂量的免疫抑制剂环孢茵素A(25 mg bid)治疗糖尿病眼底出血,取得了不错的疗效,也证实糖屎病眼底病变是一种血管炎,与自身免疫痛变有关.其他的器官还包括垂体、大脑的血管、糖尿痛足、冠心痛、皮肤的病变和肌肉活检的免疫组化也均有证据表明,免疫损伤是导致多种慢性并发症的罪魁祸首,是一种多器官免疫损伤的结果无论是1型糖尿痛,还是2型糖尿病,肌细胞表面或多或少均有免疫复合物沉积.这些免疫复合物的存在势必会影响胰岛素与肌肉细胞表面的胰岛素受体结合,即胰岛素抵抗的存在.为此,我们在临床上广泛采用了免疫抑制治疗的理念治疗糖尿病.应用小剂量胰岛素和小剂量环孢素A治疗早期发现的糖尿病取得了不错的疗效.唯一担心的副作用发生在肝脏,即部分糖尿病患者会出现总胆红索升高,而也有一部分患者总胆红素不升高.     

Conjunction error rates on a continuous recognition memory test: little evidence for recollection

Two experiments examined conjunction memory errors on a continuous recognition task where the lag between parent words (e.g., blackmail, jailbird) and later conjunction lures (blackbird) was manipulated. In Experiment 1, contrary to expectations, the conjunction error rate was highest at the shortest lag (1 word) and decreased as the lag increased. In Experiment 2 the conjunction error rate increased significantly from a 0- to a 1-word lag, then decreased slightly from a 1- to a 5-word lag. The results provide mixed support for simple familiarity and dual-process accounts of recognition. Paradoxically, searching for an item in memory does not appear to be a good encoding task.     

Bias and error rates for premorbid IQ estimators: comment on Veiel and Koopman (2001)

H. O. F. Veiel and R. F. Koopman (2001) advance statistical and legal theses. They correctly point out that the usual regression formula for estimating a pre-event IQ underestimates high IQs and overestimates low IQs (due to regression to the mean). They call this a conditional bias and show it can be sizeable. The author takes issue with their claim that a new estimator they propose should be used in place of the usual formulas, because it negates this statistical bias. Their argument against the usual estimator conflates statistical bias and legal bias. Their discussion in favor of their new estimator mentions, but does not derive a general formula for, a gross loss of precision entailed by use of the new estimator. The author quantifies this loss of precision and, using Veiel and Koopman's numerical example, shows that their estimator quadruples error.     

Statistical power analyses using G<Superscript>*</Superscript>Power 3.1: Tests for correlation and regression analyses

G^*Power is a free power analysis program for a variety of statistical tests. We present extensions and improvements of the version introduced by Faul, Erdfelder, Lang, and Buchner (2007) in the domain of correlation and regression analyses. In the new version, we have added procedures to analyze the power of tests based on (1) single-sample tetrachoric correlations, (2) comparisons of dependent correlations, (3) bivariate linear regression, (4) multiple linear regression based on the random predictor model, (5) logistic regression, and (6) Poisson regression. We describe these new features and provide a brief introduction to their scope and handling.     

Short-term recognition memory for pitch: Effect of a priori probability on response times and error rates

A previous finding (Bindra, Williams, & Wise, 1965) that Ss respond “different” more quickly than “same” when comparing the pitch of two tones was confirmed. It was found that this effect could be enhanced by making same trials less frequent and that the effect could be reversed by making same trials more frequent. The experimental treatment also affected the error rates of the two responses. Error and latency results are discussed in terms of a model of R T based upon a sequential-sampling and decision procedure, and are shown to be compatible with such a model. A decision bias towards the response “different” must be taken into account when the model is applied. This bias appears to be influenced by the size of the interstimulus interval used.