Visual statistical decisions

To identify variables that underlie intuitive judgments about the sizes of groups of similar objects, we asked people to judge the relative heights of vertical bars briefly shown, two groups at a time, on a computer display. Randomly selected normal deviates determined individual bar height. Average differences in height and group sizes were also randomly varied. Twenty-eight participants judged 250 differences each, which were then submitted to multiple regression analysis and psychophysical inspection. The total number of bars sharpened discrimination, whereas variance dulled it. Critical ratio (CR), the forerunner to the modern t test, emerged as the most important predictor; little additional variance was explained by other factors. The difference in the number of bars was a reliable factor, favoring the greater number of bars. Confidence limits around thresholds, defined as CRs needed to say “possibly greater,” surrounded 1.65; as a z value, this corresponded to a one-tailed probability of .05. Judgments about noisy stimuli thus seem to be based on a statistical process and to employ a probability criterion similar to that used in the formal statistical evaluation of experimental findings—namely, p<.05.     

The life of p: “Just significant” results are on the rise

Null hypothesis significance testing uses the seemingly arbitrary probability of .05 as a means of objectively determining whether a tested effect is reliable. Within recent psychological articles, research has found an overrepresentation of p values around this cut-off. The present study examined whether this overrepresentation is a product of recent pressure to publish or whether it has existed throughout psychological research. Articles published in 1965 and 2005 from two prominent psychology journals were examined. Like previous research, the frequency of p values at and just below .05 was greater than expected compared to p frequencies in other ranges. While this overrepresentation was found for values published in both 1965 and 2005, it was much greater in 2005. Additionally, p values close to but over .05 were more likely to be rounded down to, or incorrectly reported as, significant in 2005 than in 1965. Modern statistical software and an increased pressure to publish may explain this pattern. The problem may be alleviated by reduced reliance on p values and increased reporting of confidence intervals and effect sizes.     

A REASSESSMENT OF STATISTICAL POWER ANALYSIS IN HUMAN COMMUNICATION RESEARCH

This study conducted a statistical power analysis of 64 articles appearing in the first four volumes of Human Communication Research, 1974–1978. Each article was examined, using Cohen's revised handbook, assuming nondirectional null hypotheses and an alpha level of .05. Statistical power, the probability of rejecting a false null hypothesis, was calculated for small, medium, and large experimental effect sizes and averaged by article and volume. Results indicated that the average probability of beta errors appears to have decreased over time, providing a greater chance of rejecting false null hypotheses, but this also raised several power-related issues relevant to communication research in general.     

16PF in screening for appropriateness of mentors

The utility of the Sixteen Personality Factor Questionnaire, Fifth Edition (16PF) as an indicator of mentor effectiveness was examined. A random sample of the 16PF scores of 74 mentors was drawn from a population of 837 mentors from Big Brothers Big Sisters. Caseworkers rated mentor's effectiveness using a rubric developed for this purpose. The rubric showed good interrater agreement. Caseworkers' ratings of mentor's effectiveness was used to rate mentors systematically as appropriate or inappropriate. The 16PF scores of mentors were compared at an alpha level of .05 for appropriate and inappropriate groups using independent t tests and multivariate analyses of variance, which reflected significant differences between male and female mentors on Factors E and Q3. Significant differences were also found between "appropriate" and "inappropriate" mentors on Factors L and Q4. These differences reflected only moderate effect sizes and lacked practical significance or meaning. The results suggest that, while the 16PF discriminates statistically between "appropriate" and "inappropriate" mentors, in terms of practical significance, the questionnaire is not particularly useful as an initial screening tool.     

Researchers misunderstand confidence intervals and standard error bars

Little is known about researchers' understanding of confidence intervals (CIs) and standard error (SE) bars. Authors of journal articles in psychology, behavioral neuroscience, and medicine were invited to visit a Web site where they adjusted a figure until they judged 2 means, with error bars, to be just statistically significantly different (p < .05). Results from 473 respondents suggest that many leading researchers have severe misconceptions about how error bars relate to statistical significance, do not adequately distinguish CIs and SE bars, and do not appreciate the importance of whether the 2 means are independent or come from a repeated measures design. Better guidelines for researchers and less ambiguous graphical conventions are needed before the advantages of CIs for research communication can be realized.     

Comparing Data Sets: Implicit Summaries of the Statistical Properties of Number Sets

Comparing datasets, that is, sets of numbers in context, is a critical skill in higher order cognition. Although much is known about how people compare single numbers, little is known about how number sets are represented and compared. We investigated how subjects compared datasets that varied in their statistical properties, including ratio of means, coefficient of variation, and number of observations, by measuring eye fixations, accuracy, and confidence when assessing differences between number sets. Results indicated that participants implicitly create and compare approximate summary values that include information about mean and variance, with no evidence of explicit calculation. Accuracy and confidence increased, while the number of fixations decreased as sets became more distinct (i.e., as mean ratios increase and variance decreases), demonstrating that the statistical properties of datasets were highly related to comparisons. The discussion includes a model proposing how reasoners summarize and compare datasets within the architecture for approximate number representation.     

The Power to Detect and Predict Individual Differences in Intra-Individual Variability Using the Mixed-Effects Location-Scale Model

Our goal is to provide empirical scientists with practical tools and advice with which to test hypotheses related to individual differences in intra-individual variability using the mixed-effects location-scale model. To that end, we evaluate Type I error rates and power to detect and predict individual differences in intra-individual variability using this model and provide empirically-based guidelines for building scale models that include random and/or systematically-varying fixed effects. We also provide two power simulation programs that allow researchers to conduct a priori empirical power analyses. Our results aligned with statistical power theory, in that, greater power was observed for designs with more individuals, more repeated occasions, greater proportions of variance available to be explained, and larger effect sizes. In addition, our results indicated that Type I error rates were acceptable in situations when individual differences in intra-individual variability were not initially detectable as well as when the scale-model individual-level predictor explained all initially detectable individual differences in intra-individual variability. We conclude our paper by providing study design and model building advice for those interested in using the mixed-effects location-scale model in practice.     

Weighted vest effects on impact forces and joint work during vertical jump landings in men and women

Weighted vest (WV) use during vertical jump landings (VJL) does not appear to alter peak vertical ground reaction forces (GRF) or peak joint torques. However, WV effects on joint work and sex differences during VJL are not well understood. This study assessed WV effects on vertical GRF and sagittal joint work during VJL in men and women. Twelve men and 12 women performed VJL wearing a WV with zero added mass (unloaded) and with 10% body mass (loaded) while GRF and kinematic data were obtained. Mixed-model analyses of variance (α = 0.05) and effect sizes (ES) were used to assess differences between sexes and/or load conditions. Regardless of sex, greater landing height (p < 0.001; ES = 0.37) and peak vertical GRF (p = 0.001; ES 0.51) occurred when unloaded, while greater landing time (p = 0.001; ES = 0.46) and negative lower extremity work (p < 0.001; ES = 0.41) occurred when loaded through greater negative work about the hip (p = 0.001; ES = 0.27) and ankle (p = 0.020; ES = 0.27). No differences in hip (p = 0.753; ES = 0.03), knee (p = 0.588; ES = 0.07), or ankle (p = 0.580; ES = 0.09) joint displacement were detected between loaded and unloaded conditions. Men exhibited greater landing heights (p < 0.001; ES = 2.49) and greater peak vertical GRF than women (p = 0.007; ES = 1.18), though women exhibited greater negative lower extremity work (p < 0.001; ES = 1.98) than men through greater negative knee (p < 0.001; ES = 1.98) and ankle (p = 0.032; ES = 0.94) work. No sex differences were detected for joint angular displacement about the hip (p = 0.475; ES = 0.30), knee (p = 0.666; ES = 0.18), or ankle (p = 0.084; ES = 0.71). These data revealed a unique load accommodation strategy during VJL with a WV characterized by greater lower extremity joint work performed via increased joint torque despite lesser landing height and peak vertical GRF. Women appear to perform greater lower extremity joint work than men during VJL despite lesser landing height and peak vertical GRF. Current and prospective WV users should be aware of their load accommodation strategy during VJL with an external load. Women may consider developing more refined load accommodation strategies for VJL regardless of whether external loading is applied to avoid performing excessive amounts of lower extremity work.     

Emotion and culture: A meta-analysis

A meta-analysis of 190 cross-cultural emotion studies, published between 1967 and 2000, was performed to examine (1) to what extent reported cross-cultural differences in emotion variables could be regarded as valid (substantive factors) or as method-related (statistical artefacts, cultural bias), and (2) which country characteristics could explain valid cross-cultural differences in emotion. The relative contribution of substantive and method-related factors at sample, study, and country level was investigated and country-level explanations for differences in emotions were tested. Results indicate that a correction for statistical artefacts and method-related factors reduced the observed cross-cultural effect sizes considerably. After controlling for valence (positive vs. negative emotions) and kind of study (self-report vs. recognition studies), the remaining cross-cultural variance was associated with subsistence mode, political system, values, and religiosity. Values explained more variance than did ecological or sociopolitical variables. It was concluded that both method-related factors (13.8% of variance explained) and culture-level factors (27.9% of variance explained) underlie observed cross-cultural differences.     

A comparison of maladaptive behaviors of athletes and nonathletes.

A comparison of maladaptive behavior tendencies of men and women who were athletes and nonathletes was undertaken. Participating students (N = 200) were divided into four groups: male athletes, male nonathletes, female athletes, and female nonathletes. Maladaptive behavior tendencies were determined from responses on C. MacAndrew's (1965) Alcoholism Scale. The statistical analysis used was an independent groups 2 x 2 analysis of variance to determine significant main effects and interaction effects. The mean maladaptive behavior score (MBS) for athletes (M = 21.87) was significantly higher (p < .05) than the MBS for nonathletes (M = 20.24). The MBS for the men (M = 21.68) was significantly higher (p < .05) than the MBS for the women (M = 20.43). No significant interaction (p > .05) between gender and athletic status was found. Male athletes are more likely than the other 3 groups to have maladaptive behavior tendencies. Research directed toward greater understanding and the development of preventive and coping techniques for this population is needed.     

Personality Inventory Responses of Males With Medically Intractable Seizures

The psychosocial adjustment of 50 male patients to intractable seizures was assessed by comparing their responses to a combined version of the Minnesota Multiphasic Personality inventory (MMPI) and the California Psychological inventory (CPI) to the responses of 50 medical, psychiatric, or nonclinical controls who denied seizures. The two groups were significantly different (p < .01) on one MMPI and 10 CPI scales. Significant (p < .01) between-group differences were also rejected in 29 of the 704 personality inventory items. Those items were rationally clustered according to content into six conceptually, identifiable subscales; 30 additional items with similar content that were significant at the .05 level were added to those subscales. Comparison of subscale scores of an additional 30 seizure and 30 nonseizure subjects using analysis of variance revealed F values that reached statistical significance (p < .05) in four cases and approached significance (p = .07) in another. Applying coefficients derived from discriminant analysis of the first samples correctly classified 99% of the original patients, and 85% of the validation subjects. Results reveal a logical, understandable, and largely adaptive response to intractable seizures and offer little support for the concept of a dysfunctional or pathological interictal personality style.     

Level of Perceptual Development as Reflected in Responses to the Holtzman Inkblot Technique

To investigate the relationships between chronological age and scores on 10 variables from the Holtzman Inkblot Technique, 586 normal Ss comprising five criterion age-groups ranging from 5.2 to 19.5 years were tested. Each group had an equal number of males and females. Following a statistical correction for number of rejections, a sex-by-age analysis of variance revealed no significant sex differences or sex-by-age interactions. However, significant age-group differences were found for all 10 variables, six of them resulting in steadily increasing means across the five groups. These age trends are consistent with the sequence of perceptual change outlined by developmental theory, and are interpreted as indicating a developmental shift in the dominance of perceptual functions.     

Personality inventory responses of males with medically intractable seizures.

The psychosocial adjustment of 50 male patients to intractable seizures was assessed by comparing their responses to a combined version of the Minnesota Multiphasic Personality Inventory (MMPI) and the California Psychological Inventory (CPI) to the responses of 50 medical, psychiatric, or nonclinical controls who denied seizures. The two groups were significantly different (p < .01) on one MMPI and 10 CPI scales. Significant (p < .01) between-group differences were also reflected in 29 of the 704 personality inventory items. Those items were rationally clustered according to content into six conceptually identifiable subscales; 30 additional items with similar content that were significant at the .05 level were added to those subscales. Comparison of subscale scores of an additional 30 seizure and 30 nonseizure subjects using analysis of variance revealed F values that reached statistical significance (p < .05) in four cases and approached significance (p = .07) in another. Applying coefficients derived from discriminant analysis of the first samples correctly classified 99% of the original patients and 85% of the validation subjects. Results reveal a logical, understandable, and largely adaptive response to intractable seizures and offer little support for the concept of a dysfunctional or pathological interictal personality style.     

New Methods for Comparing Groups

Abstract— A commonly used method for comparing groups of individuals is the analysis of variance (ANOVA) F test. When the assumptions underlying the derivation of this test are true, its power, meaning its probability of detecting true differences among the groups, competes well with all other methods that might be used. But when these assumptions are false, its power can be relatively low. Many new statistical methods have been proposed—ones that are aimed at achieving about the same amount of power when the assumptions of the F test are true but which have the potential of high power in situations where the F test performs poorly. A brief summary of some relevant issues and recent developments is provided. Some related issues are discussed and implications for future research are described.     

Body image of obese adolescent girls in a high school and clinical setting

There have been many investigations of body image in cases of anorexia nervosa in adolescent females. However, there has been limited research with normal adolescent girls who happen to be overweight. In this study, 27 girls found to be obese (at least 20 percent above average body weight for age, sex and height) among 203 girls in home economics classes of four suburban high schools were compared to 20 girls seeking help for their obesity on an outpatient basis in hospital-affiliated programs for weight reduction in a nearby city. The two groups were assessed and compared regarding body image factors. The clinical subjects showed a significantly greater difference in their selection of outline drawings of the female figure perceived to represent their actual as compared to ideal body sizes. No significant difference was found in articulation of body concept as revealed in human figure drawings judged according to Witkin's Articulation of Body Concept (ABC) Scale, and in the mean number of uncomplimentary adjectives used to describe present appearance. However, the difference between the mean scores of the two groups in the latter two variables, when submitted to t tests, were shown to approach significance (.017 less than p less than .05).     

Worry in Adults and Children: Developmental Differences in the Importance of Probability and Cost Judgments

This study investigated developmental differences in the relationship of probability and cost estimates to worrying. Adults, younger children (M age = 8.67 years) and older children (M age = 11.06 years) rated the extent to which they worry about a list of negative social and physical outcomes and provided subjective probability and cost estimates for the same outcomes. Adults reported worrying more about social outcomes and rated them as less ‘bad’ (or costly) but more likely to occur than physical outcomes. Unlike adults, children in both age groups reported worrying more about physical outcomes. However, similar to adults, they also rated social outcomes as less ‘bad’ but more likely to occur than physical outcomes. Regression analyses showed that probability ratings were the best predictors of worry in adults, both probability and cost ratings equally predicted worry in older children, but only cost ratings predicted worry in younger children.

Short-term memory capacity: magic number or magic spell?

Previous experiments have found that memory span is greater for items that can be pronounced more quickly. For a variety of materials the span equals the number of items that can be pronounced in about 1.5 s, presumably the duration of the verbal trace. This suggests a model for immediate recall: The probability of correctly recalling a list equals the probability that the time to recite the list is less than the variable duration of the trace. Recall probability for lists of various lengths was determined for six materials. Later, subjects read the lists aloud. The standard normal deviates corresponding to probability of correct recall were linear in pronunciation time. Evidently, over subjects, a normal distribution is a reasonable approximation of the distribution of the trace duration. The mean and variance of the trace duration were estimated. The mean (1.88 s) agrees well with previous estimates, and the model accounts for 95% of the variance in immediate recall.     

Early Versus Late Onset of Spinal Cord Injury Among the Elderly

The objective of this study is to compare elderly individuals with late (60 years old) versus early (<60 years old) onset spinal cord injury (SCI) across quality of life (QOL) domains for which cross-sectional design was used. The outcome measures selected were secondary medical complications (e.g., pneumonia, autonomic dysreflexia, number of days hospitalized), Functional Independence Measure (FIM), Satisfaction With Life Scale (SWLS), and the Craig Handicap Assessment and Reporting Technique (CHART). Analyses between groups showed that individuals with SCI onset 60 years of age or older were significantly older, had a greater proportion of incomplete lesions, were more likely to have SCI resulting from medical complication, and were less likely to be working. After controlling for differences in demographic and lesion characteristics, the majority of QOL domains were similar between groups. However, overall self-reported handicap (CHART-total score) was significantly greater among elderly with late onset SCI, particularly in the areas of physical independence and social integration. Differences in QOL between elderly with late versus early onset SCI were most prominent in the area of physical independence and social integration. The importance of appropriate statistical control, theoretical implications, and future directions are discussed.     

Magnitude of relationship between burnout and absenteeism: a preliminary study

This study examined the influence of guilt related to a negative attitude toward patients and its relation with burnout and absenteeism. The sample consisted of 717 nursing professionals. Depersonalization was evaluated by the Maslach Burnout Inventory and Guilt was evaluated by one item. To estimate Absenteeism, participants were asked about the number of workdays they had missed in the past year. Hierarchical multiple regression analyses make it possible to conclude that guilt explains work absenteeism, and the interaction between depersonalization and guilt (Incr. R2 = .008, p < .05) indicates significant differences in the number of work days missed in the last year. Conclusions are limited, as these effects are quite weak: all variables together only explain about 4% of the shared variance in absenteeism. Researchers might assess whether feelings of guilt help explain the relationship between burnout and symptoms such as absenteeism.     

Intersubject variability in perceived oscillation as a function of threshold for binocular disparity.

Four groups of 15 female subjects each were classified along a quantitative dimension for proportion of perceived oscillation and threshold for binocular disparity. Analyses of variance showed that significant differences in proportion of perceived oscillation were accompanied by significant differences in threshold for binocular disparity for perceivers of "high" and "low" oscillation (p less than .05). For perceivers of "intermediate" oscillation significant differences in proportion of perceived oscillation but not threshold for binocular disparity were found. It was suggested that: (1) intersubject variability in perceived oscillation may be governed by the threshold for binocular disparity, (2) "low" perceivers may be especially sensitive to the magnitude of the cue, (3) "intermediate" perceivers' subjective reports may be primarily dependent on response criteria and the multiplicity of subjective factors which constitute it, (4) "high" perceivers apparently have least response sensitivity and they cannot maintain a consistent response criterion.