A reexamination of black-white mean differences in work performance: more data, more moderators

This study is the largest meta-analysis to date of Black-White mean differences in work performance. The authors examined several moderators not addressed in previous research. Findings indicate that mean racial differences in performance favor Whites (d = 0.27). Effect sizes were most strongly moderated by criterion type and the cognitive loading of criteria, whereas data source and measurement level were influential moderators to a lesser extent. Greater mean differences were found for highly cognitively loaded criteria, data reported in unpublished sources, and for performance measures consisting of multiple item scales. On the basis of these findings, the authors hypothesize several potential determinants of mean racial differences in job performance.     

A meta-analytic review of gender differences in perceptions of sexual harassment

Research on gender differences in perceptions of sexual harassment informs an ongoing legal debate regarding the use of a reasonable person standard instead of a reasonable woman standard to evaluate sexual harassment claims. The authors report a meta-analysis of 62 studies of gender differences in harassment perceptions. An earlier quantitative review combined all types of social-sexual behaviors for a single meta-analysis; the purpose of this study was to investigate whether the magnitude of the female-male difference varies by type of behavior. An overall standardized mean difference of 0.30 was found, suggesting that women perceive a broader range of social-sexual behaviors as harassing. However, the meta-analysis also found that the female-male difference was larger for behaviors that involve hostile work environment harassment, derogatory attitudes toward women, dating pressure, or physical sexual contact than sexual propositions or sexual coercion.     

Developing Clear and Useful Scoring Procedures for Goal Attainment Scaling in Longitudinal Case Studies

This article examines scoring procedures for Goal Attainment Scaling (GAS) and suggests a new scoring procedure to communicate GAS outcomes in longitudinal studies. Analysis is based on longitudinal case studies that evaluate transitioning from psychiatric institutions to independent living. Five scoring methods are identified: raw scores; sum of score differences; mean scores; T-scores; and graphing raw scores. Graphing raw scores is presented as an alternative method of expressing GAS scores for studies with few participants and multiple time points. This enables easy interpretation of goal attainment, and allows for fluctuation in performance as is typical in clinical practice.     

Fixed Effects vs. Random Effects Meta-Analysis Models: Implications for Cumulative Research Knowledge

Research conclusions in the social sciences are increasingly based on meta-analysis, making questions of the accuracy of meta-analysis critical to the integrity of the base of cumulative knowledge. Both fixed effects (FE) and random effects (RE) meta-analysis models have been used widely in published meta-analyses. This article shows that FE models typically manifest a substantial Type I bias in significance tests for mean effect sizes and for moderator variables (interactions), while RE models do not. Likewise, FE models, but not RE models, yield confidence intervals for mean effect sizes that are narrower than their nominal width, thereby overstating the degree of precision in meta-analysis findings. This article demonstrates analytically that these biases in FE procedures are large enough to create serious distortions in conclusions about cumulative knowledge in the research literature. We therefore recommend that RE methods routinely be employed in meta-analysis in preference to FE methods.     

Is the meta-analysis of correlation coefficients accurate when population correlations vary?

One conceptualization of meta-analysis is that studies within the meta-analysis are sampled from populations with mean effect sizes that vary (random-effects models). The consequences of not applying such models and the comparison of different methods have been hotly debated. A Monte Carlo study compared the efficacy of Hedges and Vevea's random-effects methods of meta-analysis with Hunter and Schmidt's, over a wide range of conditions, as the variability in population correlations increases. (a) The Hunter-Schmidt method produced estimates of the average correlation with the least error, although estimates from both methods were very accurate; (b) confidence intervals from Hunter and Schmidt's method were always slightly too narrow but became more accurate than those from Hedges and Vevea's method as the number of studies included in the meta-analysis, the size of the true correlation, and the variability of correlations increased; and (c) the study weights did not explain the differences between the methods.     

THE RELATIONSHIP BETWEEN SUPERVISORY RATINGS AND RESULTS-ORIENTED MEASURES OF PERFORMANCE: A META-ANALYSIS

A meta-analysis was conducted with 23 studies to assess the relationship between supervisory ratings and results-oriented measures of performance. As hypothesized, the corrected mean correlation was higher when a relative (versus absolute) rating format was used and when a composite (versus overall) rating method was used. These differences did not, however, account for all of the remaining variance around the relationship between ratings and results. Suggestions are offered for the direction of future research and practice.     

Sex differences in visual-spatial working memory: A meta-analysis

Visual-spatial working memory measures are widely used in clinical and experimental settings. Furthermore, it has been argued that the male advantage in spatial abilities can be explained by a sex difference in visual-spatial working memory. Therefore, sex differences in visual-spatial working memory have important implication for research, theory, and practice, but they have yet to be quantified. The present meta-analysis quantified the magnitude of sex differences in visual-spatial working memory and examined variables that might moderate them. The analysis used a set of 180 effect sizes from healthy males and females drawn from 98 samples ranging in mean age from 3 to 86 years. Multilevel meta-analysis was used on the overall data set to account for non-independent effect sizes. The data also were analyzed in separate task subgroups by means of multilevel and mixed-effects models. Results showed a small but significant male advantage (mean d = 0.155, 95 % confidence interval = 0.087-0.223). All the tasks produced a male advantage, except for memory for location, where a female advantage emerged. Age of the participants was a significant moderator, indicating that sex differences in visual-spatial working memory appeared first in the 13-17 years age group. Removing memory for location tasks from the sample affected the pattern of significant moderators. The present results indicate a male advantage in visual-spatial working memory, although age and specific task modulate the magnitude and direction of the effects. Implications for clinical applications, cognitive model building, and experimental research are discussed.     

A meta-analytic review of gender differences in ADHD

The present study examined gender differences in ADHD through a meta-analysis. Effect size estimates for the primary symptoms and correlates of ADHD were calculated in an attempt to replicate and extend a previous meta-analysis on gender differences in the disorder. Relatively lenient inclusion criteria were used in order to maximize the number of studies included in the effect sizes. The results indicated that in comparison to ADHD boys, ADHD girls had lower ratings on hyperactivity, inattention, impulsivity, and externalizing problems. In addition, ADHD girls had greater intellectual impairments and more internalizing problems than ADHD boys. Overall, the results of the current meta-analysis indicated general agreement with the previous meta-analysis. The clinical implications of these gender differences and future research considerations are discussed.     

Age-related differences in performance on the Wisconsin card sorting test: a meta-analytic review

Two meta-analyses investigating age-related differences in performance on a popular measure of executive function, the Wisconsin Card Sorting Test (WCST), are reported. The 1st meta-analysis examined age-related changes in performance for the number of categories achieved, and the 2nd meta-analysis examined performance for the number of perseverative errors committed. Results indicated that robust age differences were present on both measures. Further analysis of moderator variables revealed reliable effects of education and test version on both measures, whereas test modality led to marginally significant differences in effect sizes obtained only for the number of categories achieved. Findings are discussed along with current accounts of age differences in performance of the WCST.     

On the Multilevel Nature of Meta-Analysis: A Tutorial,Comparison of Software Programs,and Discussion of Analytic Choices

The term “multilevel meta-analysis” is encountered not only in applied research studies, but in multilevel resources comparing traditional meta-analysis to multilevel meta-analysis. In this tutorial, we argue that the term “multilevel meta-analysis” is redundant since all meta-analysis can be formulated as a special kind of multilevel model. To clarify the multilevel nature of meta-analysis the four standard meta-analytic models are presented using multilevel equations and fit to an example data set using four software programs: two specific to meta-analysis (metafor in R and SPSS macros) and two specific to multilevel modeling (PROC MIXED in SAS and HLM). The same parameter estimates are obtained across programs underscoring that all meta-analyses are multilevel in nature. Despite the equivalent results, not all software programs are alike and differences are noted in the output provided and estimators available. This tutorial also recasts distinctions made in the literature between traditional and multilevel meta-analysis as differences between meta-analytic choices, not between meta-analytic models, and provides guidance to inform choices in estimators, significance tests, moderator analyses, and modeling sequence. The extent to which the software programs allow flexibility with respect to these decisions is noted, with metafor emerging as the most favorable program reviewed.     

2004至2016年中国大学生身体自尊的变迁:一项横断历史研究

大学生身体自尊是如何随年代变化的呢？研究对2004年至2016年107篇使用身体自尊量表的研究进行了横断历史研究,涉及大学生30738名。结果显示：（1）我国大学生身体自我价值感、运动能力和身体素质与年代有着显著正相关关系;（2）男女大学生身体自尊水平都有了一定提升;（3）男生在身体自尊各个因子的得分高于女生0.2~0.3个标准差,显示出了大学生在身体自尊水平上的性别差异;（4）从大学生整个群体来看,经济因素、高等教育因素与大学生身体自尊具有显著相关,而从不同性别群体来看,经济因素与人口压力对于男大学生的身体自尊影响更为明显。     

Meta-analysis of work–family conflict mean differences: Does national context matter?

Work–family conflict continues to be a topic of considerable interest to researchers and practitioners across the globe. In the current study, meta-analysis is used to compare cross-national mean differences in work-to-family conflict and family-to-work conflict across cultural, institutional, and economic aspects of context. No significant differences in work-to-family conflict were detected. Reports of family-to-work conflict were higher in more collectivistic versus more individualistic cultures, in countries with a higher versus a lower economic gender gap, and in countries other than the U.S. versus the U.S.     

The effectiveness of school-based anger interventions and programs: A meta-analysis

Twenty peer-reviewed journal articles that described outcomes of interventions that took place in school settings and either focused on anger or included anger as a dependent variable were meta-analyzed. No differences in outcomes were found for group comparisons by school setting, special education status, entrance criteria, or treatment agents. Although 60% of articles discussed its importance, only two articles actually measured treatment integrity. Across outcomes, the weighted mean effect size of the interventions post treatment was determined to be .31. The largest effects were found for anger and externalizing behaviors, internalizing, and social skills, with mean effect sizes of .54, .43, and .34 respectively. Weighted mean effect sizes for follow-up studies were also calculated, but given the small number of studies that reported follow-up effects, those must be interpreted with caution. The results of this meta-analysis are discussed as they relate to research, practice, and intervention with children.     

Developmental change in the relation between simple and complex spans: a meta-analysis

In the present meta-analysis the effects of developmental level on the correlation between simple and complex span tasks were investigated. Simple span-complex span correlation coefficients presented in 52 independent samples (7,060 participants) were regressed on a variable representing mean age of sample (range: 4.96-22.80 years), using analyses adapted for meta-analytic purposes. The results showed strong positive relations between mean age of sample and simple span-complex span correlation coefficients, suggesting that the relation between simple and complex span tasks became stronger with increasing age. These results could not be accounted for by study-related differences in measurement reliability, restricted range, or sample size. A new theoretical account explaining these findings is outlined.     

A Bayesian Power Analysis Procedure Considering Uncertainty in Effect Size Estimates from a Meta-analysis

In conventional frequentist power analysis, one often uses an effect size estimate, treats it as if it were the true value, and ignores uncertainty in the effect size estimate for the analysis. The resulting sample sizes can vary dramatically depending on the chosen effect size value. To resolve the problem, we propose a hybrid Bayesian power analysis procedure that models uncertainty in the effect size estimates from a meta-analysis. We use observed effect sizes and prior distributions to obtain the posterior distribution of the effect size and model parameters. Then, we simulate effect sizes from the obtained posterior distribution. For each simulated effect size, we obtain a power value. With an estimated power distribution for a given sample size, we can estimate the probability of reaching a power level or higher and the expected power. With a range of planned sample sizes, we can generate a power assurance curve. Both the conventional frequentist and our Bayesian procedures were applied to conduct prospective power analyses for two meta-analysis examples (testing standardized mean differences in example 1 and Pearson's correlations in example 2). The advantages of our proposed procedure are demonstrated and discussed.     

Contextual interference: a meta-analytic study

A meta-analysis of the contextual interference effect produced 139 estimates of effect sizes from 61 studies. The average overall effect size was .38. The effect size for basic research (.57) was significantly different from applied research (.19). Significant differences were also obtained between the effect sizes for adults (.50) and those for younger learners (.10). Power for retention and transfer scores was not significantly different. The overall mean power of the studies reviewed was .43.     

Traditional vs. Alternative Teacher Preparation Programs: A Meta-Analysis

A meta-analysis is presented of the academic achievement effects on students taught by teachers from alternative teacher preparation (ATP) programs, compared to students taught by teachers from traditional teacher preparation (TTP) programs. The literature has indicated mixed results on the student-level academic outcomes of ATP programs. Findings from this meta-analysis indicate an overall statistically significant, yet small, difference in ATP and TTP programs (g?=?0.03; 95% CI?=?0.01, 0.04, p?=?0.001), with the mean achievement of students who had ATP teachers was about 0.03 standard deviations above that of students who had TTP teachers. Further, there were differences in student achievement by type of ATP program, school level, and academic subject area. These results, as well as implications for policy and practice are discussed.     

Use of the estimated intraclass correlation for correcting differences in effect size by level

In a meta-analysis of intervention or group comparison studies, researchers often encounter the circumstance in which the standardized mean differences (d-effect sizes) are computed at multiple levels (e.g., individual vs. cluster). Cluster-level d-effect sizes may be inflated and, thus, may need to be corrected using the intraclass correlation (ICC) before being combined with individual-level d-effect sizes. The ICC value, however, is seldom reported in primary studies and, thus, may need to be computed from other sources. This article proposes a method for estimating the ICC value from the reported standard deviations within a particular meta-analysis (i.e., estimated ICC) when an appropriate default ICC value (Hedges, 2009b) is unavailable. A series of simulations provided evidence that the proposed method yields an accurate and precise estimated ICC value, which can then be used for correct estimation of a d-effect size. The effects of other pertinent factors (e.g., number of studies) were also examined, followed by discussion of related limitations and future research in this area.     

On reconciling conflicting meta-analytic findings regarding integrity test validity

We react to the Van Iddekinge, Roth, Raymark, and Odle-Dusseau (2012a) meta-analysis of the relationship between integrity test scores and work-related criteria, the earlier Ones, Viswesvaran, and Schmidt (1993) meta-analysis of those relationships, the Harris et al. (2012) and Ones, Viswesvaran, and Schmidt (2012) responses, and the Van Iddekinge, Roth, Raymark, and Odle-Dusseau (2012b) rebuttal. We highlight differences between the findings of the 2 meta-analyses by focusing on studies that used predictive designs, applicant samples, and non-self-report criteria. We conclude that study exclusion criteria, correction for artifacts, and second order sampling error are not likely explanations for the differences in findings. The lack of detailed documentation of all effect size estimates used in either meta-analysis makes it impossible to ascertain the bases for the differences in findings. We call for increased detail in meta-analytic reporting and for better information sharing among the parties producing and meta-analytically integrating validity evidence.