A Meta-Analysis of Research on Hindsight Bias

We conducted a meta-analysis of research on hindsight bias to gain an up-to-date summary of the overall strength of hindsight effects and to test hypotheses about potential moderators of hindsight distortion. A total of 95 studies (83 published and 12 unpublished) were included, and 252 independent effect sizes were coded for moderator variables in 3 broad categories involving characteristics of the study, of measurement, and of the experimental manipulation. When excluding missing effect sizes, the overall mean effect size was Md = .39 with a 95% confidence interval of .36 to .42. Five main findings emerged: (a) effect sizes calculated from objective probability estimates were larger than effect sizes calculated from subjective probability estimates; (b) effect sizes of studies that used almanac questions were larger than effect sizes of studies that used real-world events or case histories; (c) studies that included neutral outcomes resulted in larger effect sizes than studies that used positive or negative outcomes; (d) studies that included manipulations to increase hindsight bias resulted in significantly larger effect sizes than studies in which there were no manipulations to reduce or increase hindsight bias; and (e) studies that included manipulations to reduce hindsight bias did not produce lower effect sizes. These findings contribute to our understanding of hindsight bias by updating the state of knowledge, widening the range of known moderator variables, identifying factors that may activate different mediating processes, and highlighting critical gaps in the research literature.     

Meta-analyses of ethnic match as a predictor of dropout,utilization, and level of functioning

Meta-analyses were performed on 7 studies of ethnic match and psychotherapy. From 13 comparisons of ethnic match and dropout, a small overall effect size was found (r = .03). From 16 comparisons of ethnic match and number of sessions attended, a small overall effect size was found (r = .04). Effect sizes for dropout and utilization were heterogeneous, with ethnic minority groups having larger effect sizes than Caucasian Americans. From 9 comparisons of ethnic match and termination Global Assessment Score, a negligible overall effect size was found (r = .01). The small dropout and utilization effect sizes indicate that ethnic match is not a significant clinical predictor of decreasing dropout after the first session or increasing number of sessions attended.     

Ingroup role models and underrepresented students’ performance and interest in STEM: A meta-analysis of lab and field studies

This meta-analysis synthesizes research on using ingroup role models to improve the performance and interest of underrepresented students in science, technology, engineering, and math (STEM). A systematic literature search resulted in forty-five studies that met the selection criteria, including the presence of a comparison group. Both lab and field studies suffered from small sample bias, with smaller sample sizes predicting larger effect sizes among lab studies, but smaller effect sizes among field studies. Correcting for small sample bias, ingroup role models had a small, but significant positive overall effect (d?=?0.20) among field studies and a non-significant overall effect (d?=?0.04) among lab studies. The only significant moderator was level of interaction, with in-person role models having smaller effects among lab studies (p?=?.008). Implications for interventions to increase the representation of female and underrepresented minority students in STEM and future directions for research are discussed.

     

PERSONALITY MEASURES AS PREDICTORS OF JOB PERFORMANCE: A META-ANALYTIC REVIEW

The purpose of this study was to investigate conflicting findings in previous research on personality and job performance. Meta-analysis was used to (a) assess the overall validity of personality measures as predictors of job performance, (b) investigate the moderating effects of several study characteristics on personality scale validity, and (c) appraise the predictability of job performance as a function of eight distinct categories of personality content, including the "Big Five" personality factors. Based on review of 494 studies, usable results were identified for 97 independent samples (total N = 13,521). Consistent with predictions, studies using confirmatory research strategies produced a corrected mean personality scale validity (.29) that was more than twice as high as that based on studies adopting exploratory strategies (.12). An even higher mean validity (.38) was obtained based on studies using job analysis explicitly in the selection of personality measures. Validities were also found to be higher in longer tenured samples and in published articles versus dissertations. Corrected mean validities for the "Big Five" factors ranged from .16 for Extroversion to .33 for Agreeableness. Weaknesses in the reporting of validation study characteristics are noted, and recommendations for future research in this area are provided. Contrary to conclusions of certain past reviews, the present findings provide some grounds for optimism concerning the use of personality measures in employee selection.     

Examining psychokinesis: the interaction of human intention with random number generators--a meta-analysis

Séance-room and other large-scale psychokinetic phenomena have fascinated humankind for decades. Experimental research has reduced these phenomena to attempts to influence (a) the fall of dice and, later, (b) the output of random number generators (RNGs). The meta-analysis combined 380 studies that assessed whether RNG output correlated with human intention and found a significant but very small overall effect size. The study effect sizes were strongly and inversely related to sample size and were extremely heterogeneous. A Monte Carlo simulation revealed that the small effect size, the relation between sample size and effect size, and the extreme effect size heterogeneity found could in principle be a result of publication bias.     

The Effect Size of Variables Associated With Creativity: A Meta-Analysis

The main purpose of this study was to use meta-analysis to investigate the mean effect size of relevant variables associated with creative person, process, product, and environment. Altogether, 2,013 effect sizes from 111 studies were analyzed. The unweighted grand mean effect size of the 111 studies was 0.69, with a standard deviation (SD) of 0.63. Such result was significantly different from 0 at t (110) = 11.52, p < .01. When the averaged effect size of each study was weighted with the sample size of that study, the weighted grand mean effect size was 0.72. The important findings were: (a) the mean effect sizes associated with problem-solving creativity and verbal creativity were significantly larger than those associated with emotional creativity and nonverbal creativity, (b) variables having a large mean effect size were prestige of honors/awards, working circumstances favorable for creativity, defining problem, and retrieving knowledge, (c) most of the mean effect sizes of the problem solving procedures on the measures of problem solving exceeded the medium (0.5) of Cohen's guidelines. Areas to be further explored are suggested.     

A Meta‐Analytic Review of Child‐Centered Play Therapy Approaches

The authors explored the overall effectiveness of child‐centered play therapy (CCPT) approaches through a meta‐analytic review of 52 controlled outcome studies between 1995 and 2010. Hierarchical linear modeling techniques estimated a statistically significant moderate treatment effect size (.47) for CCPT, as well as statistically significant relationships between effect size and study characteristics, including child's age, child's ethnicity, caregiver involvement, treatment integrity, publication status, and presenting issue.     

Social cognitive theory and cancer patients' quality of life: a meta-analysis of psychosocial intervention components.

Social cognitive theory (SC) provides a theoretical framework to evaluate improved quality-of-life (QOL) outcomes through interventions with cancer patients To assess whether inclusion of SCT components predicted better outcomes, focused comparisons were used to integrate results from 38 randomized studies. Interventions with more SCT components had significantly larger effect sizes than studies with fewer or no SCT components for the overall analysis (Z = 3.72, p < .01). Subanalyses of affective, social, objective physical outcome, and specific QOL measures revealed that SCT-based interventions had significantly higher effect sizes; inclusion of SCT components resulted in significantly lower effect sizes on subjective physical and functional outcomes. Results suggest that using SCT-based interventions maximizes improvement in overall QOL outcomes for adult cancer patients.     

Functional Disability in Mild Cognitive Impairment: A Systematic Review and Meta-Analysis

Accumulating evidence suggests that the pre-dementia syndrome mild cognitive impairment (MCI) is characterized by decrements in instrumental activities of daily living (IADL). The current review was a quantitative synthesis of the available literature to objectively characterize IADL disability in MCI while clarifying inconsistencies in findings across studies. It was hypothesized that individuals with MCI would display significantly greater functional impairment relative to cognitively intact controls. Candidate moderators specified a priori included functional assessment approach, MCI subtype, depressive symptoms, and language conducted. Online databases (PubMed/MEDLINE and PsycINFO) and reference lists were searched to identify peer-reviewed publications assessing IADL in MCI compared to normal aging. A total of 151 effect sizes derived from 106 studies met inclusionary criteria (N?=?62,260). Random effects models yielded a large overall summary effect size (Hedges’ g?=?0.76, 95 % confidence interval: 0.68???0.83, p?<?.001) confirmed in multi-level analyses adjusted for nesting of effect sizes within studies (g?=?0.78, 95 % confidence interval: 0.69???0.87). Functional assessment strategy and MCI subtype were significant moderators of effect size, whereas depressive symptoms and language were not. Results convincingly demonstrate that MCI is associated with significant difficulties in the performance of complex everyday tasks. It appears that functional decline, like cognitive decline, exists on a continuum from healthy aging to dementia onset. Implications for clinical practice and research priorities are discussed.     

Borderline Personality Disorder and childhood sexual abuse: a meta-analytic study.

A meta-analysis of the published literature was performed to evaluate the common effect size (r) of the association between childhood sexual abuse (CSA) and Borderline Personality Disorder (BPD). An online search from 1980 to 1995 was used to gather published papers. r coefficient was used as an effect size measure, and 21 studies were retrieved. A moderate pooled r (.279) for the association between CSA and BPD was observed. Pooled effect size estimates were also obtained for CSA parameters. No significant effect of moderators was evidenced. The results of this study did not support the hypothesis that CSA is a major psychological risk factor or a causal antecedent of BPD.     

Conditional effects of interventions in note-taking procedures on learning: A meta-analysis

Abstract:  A meta-analysis was conducted on 17 studies (27 separate samples and 73 effect sizes) examining the effects of interventions in the note-taking procedures on learning from lectures or texts. The overall mean weighted effect size ( d ) for the intervention groups versus the no intervention groups was 0.02. To explore what factors influence the intervention effects, moderator analyses were conducted separately for intervention type, schooling level, presentation method, publication source, and publication year. The results indicated that students at a lower schooling level gained some benefits from the interventions, whereas students at a higher schooling level did not. Also, the intervention effects were greater for journal articles than for doctoral dissertations and Education Resources Information Center reports. Other variables did not significantly account for the variance in effect sizes.     

Quantifying the effects of the differential outcomes procedure in humans: A systematic review and a meta‐analysis

We present a systematic review and a meta‐analysis comparing the differential outcomes procedure to a nondifferential outcomes procedure among clinical and nonclinical populations. Sixty distinct experiments were included in the systematic review, 43 of which were included in the meta‐analysis. We calculated pooled effect sizes for accuracy (overall accuracy, test accuracy, transfer accuracy) and acquisition outcomes (latency, errors, and trials to mastery). The meta‐analysis revealed significant medium‐to‐large effect sizes for all three accuracy measures (pooled effect size range, 0.57 to 1.30). We found relatively greater effect sizes among clinical populations (effect size = 1.04). The single‐subject experimental literature included in the systematic review was consistent with the findings from the group studies, demonstrating improvements in accuracy and speed of learning for the majority of participants. Moderator and subgroup analyses suggest that discrimination difficulty may induce relatively larger differential outcomes effects. The results indicate that the differential outcomes procedure can be a valuable addition to reinforcement‐based interventions.     

The Effectiveness of Solution-Focused Group Therapy in Ethnic Chinese School Settings: A Meta-Analysis

In order to assess the effectiveness and the moderators of the effects of solution-focused group therapy (SFGT) in ethnic Chinese school settings, we used a meta-analytic method to synthesize 24 studies from Taiwan and China. The results showed that the overall effect sizes of SFGT’s immediate and follow-up effects were 1.03 and 1.09, respectively, and were positive and large in ethnic Chinese school settings. The overall SFGT immediate effect size was large for all school levels, except for junior high schools (d = .61). The overall SFGT immediate effect size for internalizing behavior problems was 1.06, and for family and relationship problems it was .94. This study offers significant evidence pointing to SFGT’s high degree of effectiveness in ethnic Chinese school settings. Implications for the use of SFGT in school settings and Chinese culture are discussed.     

Are negative affective states associated with HIV sexual risk behaviors? A meta-analytic review.

This meta-analytic review examined whether negative affective states (depressive symptomatology, anxiety, anger) are associated with sexual behaviors that place people at risk for contracting or transmitting HIV. The results from 34 study samples were included in the analysis. Contrary to popular belief, the findings as a whole provide little evidence that negative affect is associated with increased sexual risk behavior. The average weighted correlation for the overall association was .05. The effect size was nonsignificantly higher for anger (r = .10) than for depressive symptoms (r = .04) or anxiety (r = .03). The variability of effect sizes was not accounted for by type of sexual risk measure, subject population, or methodological aspects of the studies. Conceptual and methodological limitations of the literature are identified and directions for future research are discussed.     

心理学常用效应量的选用与分析

效应量在量化方面弥补了零假设检验的不足。除了报告检验结果外, 许多期刊还要求在研究报告中包括效应量。效应量可以分为三大类别：差异类、相关类和组重叠类, 它们在不同的研究设计(如单因素和多因素被试间、被试内和混合实验设计)或在不同的数据条件下(如小样本、方差异质等)可能有不同的计算方法和用法, 但许多效应量可以相互转换。我们梳理出一个表格有助应用工作者根据研究目的和研究类型选用合适的效应量。     

Single-case effect size calculation: Comparing regression and non-parametric approaches across previously published reading intervention data sets

Growing from demands for accountability and research-based practice in the field of education, there is recent focus on developing standards for the implementation and analysis of single-case designs. Effect size methods for single-case designs provide a useful way to discuss treatment magnitude in the context of individual intervention. Although a standard effect size methodology does not yet exist within single-case research, panel experts recently recommended pairing regression and non-parametric approaches when analyzing effect size data. This study compared two single-case effect size methods: the regression-based, Allison-MT method and the newer, non-parametric, Tau-U method. Using previously published research that measured the Words read Correct per Minute (WCPM) variable, these two methods were examined by comparing differences in overall effect size scores and rankings of intervention effect. Results indicated that the regression method produced significantly larger effect sizes than the non-parametric method, but the rankings of the effect size scores had a strong, positive relation. Implications of these findings for research and practice are discussed.     

Construct validity of perceptual style: Role of stimulus size in the embedded-figures test and the rod-and-frame test

Three experiments were performed in order to test the construct validity of perceptual field dependence (FD) and field independence (FI). In Experiment 1, performance of subjects on two sizes of Form A of Witkin’s Embedded Figures Test (EFT) with a size-ratio of 1 to 4 was compared. The size of the standard EFT was taken as unity. A nonsignificant size effect was found for all subjects as a group, and a significant rank correlation was found between performances on the two sizes of the EFT. Furthermore, the performance of field-dependent subjects (as defined by performance on the standard EFT) and field-independent subjects did not interact significantly with the size factor. In Experiment 2, the physical size of the EFT changed from 1 to 8. A nonsignificant, overall size effect was again found, but the EFT performance of fielddependent and field-independent subjects interacted significantly with the size factor. In Experiment 3, the performance of subjects on two sizes of the EFT (i.e., 1 to 8) and two sizes of the standard rod-and-frame test (RFT) (i.e., 1 to 4.5) were compared. A significant stimulus-size effect was found in the EFT, with all subjects becoming morefield-independent, and a significant stimulus-size effect was found in the RFT with the same subjects becoming morefield-dependent. Finally, the EFT performance of the field-dependent and field-independent subjects again displayed a significant interaction with the size of the embedded figures. A parallel-serial processingstyles hypothesis is proposed to account for the EFT data.     

The nature and severity of cognitive impairment associated with adjuvant chemotherapy in women with breast cancer: a meta-analysis of the current literature

OBJECTIVE: Several studies have identified that adjuvant chemotherapy for breast cancer is associated with cognitive impairment; however, the magnitude of this impairment is unclear. This study assessed the severity and nature of cognitive impairment associated with adjuvant chemotherapy by conducting a meta-analysis of the published literature to date. METHOD: Six studies (five cross-sectional and one prospective) meeting the inclusion criteria provided a total of 208 breast cancer patients who had undergone adjuvant chemotherapy, 122 control participants and 122 effect sizes (Cohen's d) falling into six cognitive domains. First, the mean of all the effect sizes within each cognitive domain was calculated (separately for cross-sectional and prospective studies); second, a mean effect size was calculated for all of the effect sizes in each cross-sectional study; and third, regression analyses were conducted to determine any relationships between effect size for each study and four different variables. RESULTS: For the cross-sectional studies, each of the cognitive domains assessed (besides attention) showed small to moderate effect sizes (-0.18 to -0.51). The effect sizes for each study were small to moderate (-0.07 to -0.50) and regression analysis detected a significant negative logarithmic relationship (R2 = .63) between study effect size and the time since last receiving chemotherapy. For the prospective study, effect sizes ranged from small to large (0.11-1.09) and indicated improvements in cognitive function from the beginning of chemotherapy treatment to 3 weeks and even 1 year following treatment. CONCLUSION: This meta-analysis suggests that cognitive impairment occurs reliably in women who have undergone adjuvant chemotherapy for breast cancer but that the magnitude of this impairment depends on the type of design that was used (i.e., cross-sectional or prospective). Thus, more prospective studies are required before definite conclusions about the effects of adjuvant chemotherapy on cognition can be made.     

Decision rules for selecting effect sizes in meta-analysis: a review and reanalysis of psychotherapy outcome studies

This study deals with some of the judgmental factors involved in selecting effect sizes from within the studies that enter a meta-analysis. Particular attention is paid to the conceptual redundancy rule that Smith, Glass, and Miller (1980) used in their study of the effectiveness of psychotherapy for deciding which effect sizes should and should not be counted in determining an overall effect size. Data from a random sample of 25 studies from Smith et al.'s (1980) population of psychotherapy outcome studies were first recoded and then reanalyzed meta-analytically. Using the conceptual redundancy rule, three coders independently coded effect sizes and identified more than twice as many of them per study as did Smith et al. Moreover, the treatment effect estimates associated with this larger sample of effects ranged between .30 and .50, about half the size claimed by Smith et al. Analyses of other rules for selecting effect sizes showed that average effect estimates also varied with these rules. Such results indicate that the average effect estimates derived from meta-analyses may depend heavily on judgmental factors that enter into how effect sizes are selected within each of the individual studies considered relevant to a meta-analysis.     

Test-retest reliability and standard error of measurement for the test of variables of attention (T.O.V.A.) with healthy school-age children

Test-retest reliability of the Test of Variables of Attention (T.O.V.A.) was investigated in two studies using two different time intervals: 90 min and 1 week (2 days). To investigate the 90-min reliability, 31 school-age children (M = 10 years, SD = 2.66) were administered the T.O.V.A. then read ministered the test 90 min afterward. Significant reliability coefficients were obtained across omission (.70), commission (.78), response time (.84), and response time variability (.87). For the second study, a different sample of 33 school-age children (M = 10.01 years, SD = 2.59) were administered the test then read ministered the test 1 week later. Significant reliability coefficients were obtained for omission (.86), commission (.74), response time (.79), and response time variability (.87). Standard error of measurement statistics were calculated using the obtained coefficients. Commission scores were significantly higher on second trials for each retest interval.