HARKing's Threat to Organizational Research: Evidence From Primary and Meta‐Analytic Sources

We assessed presumed consequences of hypothesizing after results are known (HARKing) by contrasting hypothesized versus nonhypothesized effect sizes among 10 common relations in organizational behavior, human resource management, and industrial and organizational psychology research. In Study 1, we analyzed 247 correlations representing 9 relations with individual performance in 136 articles published in Journal of Applied Psychology and Personnel Psychology and provide evidence that correlations are significantly larger when hypothesized compared to nonhypothesized. In Study 2, we analyzed 281 effect sizes from a meta‐analysis on the job satisfaction–job performance relation and provide evidence that correlations are significantly larger when hypothesized compared to nonhypothesized. In addition, in Study 2, we documented that hypothesized variable pairs are more likely to be mentioned in article titles or abstracts. We also ruled out 13 alternative explanations to the presumed HARKing effect pertaining to methodological (e.g., unreliability, publication year, research setting, research design, measure contextualization, publication source) and substantive (e.g., predictor–performance pair, performance measure, satisfaction measure, occupation, job/task complexity) issues. Our results suggest that HARKing seems to pose a threat to research results, substantive conclusions, and practical applications. We offer recommended solutions to the HARKing threat.     

Meta‐Analysis: A Systematic Method for Synthesizing Counseling Research

The authors provide a template for counseling researchers who are interested in quantitatively aggregating research findings. Meta‐analytic studies can provide relevant information to the counseling field by systematically synthesizing studies performed by researchers from diverse fields. Methodologically sound meta‐analyses require careful planning, diligent literature searches, detailed coding of study information, and knowledge of meta‐analytic approaches to statistical analyses. The authors provide steps to guide counseling researchers in conducting meta‐analytic reviews.     

The Nature of Psychological Reactance Revisited: A Meta‐Analytic Review

Psychological reactance ( Brehm, 1966 ; Brehm & Brehm, 1981 ) has been a long‐standing topic of interest among scholars studying the design and effects of persuasive messages and campaigns. Yet, until recently, reactance was considered to be a motivational state that could not be measured. Dillard and Shen (2005) argued that reactance can be conceptualized as cognition and affect and made amenable to direct measurement. This article revisits Dillard and Shen's (2005) questions about the nature of psychological reactance and reports a test designed to identify the best fitting model of reactance. A meta‐analytic review of reactance research was conducted (K = 20, N = 4,942) and the results were used to test path models representing competing conceptualizations of reactance. The results offer evidence that the intertwined model—in which reactance is modeled as a latent factor with anger and counterarguments serving as indicators—best fit the data.     

Cohen's d Corrected for Case IV Range Restriction: A More Accurate Procedure for Evaluating Subgroup Differences in Organizational Research

Organizational and staffing researchers are often interested in evaluating whether subgroup differences exist (e.g., between Caucasian and African‐American individuals) on predictors of job performance. To investigate subgroup differences, researchers often will collect data from current employees to make inferences about subgroup differences among job applicants. However, the magnitude of subgroup differences (i.e., Cohen's d) within incumbent samples may be different (i.e., smaller) than the magnitude of subgroup differences in applicant samples because selection of applicants typically reduces the variance of scores on the predictors (i.e., because lower scoring applicants are not selected). If researchers seek to generalize a d value in an incumbent sample to the applicant population, they may use Bobko, Roth, and Bobko's (correcting the effect size of d for range restriction and unreliability, 2001) Case II or III correction. By extension, Hunter, Schmidt, and Le (implications of direct and indirect range restriction for meta‐analysis methods and findings, 2006) have proposed a Case IV correction, which is more realistic than Bobko et al.'s approach. Therefore, this paper develops a Case IV correction for d (i.e., d_c4). The simulation results showed that the d_c4 was generally accurate across 6,000 simulation conditions. Moreover, 2 published datasets were reanalyzed to show the influence of the Case IV correction on d. In addition, implications and future directions of the d_c4 are discussed.     

The Longevity Effects of Religious and Nonreligious Participation: A Meta‐Analysis and Meta‐Regression

Scholars have long argued that the reduced mortality risk associated with frequent participation in religious services derives from two sources: social participation and religious belief efficacy. In contrast, the reduced mortality risk associated with participation in nonreligious groups is thought to derive solely from the social participation component. This study tests the religious efficacy hypothesis by comparing the effects of religious participation with nonreligious participation using meta‐analyses of 312 mortality risk estimates from 74 publications (providing data on more than 300,000 persons). We found no significant difference between the mean hazard ratio (HR) for low religious participation (HR, 1.32; 95% CI, 1.24–1.41) and the mean HR for low nonreligious participation (HR, 1.25; 95% CI, 1.17–1.33). These findings suggest that the positive health effects of religious participation may largely be attributed to the social participation component, rather than to the religious component of the act.     

Prevalence of Nonsuicidal Self‐Injury in Nonclinical Samples: Systematic Review,Meta‐Analysis and Meta‐Regression

Published prevalence estimates of nonsuicidal self‐injury (NSSI) among nonclinical samples are highly heterogeneous, raising concerns about their reliability and hindering attempts to explore the alleged increase in NSSI over time. Accordingly, the objectives of this study were to investigate the influence of methodological factors on heterogeneity in NSSI prevalence estimates, explore changes over time, and estimate overall international NSSI prevalence. Results showed that methodological factors contributed over half (51.6%) of the heterogeneity in prevalence estimates, and, after adjusting for these factors, NSSI prevalence did not increase over time. Overall, pooled NSSI prevalence was 17.2% among adolescents, 13.4% among young adults, and 5.5% among adults. Clearly, development of standardized methodology in NSSI research is crucial if accurate estimates are desired.     

Pornography Consumption and Satisfaction: A Meta‐Analysis

A classic question in the communication literature is whether pornography consumption affects consumers' satisfaction. The present paper represents the first attempt to address this question via meta‐analysis. Fifty studies collectively including more than 50,000 participants from 10 countries were located across the interpersonal domains of sexual and relational satisfaction and the intrapersonal domains of body and self satisfaction. Pornography consumption was not related to the intrapersonal satisfaction outcomes that were studied. However, pornography consumption was associated with lower interpersonal satisfaction outcomes in cross‐sectional surveys, longitudinal surveys, and experiments. Associations between pornography consumption and reduced interpersonal satisfaction outcomes were not moderated by their year of release or their publication status. But analyses by sex indicted significant results for men only.     

Learning by Preparing‐to‐Teach and Teaching: A Meta‐Analysis

The present research meta‐analytically examined the effects of preparing‐to‐teach and subsequent teaching (relative to mere studying without teaching expectancy) on the acquisition of domain knowledge. The synthesis of 28 studies indicated that the estimated effect sizes (Hedges' gs) were 0.35 for preparing‐to‐teach and 0.56 for teaching with preparing‐to‐teach. Both preparing‐to‐teach and teaching with preparing‐to‐teach were effective in promoting deep learning (as well as surface learning) and even after a delay. The learning benefits of teaching with preparing‐to‐teach were larger when students expected and engaged in an interactive teaching activity than when they expected and engaged in a non‐interactive teaching activity. The mere expectation of interactive teaching also produced larger learning effects than the expectation of non‐interactive teaching. These results suggest that preparing‐to‐teach and teaching, interactive teaching after preparing to do so in particular, are useful for enhancing learning.     

Intervention Studies on Forgiveness: A Meta‐Analysis

In this meta‐analysis, 9 published studies (N = 330) that investigated the efficacy of forgiveness interventions within counseling were examined. After a review of theories of forgiveness, it was discovered that the studies could logically be grouped into 3 categories: decision‐based, process‐based group, and process‐based individual interventions. When compared with control groups, for measures of forgiveness and other emotional health measures, the decision‐based interventions showed no effect, the process‐based group interventions showed significant effects, and the process‐based individual interventions showed large effects. Consequently, effectiveness has been shown for use of forgiveness in clinical and other settings.     

Counseling Outcomes for Youth With Oppositional Behavior: A Meta‐Analysis

This meta‐analysis of clinical trials explored the effectiveness of treatments targeted at changing oppositional behavior. The meta‐analysis included 31 peer‐reviewed treatment studies with school‐age youth ages 6–17 years identified as having oppositional defiant disorder. Results were synthesized using a random effects model for mean difference and mean gain effect size estimates. Findings suggest that counseling/psychotherapy is effective in treating oppositional behavior in school‐age youth at termination, but the long‐term efficacy of such treatment is less stable.     

The Home Field Advantage in Athletics: A Meta‐Analysis

This meta‐analysis examined the home‐field advantage in athletics, with an emphasis on potential moderators. The goal of this research was to quantify the probability of a home victory, thus only studies that included win–loss data were included in the meta‐analysis. A significant advantage for home teams was observed across all conditions (M_π= .604); and time era, season length, game type, and sport moderated the effect. Furthermore, it was found that season length mediated the effect of sport such that differences between sports could be attributed to some sports having longer seasons than other sports. This research has implications for athletes, fans, and the media alike.     

The Persuasive Effects of Metaphor: A Meta‐Analysis

Empirical investigations of metaphor's persuasive effects have produced mixed results. In an effort to integrate the literature, we present a review and meta‐analytic summary of existing studies. Six explanations for the potential suasory advantage of metaphor over literal language were reviewed: (a) pleasure or relief, (b) communicator credibility, (c) reduced counterarguments, (d) resource‐matching, (e) stimulated elaboration, and (f) superior organization. Next, a meta‐analysis was conducted and the impact of seven moderator variables was tested. The overall effect for the metaphor‐literal comparison for attitude change was r = .07, which supported the claim that metaphors enhance persuasion. The effect rose to r = .42 under optimal conditions, when a single, nonextended metaphor was novel, had a familiar target, and was used early in a message. Metaphor appeared to exert a small effect on perceptions of source dynamism (r = .06), but showed no demonstrable impact on competence (r =?.01) or character (r =?.02). Of the six theories considered, the superior organization explanation of metaphor's persuasive impact was most supported by the results.     

The Space–Time Congruency Effect: A Meta‐Analysis

Several reaction time (RT) studies report faster responses when responses to temporal information are arranged in a spatially congruent manner than when this arrangement is incongruent. The resulting space–time congruency effect is commonly attributed to a culturally salient localization of temporal information along a mental timeline (e.g., a mental timeline that runs from left to right). The present study aims to provide a compilation of the published RT studies on this time–space association in order to estimate the size of its effect and the extent of potential publication bias in this field of research. In this meta‐analysis, three types of task are distinguished due to hitherto existing empirical findings. These findings suggest that the extent to which time is made relevant to the experimental task has a systematic impact on whether or not the mental timeline is activated. The results of this meta‐analysis corroborate these considerations: First, experiments that make time a task‐relevant dimension have a mean effect size of d = 0.46. Second, in experiments in which time is task irrelevant, the effect size does not significantly deviate from zero. Third, temporal priming studies have a surprisingly high mean effect size of d = 0.47, which, however, should be adjusted to d = 0.36 due to publication bias.     

Therapists' Integration of Religion and Spirituality in Counseling: A Meta‐Analysis

The authors conducted a 26‐study meta‐analysis of 5,759 therapists and their integration of religion and spirituality in counseling. Most therapists consider spirituality relevant to their lives but rarely engage in spiritual practices or participate in organized religion. Marriage and family therapists consider spirituality more relevant and participate In organized religion to a greater degree than therapists from other professions. Across professions, most therapists surveyed (over 80%) rarely discuss spiritual or religious issues in training. In mixed samples of religious and secular therapists, therapists' religious faith was associated with using religious and spiritual techniques in counseling frequently, willingness to discuss religion in therapy, and theoretical orientation.     

Meta‐stereotypes,Social Image and Help Seeking: Dependency‐Related Meta‐stereotypes Reduce Help‐Seeking Behaviour

People who need help can be reluctant to seek it. This can be due to social image concerns. Here, we investigate if these concerns may be prompted by a salient negative meta‐stereotype: the belief that one's group is judged negatively by another group. Specifically, we researched group members' help‐seeking behaviour in the context of a dependency‐related meta‐stereotype. In a two‐condition study (N = 45), we manipulated participants' belief that their national group was judged dependent by a significant out‐group. We then examined their subsequent help‐seeking behaviour on a real‐world task. Participants whose social identity as a group member was salient showed greater reluctance to seek help when the meta‐stereotype was made prominent compared with when it was not. This suggests that, in a context where social image and social identity concerns are relevant, group members are willing to sacrifice the possibility of accessing needed help in order to avoid confirming a negative stereotype of their group. The implications of these results for helping transactions and community development are discussed. Copyright © 2012 John Wiley & Sons, Ltd.     

The Vietnam Veteran Revisited: Implications for Counseling

Vietnam veterans are plagued by an emotional sense of having failed at what society once considered an important job; careful attention, therefore, must be paid to the type of counseling that counselors give them. This article discusses common concerns among the Vietnam veterans and offers suggestions for initiating programs of service.     

Cumulating Evidence about the Social Animal: Meta‐Analysis in Social‐Personality Psychology

Like most scientific fields, social‐personality psychology has experienced an explosion of research related to such central topics as aggression, attraction, gender, group processes, motivation, personality, and persuasion, to name a few. The proliferation of research can be a monster unless it is tamed with the scientific review strategy of meta‐analysis, literally analyses of past analyses that produce a quantitative and empirical history of research on a particular phenomenon. The purpose of this article is to outline the basic process and statistics of meta‐analysis, as they pertain to social‐personality psychology. Meta‐analysis involves: (i) defining the problem under review; (ii) gathering qualified reports and putting their findings and methods into a database, (iii) analyzing that database, and (iv) interpreting the results and reporting them. Use of meta‐analytic strategies has paralleled the knowledge explosion in social‐personality psychology, but must be used and consumed with careful discernment if the cumulated evidence about the social animal, Homo sapiens, is to have maximal value.     

Gender Differences in Loneliness Across the Lifespan: A Meta‐Analysis

Different theoretical contentions on gender differences in loneliness exist, often including the emergence of gender differences in particular developmental periods. To explain those ideas, the current meta‐analysis synthesizes the available evidence on gender differences in loneliness across the lifespan. Three‐level meta‐analyses were conducted with 751 effect sizes, covering 399,798 individuals (45.56% males). Results showed a close‐to‐zero overall effect (g = 0.07). Most examined moderators were non‐significant, except for age, the scope of the sampling area, and year of publication. Most importantly, all effects were small, suggesting that across the lifespan mean levels of loneliness are similar for males and females.