Reward, punishment, and cooperation: a meta-analysis

How effective are rewards (for cooperation) and punishment (for noncooperation) as tools to promote cooperation in social dilemmas or situations when immediate self-interest and longer term collective interest conflict? What variables can promote the impact of these incentives? Although such questions have been examined, social and behavioral scientists provide different answers. To date, there is no theoretical and/or quantitative review of rewards and punishments as incentives for cooperation in social dilemmas. Using a novel interdependence-theoretic framework, we propose that rewards and punishments should both promote cooperation, and we identify 2 variables—cost of incentives and source of incentives—that are predicted to magnify the effectiveness of these incentives in promoting cooperation.A meta-analysis involving 187 effect sizes revealed that rewards and punishments exhibited a statistically equivalent positive effect on cooperation (d =0.51 and 0.70, respectively). The effectiveness of incentives was stronger when the incentives were costly to administer, compared to free. Centralization of incentives did not moderate the effect size. Punishments were also more effective during iterated dilemmas when participants continued to interact in the same group, compared to both (a) iterated dilemmas with reassignment to a new group after each trial and (b) one-shot dilemmas. We also examine several other potential moderators, such as iterations, partner matching, group size, country, and participant payment. We discuss broad conclusions, consider implications for theory, and suggest directions for future research on rewards and punishment in social dilemmas.     

The statistical recommendations of the American Psychological Association Publication Manual: Effect sizes,confidence intervals,and meta‐analysis

Estimation based on effect sizes, confidence intervals, and meta‐analysis usually provides a more informative analysis of empirical results than does statistical significance testing, which has long been the conventional choice in psychology. The sixth edition of the American Psychological Association Publication Manual now recommends that psychologists should, wherever possible, use estimation and base their interpretation of research results on point and interval estimates. We outline the Manual's recommendations and suggest how they can be put into practice: adopt an estimation framework, starting with the formulation of research aims as ‘How much?’ or ‘To what extent?’ questions. Calculate from your data effect size estimates and confidence intervals to answer those questions, then interpret. Wherever appropriate, use meta‐analysis to integrate evidence over studies. The Manual's recommendations can assist psychologists improve they way they do their statistics and help build a more quantitative and cumulative discipline.     

Effect Size Strengths in Subjective Well-Being Research

Subjective well-being (SWB) research is characterized by many large samples, which often results in virtually all variables being significantly related to well-being, even if the associations are small. In this article we explore the strengths of associations between various predictors and SWB outcomes. In addition to standard effect-size statistics, we also examined the range of the SWB scale covered in the distribution of the predictor, allowing us to estimate the strength of influence of each variable, independent of variability in the sample. We analyzed just a few variables to illustrate what our approach reveals. Our analyses included a representative sample of both the world and the United States, and our data included three types of SWB (life satisfaction (LS), positive affect (PA), and negative affect (NA)). The largest effect sizes emerged for societal characteristics, such as between-nations differences, as well as personal characteristics, such as perceived social support. Small or very small effect sizes were consistently found for demographic characteristics, such as sex, age, and marital status. Other effect sizes varied by the type of SWB being considered. For example, income resulted in a large effect size for LS, but small to medium effect sizes for PA and NA. We suggest that when scholars report and interpret the associations of predictor variables with SWB, they consider the strengths of their significant associations.

     

Considering the opposite: a corrective strategy for social judgment

It is proposed that several biases in social judgment result from a failure--first noted by Francis Bacon--to consider possibilities at odds with beliefs and perceptions of the moment. Individuals who are induced to consider the opposite, therefore, should display less bias in social judgment. In two separate but conceptually parallel experiments, this reasoning was applied to two domains--biased assimilation of new evidence on social issues and biased hypothesis testing of personality impressions. Subjects were induced to consider the opposite in two ways: through explicit instructions to do so and through stimulus materials that made opposite possibilities more salient. In both experiments the induction of a consider-the-opposite strategy had greater corrective effect than more demand-laden alternative instructions to be as fair and unbiased as possible. The results are viewed as consistent with previous research on perseverance, hindsight, and logical problem solving, and are thought to suggest an effective method of retraining social judgment.     

Out with .05, in with Replication and Measurement: Isolating and Working with the Particular Effect Sizes that are Troublesome for Inferential Statistics

It is difficult to obtain adequate power to test a small effect size with a set criterion alpha of 0.05. Probably an inferential test will indicate non-statistical significance and not be published. Rarely, statistical significance will be obtained, and an exaggerated effect size calculated and reported. Accepting all inferential probabilities and associated effect sizes could solve exaggeration problems. Graphs, generated through Monte Carlo methods, are presented to illustrate this. The first graph presents effect sizes (Cohen's d) as lines from 1 to 0 with probabilities on the Y axis and the number of measures on the X axis. This graph shows effect sizes of .5 or less should yield non-significance with sample sizes below 120 measures. The other graphs show results with as many as 10 small sample size replications. There is a convergence of means with the effect size as sample size increases and measurement accuracy emerges.     

Are effect sizes in self-efficacy field changing over time? A meta–meta analysis

Theories relating to self-efficacy have developed rapidly since Bandura first proposed the concept in 1977. In the past two decades, psychologists have carried out numerous studies to research the cultural and psychological changes in social development. The research topic of this study is whether self-efficacy changes over time. This study uses a meta–meta analysis and includes 13 meta-analyses, including 536 effect sizes, with a total sample size of 421,880. We find that individual self-efficacy increases over time, which may be related to social development trends. However, the effects of interventions on self-efficacy remain similar (Q_model = 1.807, df = 1, p > .05), and a possible explanation is that time effects of self-efficacy confuse the effects of intervention, because both in the intervention group and control group, the average of self-efficacy increases over time. And we find that a general decline in the predictive effects of self-efficacy (Q_model = 5.117, df = 1, p = .024), especially the ability to predict relatively objective variables (e.g. job performance, teaching effectiveness, and transfer of training). A possible explanation is that as social development people tend to overestimate their self-efficacy. Another possible explanation is that the effect sizes in the original studies being overrated, may due to intentional selective reporting or unintentional statistical errors.     

Reproducible research in sport and exercise psychology: The role of sample sizes

ObjectivesWe aim to introduce the discussion on the crisis of confidence to sport and exercise psychology. We focus on an important aspect of this debate, the impact of sample sizes, by assessing sample sizes within sport and exercise psychology. Researchers have argued that publications in psychological research contain numerous false-positive findings and inflated effect sizes due to small sample sizes.MethodWe analyse the four leading journals in sport and exercise psychology regarding sample sizes of all quantitative studies published in these journals between 2009 and 2013. Subsequently, we conduct power analyses.ResultsA substantial proportion of published studies does not have sufficient power to detect effect sizes typical for psychological research. Sample sizes and power vary between research designs. Although many correlational studies have adequate sample sizes, experimental studies are often underpowered to detect small-to-medium effects.ConclusionsAs sample sizes are small, research in sport and exercise psychology may suffer from false-positive results and inflated effect sizes, while at the same time failing to detect meaningful small effects. Larger sample sizes are warranted, particularly in experimental studies.     

On effect size

The call for researchers to report and interpret effect sizes and their corresponding confidence intervals has never been stronger. However, there is confusion in the literature on the definition of effect size, and consequently the term is used inconsistently. We propose a definition for effect size, discuss 3 facets of effect size (dimension, measure/index, and value), outline 10 corollaries that follow from our definition, and review ideal qualities of effect sizes. Our definition of effect size is general and subsumes many existing definitions of effect size. We define effect size as a quantitative reflection of the magnitude of some phenomenon that is used for the purpose of addressing a question of interest. Our definition of effect size is purposely more inclusive than the way many have defined and conceptualized effect size, and it is unique with regard to linking effect size to a question of interest. Additionally, we review some important developments in the effect size literature and discuss the importance of accompanying an effect size with an interval estimate that acknowledges the uncertainty with which the population value of the effect size has been estimated. We hope that this article will facilitate discussion and improve the practice of reporting and interpreting effect sizes.     

Practical Guide for Reporting Effect Size in Quantitative Research in the Journal of Counseling & Development

The purpose of this article is to assist researchers in meeting the requirement of reporting effect sizes in quantitative research studies submitted to the Journal of Counseling & Development. This requirement is detailed in the “Guidelines for Authors” included in this issue. The authors provide practical information on generating, reporting, and interpreting effect size estimates for various types of statistical analyses. Information is provided on the meaning of effect sizes within the larger knowledge base.     

A socioecological perspective on primate cognition,past and present

The papers in this special issue examine the relationship between social and ecological cognition in primates. We refer to the intersection of these two domains as socioecological cognition. Examples of socioecological cognition include socially learned predator alarm calls and socially sensitive foraging decisions. In this review we consider how primate cognition may have been shaped by the interaction of social and ecological influences in their evolutionary history. The ability to remember distant, out-of-sight locations is an ancient one, shared by many mammals and widespread among primates. It seems some monkeys and apes have evolved the ability to form more complex representations of resources, integrating “what-where-how much” information. This ability allowed anthropoids to live in larger, more cohesive groups by minimizing competition for limited resources between group members. As group size increased, however, competition for resources also increased, selecting for enhanced social skills. Enhanced social skills in turn made a more sophisticated relationship to the environment possible. The interaction of social and ecological influences created a spiraling effect in the evolution of primate intelligence. In contrast, lemurs may not have evolved the ability to form complex representations which would allow them to consider the size and location of resources. This lack in lemur ecological cognition may restrict the size of frugivorous lemur social groups, thereby limiting the complexity of lemur social life. In this special issue, we have brought together two review papers, five field studies, and one laboratory study to investigate the interaction of social and ecological factors in relation to foraging. Our goal is to stimulate research that considers social and ecological factors acting together on cognitive evolution, rather than in isolation. Cross fertilization of experimental and observational studies from captivity and the field is important for increasing our understanding of this relationship. This contribution is part of the Special Issue “A Socioecological Perspective on Primate Cognition”.     

Preventing prejudice and improving intergroup attitudes: A meta-analysis of child and adolescent training programs

This article reports the results of a meta-analysis of 81 research reports containing 122 intervention–control comparisons of structured programs to reduce prejudice or promote positive intergroup attitudes in children and adolescents. Overall, the analysis revealed a mean effect size of d = 0.30, indicating low to moderate intervention effects. From the great variety of different approaches, interventions based on direct contact experiences along with social-cognitive training programs designed to promote empathy and perspective taking showed the strongest effect sizes. In addition, effects varied according to the program participant's social status (higher effects for majority groups), the target out-group (lower effect sizes for ethnic vs. disabled and aged out-groups), and the type of outcome assessment (higher effects for cognitive vs. affective and behavioral measures of intergroup attitudes). The discussion considers several limitations including the lack of implementation and follow-up research as well as future direction of research on promoting intergroup relations.     

GENDER COMPARISONS OF MATHEMATICS ATTITUDES AND AFFECT

This article reports the complex results of meta-analyses of gender differences in attitudes and affect specific to mathematics. Overall, effect sizes were small and were similar in size to gender differences in mathematics performance. When differences exist, the pattern is for females to hold more negative attitudes. Gender differences in self-confidence and general mathematics attitudes are larger among high school and college students than among younger students. Effect sizes for mathematics anxiety differ depending upon the sample (highly selected or general). One exception to the general pattern is in stereotyping mathematics as a male domain, where males hold much more stereotyped attitudes ( d = -.90). While affect and attitudes toward mathematics are not the only influences on the development of gender differences in mathematics performance, they are important, and both male and female affect and attitudes should be considered in conjunction with other social and political influences as explanations.     

On sample size calculation for 2×2 fixed‐effect ANOVA when variances are unknown and possibly unequal

The factorial 2 × 2 fixed‐effect ANOVA is a procedure used frequently in scientific research to test mean differences between‐subjects in all of the groups. But if the assumption of homogeneity is violated, the test for the row, column, and the interaction effect might be invalid or less powerful. Therefore, for planning research in the case of unknown and possibly unequal variances, it is worth developing a sample size formula to obtain the desired power. This article suggests a simple formula to determine the sample size for 2 × 2 fixed‐effect ANOVA for heterogeneous variances across groups. We use the approximate Welch t test and consider the variance ratio to derive the formula. The sample size determination requires two‐step iterations but the approximate sample sizes needed for the main effect and the interaction effect can be determined separately with the specified power. The present study also provides an example and a SAS program to facilitate the calculation process.     

Decision rules for selecting effect sizes in meta-analysis: a review and reanalysis of psychotherapy outcome studies

This study deals with some of the judgmental factors involved in selecting effect sizes from within the studies that enter a meta-analysis. Particular attention is paid to the conceptual redundancy rule that Smith, Glass, and Miller (1980) used in their study of the effectiveness of psychotherapy for deciding which effect sizes should and should not be counted in determining an overall effect size. Data from a random sample of 25 studies from Smith et al.'s (1980) population of psychotherapy outcome studies were first recoded and then reanalyzed meta-analytically. Using the conceptual redundancy rule, three coders independently coded effect sizes and identified more than twice as many of them per study as did Smith et al. Moreover, the treatment effect estimates associated with this larger sample of effects ranged between .30 and .50, about half the size claimed by Smith et al. Analyses of other rules for selecting effect sizes showed that average effect estimates also varied with these rules. Such results indicate that the average effect estimates derived from meta-analyses may depend heavily on judgmental factors that enter into how effect sizes are selected within each of the individual studies considered relevant to a meta-analysis.     

The bias in the bias: Comparative optimism as a function of event social undesirability

We present a new event-level predictor of comparative optimism: comparative optimism is larger for more socially undesirable events. A meta-analysis shows that event social undesirability predicts comparative optimism effect sizes reported in the literature, over and above the effects of other known predictors. Four experiments corroborate this finding and demonstrate the key role played by respondents’ impression management motives. The effect of social undesirability decreases with stronger than usual anonymity assurances, increases with greater impression management tendencies, and reverses when people want to make a negative impression. Because social undesirability is correlated to other known predictors of comparative optimism (e.g., controllability, severity), it is important to take its effects into account when assessing the effect of other event characteristics. The current research adds to, and bridges, the literatures on event-level predictors and impression management in comparative optimism.     

Effectiveness of an empirically supported treatment for social phobia in the field

This study examined the effectiveness of individual exposure combined with cognitive restructuring for social phobia in a clinical setting as well as the influence of sample restriction criteria on the effect size. Participants were 217 unselected patients with a primary diagnosis of social phobia who were treated by 57 therapists in four outpatient clinics of the Christoph-Dornier-Foundation of Clinical Psychology in Germany. Results 6 weeks after the end of therapy showed highly significant reductions in social phobic fears and avoidance as well as in general anxiety and symptoms of depression. However, patients who dropped out during therapy reported a significantly higher degree of depression. Results did not differ between the four outpatient clinics and are comparable with the average effect-sizes reported by meta-analytic studies of controlled efficacy research, using selected patients. Also, restricting the sample according to the selection criteria often applied in research settings did not result in higher effect sizes for the applied outcome measures. We conclude that individual cognitive behavioural therapy for social phobia can be transported from research settings to the field of mental health.     

Expertise effects on decision-making in sport are constrained by requisite response behaviours – A meta-analysis

ObjectivesA quantitative review of the effects of requisite responses and methods of stimulus presentation for assessing decision-making expertise in sport was undertaken.DesignAn electronic literature search was conducted in the online databases: SPORTDiscus with Full Text and ISI Web Knowledge All Databases. Articles for analysis were selected according to prior defined criteria.MethodsWe considered 111 effect sizes in studies involving 882 expert and non-expert participants. Effect sizes were calculated for six common protocols for measures responses: verbalized knowledge, eye movement measures, decision time, response accuracy, movement accuracy, and movement time. Two moderator variables were also considered to assess effects of research protocols on the dependent variables: “the requisite response” and “stimulus presentation”. A random effect model was used to calculate effect sizes.ResultsAnalysis of moderator variables suggested that expertise effects were more apparent for “requisite responses” when participants were required to actually perform sporting actions and for “stimulus presentation” under in situ task constraints than for other conditions.ConclusionsFuture empirical work on expertise and decision-making needs to consider task representativeness in considering requisite responses of participants in simulating performance environment conditions. Use of representative task constraints with performers required to perform sport actions in in situ conditions appeared the most functional empirical protocols to enhance validity of data.     

Child social skills training in developmental crime prevention: effects on antisocial behavior and social competence

Social skills training for children is becoming increasingly popular as a measure for developmental crime prevention. Although previous reviews of such programs have shown positive effects, they have also revealed problems of research design, outcome measures, and long-term follow up. Accordingly, this article reports on a recent meta-analysis of randomized evaluations of the effect of social skills training in preventing antisocial behavior and promoting social competence. Of 841 retrievable references, 84 research reports with a total of 136 treatment-control comparisons fulfilled the eligibility criteria. Results showed a small but significant overall positive effect of d = .39 at post-intervention and d = .28 at follow-up (3 months and later). Effect sizes were somewhat greater for outcome measures of social competence than for measures of antisocial behavior, particularly when delinquency was assessed. Cognitive-behavioral programs revealed the best results in terms of generalization over time and on outcome criteria. In addition, prevention measures indicated for children and adolescents who already manifested some behavioral problems had higher effect sizes than universal approaches. Because most studies dealt with small sample sizes, non-official outcome data, and measurements after less than one year, the results should be interpreted with caution. Further high-quality studies with long-term empirical outcome criteria are needed, particularly outside the United States.     

Owning leads to valuing: Meta-analysis of the mere ownership effect

Mere ownership effect is the phenomenon that people tend to value what they own more than what they do not own. This classic effect is considered robust, yet effect sizes vary across studies, and the effect is often confused for or confounded with other classic phenomena, such as endowment or mere exposure effects. We conducted a pre-registered meta-analysis of 26 samples published before 2019 (N = 3024), which resulted in psychological ownership on valuing effect of g ∼ 0.57 [0.46, 0.69]. Suggestive moderator analyses supported the use of replica as the strongest moderators. Mere ownership effects were different from the null across all moderator categories and in most publication bias adjustments. We consider this as suggestive evidence that psychological owning leads to valuing, yet caution that much more research is needed. All materials, data, and codes are available on https://osf.io/fdyqw/ .     

Ist psychodynamische Kurztherapie eine wirksame Behandlungsform bei psychischen Störungen?

To test the efficacy of short-term psychodynamic psychotherapy (STPP) in specific psychiatric disorders, a meta-analysis of more recent studies was performed. Its importance is grounded on the inconsistent results of previous meta-analyses regarding short-term psychodynamic psychotherapy (STPP) and the controversial discussion regarding the efficacy of psychodynamic therapy in general. In a computer-based search, studies of STPP published between 1970 and 2004 were identified. Studies of interpersonal therapy were excluded and rigorous inclusion criteria were applied. Seventeen studies fulfilled these criteria. Effect sizes were calculated for target problems, general psychiatric symptoms, and social functioning. In order to examine the stability of outcome, effect sizes were assessed separately for end of therapy and for follow-up assessment. The effect sizes of STPP were compared to those of waiting list control patients, treatments as usual, and of other forms of psychotherapy. STPP yielded significant and large pre-post effect sizes, which were not only stable, but tended to increase at follow-up. The effect sizes of STPP significantly exceeded those of waiting list and treatments as usual, but no differences were found between STPP and other forms of psychotherapy. STPP proved to be an effective treatment in different psychiatric disorders. However, further research of STPP in specific psychiatric disorders is needed, including a study of the active ingredients of STPP. Effectiveness studies should be included.